- Email: [email protected]
Using control charts to monitor financial reporting of public companies
International Journal of Accounting Information Systems 5 (2004) 109 – 127
Using control charts to monitor financial reporting of public companies Richard B. Dull a,*, David P. Tegarden b a
School of Accountancy and Legal Studies, Clemson University, 309 Sirrine Hall, Clemson, SC 29634, USA b Pamplin College of Business, Virginia Tech, USA Received 31 March 2003; received in revised form 30 November 2003; accepted 1 January 2004 Available online
Abstract There is currently much being written about increasing the frequency and timeliness of financial reporting. Comments frequently support the desirability and question the feasibility of continuous reporting. Under the current reporting/assurance paradigm, if companies report continuously, auditors must monitor and audit on a continuous basis. Current audit standards generally address the outcomes, rather than the detailed procedures that the auditor must follow to meet the objectives of the audit. The purpose of this paper is to propose and demonstrate a technique for monitoring continuous financial information using control charts of accounting information. In this study, financial data were collected and used to implement control charts. Part of the selected companies had known errors, while others had no known errors. The resulting control charts and common interpretation rules identified potential systematic problems only in the companies with known errors. The authors suggest that when continuous data become available, charts similar with these can be used in conjunction with statistical and analytical techniques to detect signals that financial processes are not in control. Refinements of this technique should assist those internal and external to organizations, who are concerned with the reliability of information produced and reported by the organization. D 2004 Elsevier Inc. All rights reserved. Keywords: Control charts; Financial reporting; Public companies
1. Introduction Over the past two decades, the proliferation of technology has led to the increased availability of financial information. Although the timing of financial reporting has not * Corresponding author. Tel.: +1-864-656-0610; fax: +1-864-656-4892. E-mail address: [email protected] (R.B. Dull). 1467-0895/$ - see front matter D 2004 Elsevier Inc. All rights reserved. doi:10.1016/j.accinf.2004.01.004
110
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
changed significantly, currently, there is much being written about increasing the frequency and timeliness of financial reporting and assurance (Vasarhelyi, 2002). Technology advances have generally increased system capabilities, enabling integrated packages to simplify and accelerate the mechanical process of financial reporting. Generally, people agree that increasing the frequency may be useful but have reservations about the feasibility of the implementation of continuous reporting and assurance. For example, Searcy et al. (2003) report the following quote from a Big 4 audit partner. The process we have is good but is designed for the annual audit. . .We will need the ability to push a button at any point in time and have the system summarize for the engagement team process issues identified to-date that can lead to risk that the financial statements are inaccurate, rather than rely only on a traditional review of workpapers, manual summarizations of issues and follow-up. [We need to be able to] use technology to actually audit as opposed to using technology to automate manual auditing procedures. Although continuous assurance was proposed over a decade ago (Groomer and Murthy, 1989; Vasarhelyi and Halper, 1991; Vasarhelyi et al., 1991), to move assurance from the current periodic model to a continuous process, auditors will need to look to nontraditional tools and monitoring techniques. Furthermore, Kogan et al. (1999) have called for and suggested research to be completed that would support the development of continuous assurance and audit. To adequately monitor on a continuous basis, data must be provided to the auditor on a (near) continuous basis, a state that is not currently available. With the increased use of tools, such as XBRL, the feasibility of (near) continuous data is becoming a reality. Using data from a steady, frequent stream, the current research suggests using control charts to assist auditors and other decision makers in the identification of patterns in the underlying processes that produce financial statements. Control charts have historically been used for monitoring manufacturing processes and identifying when processes become ‘‘out of control’’. Once interesting patterns are identified, decision makers may focus on the processes that generated the patterns. The objective of continuous monitoring is detecting data abnormalities as near as possible to the time of occurrence of the underlying event that generated the data. If an error or irregularity occurs and is detected, the situation can often be corrected, and the effect quickly mitigated. Overall, the purpose for monitoring financial information is to gain confidence that the systems are operating as intended through the ability to identify and resolve errors, irregularities, or inconsistencies. The current research demonstrates a tool, the use of control charts, to detect abnormal patterns in data, providing support for additional analysis and the detection of potential problems in the underlying system. In this paper, the authors propose to use the control chart approach (in conjunction with other analytical techniques) to monitor the financial statements as they are published. As the frequency of the financial statements increases and their timing approaches continuous, the monitoring of the near-continuous statements needs to occur in a near-continuous manner. Control charts typically are not used to monitor inputs (transactions, in the case of accounting information); they are used to monitor the output of a process (account balances).
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
111
The remainder of the paper is organized as follows. In the following section, background literature is provided for the research described in this paper. This background includes material addressing continuous auditing and assurance and an overview of control charts and their use. Section 3 provides a description of the methods used to transform the data for use in control charts. Section 4 discusses the results obtained. The final sections, Sections 5 and 6, respectively, provide descriptions of some of the limitations and future research directions and the conclusions based on this research.
2. Background 2.1. Continuous auditing and assurance As pointed out above, continuous audit and assurance was proposed over a decade ago. Groomer and Murthy (1989) proposed embedding audit modules into a client’s software. By using this method, it would be possible to capture and evaluate relevant information in a more continuous manner. Vasarhelyi and Halper (1991) described an alternative audit approach called the Continuous Process Audit Methodology (CPAM). In this approach, they suggest that ‘‘continuous process auditing can be considered as a meta form of control and can be used in monitoring control. . .by scanning for the occurrence of certain patterns. . .’’ (Vasarhelyi and Halper, 1991, p. 180). Furthermore, they propose that by having online systems monitored in a continuous manner, auditing will become more of an audit by exception instead of a periodic audit; that is, when an exception occurs, an audit will be required based on the nature of the exception. In 1997, a joint committee of the AICPA and CICA issued a report that suggested that CPAs and CAs will need to monitor the functioning of a firm’s systems to assure the reliability of the data provided by the systems (American Institute of Certified Public Accountants [AICPA], 1997). They suggested that the CPA/CA would need to ‘‘either (1) embed some level of monitoring or control in the client’s system or (2) direct regular inquiries into client processing systems/databases’’ (Vasarhelyi, 2002, p. 261). The approach described in the current paper provides the auditor an aid to determine when and where to query a client’s system. As reported by Searcy et al. (2003), a Big 4 audit partner suggests that ‘‘the use of technology tools that assist with auditing through our clients’ systems will become increasingly important’’. They also provide evidence that as continuous auditing of continuous reporting increases, the Big 4 partners believe that the clients’ expectations of an auditor’s ability and responsibility will increase in the areas of (1) reporting ongoing concern problems in a more timely manner, (2) detecting fraud, and (3) determining the reliability of financial information. Based on the above statement, it seems that as the audit profession moves toward a more continuous audit by exception philosophy, the auditor must be able to monitor the financial status of a client firm in a real-time manner. In the next subsection, the use of control charts, as a basis to raise alarms to notify an auditor to further investigate a possible problem within a client’s system, is described.
112
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
2.2. Control charts The control chart, also known as a Shewhart chart, was created in the 1920s by Dr. Walter A. Shewhart of the Bell Telephone Labs (Shewhart, 1926). It was initially created to provide a way to monitor manufacturing processes in a statistically sound manner. The control chart continues to be one of the primary tools used in statistical quality control. Shewhart suggested that control charts have two basic uses: as a judgment and as an operation (Deming, 1986). When used for judgment purposes, observations are reviewed after the fact to determine whether a process was in statistical control or not. When used as an operation, the purpose is to observe whether an ongoing process is still in control or not. From a continuous monitoring perspective, it is the latter use that is proposed. There are many different types of control charts; each type is essentially a run chart using values of observations taken from a manufacturing process and then plotting those observations around the mean. As long as the observations do not stray too far from the mean for too long, the underlying process is considered to be under control (Brassard, 1989; Harris, 1996). Furthermore, experiments conducted by Shewhart demonstrated that the underlying distribution of the data was not relevant when preparing and evaluating control charts (Shirland, 1993). Wheeler (1993, p. 29) notes that ‘‘instead of attempting to attach a meaning to each and every specific value of the time series, the Control Chart Approach concentrates on the behavior of the underlying process’’. Because accounting irregularities are frequently based on systematic issues, rather than on an isolated event or transaction, the control chart approach may be more appropriate than many of the traditional analytical review procedures when attempting to uncover irregularities. In fact, process-based issues may be more difficult to determine using traditional analytical review techniques. Manufacturing processes have been monitored frequently through the use of control charts (Shirland, 1993). If a process is in control, the control chart will appear as if the variation in the control chart is randomly distributed. If a process is out of control, the data will appear to have a pattern or some abnormality. Even if a process is in statistical control, it does not necessarily guarantee that the process will produce a valid or usable product. However, if a process is out of control, the likelihood of the process producing a usable product is low (Shirland, 1993). Wheeler (1993, p. 130) indicates that each signal identified on a control chart provides the decision maker with ‘‘an opportunity to gain more insight into the process’’. Financial accounting information has traditionally been produced annually (and quarterly for public companies). This relatively long period has allowed for the postponement of recording of many transactions, such as inventory corrections, depreciation expense, overhead application and other adjustments. With more frequent (continuous or near-continuous) reporting, the process of recording adjustment transactions would enable contemporaneous reporting with their occurrence, providing a more complete financial picture of the organization on a timely basis. This process will be required to produce continuous financial reports. Metaphorically speaking, financial statement creation parallels a manufacturing process: raw materials (data) are purchased (collected) and processed (classified and recorded), providing the finished goods (financial reports). Real-time financial reporting
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
113
should emulate the results of the manufacturing process. Mauch (1993) indicates that traditional financial measures may not represent the lag between changes in the manufacturing environment and financial reporting. By decreasing the reporting period, this lag is decreased, and in the extreme case of real-time reporting, the financial reports represent a timely representation of the production process. As control charts have been historically used in components of production processes, they should also map to components of real-time financial reports. As control charts are overlaid on the financial reporting process, one should be able to determine if the raw data or financial statement creation process is going out of control. Through the application of this manufacturing metaphor in conjunction with control charts, it is possible to identify potential problem areas within the financial statements. Such data or processing problems could be due to information systems problems, fraud, and/or accounting irregularities. Control charts have been previously suggested for management use within an organization to improve the quality of accounting processes, such as issuance of invoices and preparation of tax returns; such applications would also improve the overall reliability of the system (Walter et al., 1990; Reeve and Philpot, 1988). The current research proposes the use of control charts to raise flags to notify decision makers that additional investigations may be necessary. Because the control charts provide an alert of a potential problem, it is important that they operate at an appropriate sensitivity level. Control charts that are too sensitive will raise flags when not appropriate, causing unnecessary investigations and other problems for decision makers. Control charts with too little sensitivity may lead a decision maker to overlook serious problems that may exist. As such, control charts should be only one of many tools used by decision makers to monitor information identified as ‘‘high risk’’ to the financial health of an organization. 2.3. Analytical review and control charts Traditional auditing literature describes the characteristics of analytical procedures, including the identification, investigation and evaluation of differences between the auditors’ expectations and the recorded values (AICPA, 2001). Traditionally, trend analysis has been used as one of these procedures. Control charts, as proposed in this research, examine the state of control of the underlying system, not the expected value of a specific point, as done in trend analysis. The authors do not suggest that traditional procedures be abandoned, but as reporting becomes increasingly frequent, the auditors’ use of those procedures will likely be modified to reflect the nature of the data. Additionally, new tools will be developed to address new needs. Under the current financial reporting paradigm, control charts would be of limited usefulness because, based on annual data, it would take seven years (using the rule-of-sevens) to determine if a process is out of control. Alternately, in a continuous environment, seven data collection instances may be relatively close together, providing an opportunity to address an out-ofcontrol system issue on a timely basis. Using control charts when evaluating runs should increase in usefulness as reporting periods decrease. When using alternate rules (nonrun) or underlying data, control charts may be beneficial as analytical procedures by auditors of traditional financial information. For example, identifying items that are three or more standard deviations from the mean can
114
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
detect a potential problem, although items with single period changes may be detected by traditional techniques. Additionally, by suggesting control charts for data collection processes, the researches of Walter et al. (1990) and Reeve and Philpot (1988) support the use of the techniques to achieve organizational control goals.
3. Methods To illustrate the application of control charts to the monitoring of financial reporting, the authors developed control charts from certain historical financial data collected from several firms within three industries. The data used to develop the control charts in this paper were collected from Compustat. Due to the increased reporting frequency over annual data, quarterly data were selected. Currently, there is not a common source of data with more frequent periodicity and reliable financial data available for companies and industries. The data selected for this study encompassed the periods from first quarter 1993 through fourth quarter 2002 (10 years). In this paper, the authors demonstrate a method of monitoring financial information, including the detection of anomalies. One of the potential weaknesses of the control chart approach is that control charts are susceptible to two types of errors: Type 1 errors relate to raising false alarms and Type II errors relate to not raising an alarm when one should be raised (Tannock, 2003). As such, for the authors’ purposes, a random sample was not viable. Companies with known anomalies were required to demonstrate whether the control chart approach would detect those anomalies, i.e., avoid Type II errors. The authors selected financial information relating to three companies with a history of specific financial irregularities. To demonstrate the concept, the authors chose companies for which the symptoms of the irregularities differed. For example, companies to be selected should have known problems with that manifest with a balance sheet symptom, an income statement symptom, or a composite symptom. The firms chosen were WorldCom, Rite Aid and Oxford Health Plans. For comparison purposes, the authors also chose two additional firms in each of the corresponding industries of the ‘‘problem’’ firms. For the purpose of this paper, industry is defined as companies with identical SIC codes. The two additional firms that were chosen are not known to have a history of financial irregularities. The comparison firms for WorldCom included AT&T and BellSouth; for Rite Aid, CVS and Walgreen’s; and for Oxford Health Plans, Aetna and CIGNA. The authors are not making a judgment regarding the overall similarities of the firms, other than they share an SIC code and are not known to have material irregularities, i.e., no Type I errors. Selected account balances of the problem firms were also compared with the related account average values for their respective industries. The specific accounts investigated were based on suggestions by Mulford and Comiskey (2002) and Schilit (2002). The accounts used to illustrate the control chart concepts varied by company, based on the type of irregularity that was known to exist within the company. For WorldCom, the irregularity involved improperly classifying cost of sales as long-term assets (Securities and Exchange Commission [SEC], 2002a); the authors selected the cost of goods sold account to use for analysis in the current paper. Rite Aid was charged by the SEC (2002b)
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
115
with a variety of accounting irregularities. Among these irregularities were erroneous inventory markdowns, leading the authors to use the inventory account as the illustrative account. The accounting irregularities of the third company, Oxford Health Plans, were apparently unintentional and based on billing issues related to a computer upgrade (Schilit, 2002). For Oxford Health, the authors looked at net income, an aggregation of the nominal accounts. For this paper, the authors extracted quarterly information for a 10-year time period, providing 40 data points for the preparation of control charts and data analysis. In practice, an individual monitoring the financial statements ideally should have access to the actual values in a more continuous manner. Several procedures were used to standardize the raw extracted data. For demonstration purposes, Table 1 shows only the results from standardization procedures for the first two years (eight quarters) of the cost of goods sold account data for the WorldCom analysis. Table 1 begins with actual values for the cost of goods sold and sales accounts. Following Schilit (2002) and Welsch et al. (1976), the data were ‘‘common sized’’ by dividing the selected accounts by the total sales, for the income statement accounts (cost of goods sold for WorldCom and comparables), or total assets, for the balance sheet accounts (inventory for Rite Aid and comparables), for the same period. In the Table 1 example, cost of goods sold was divided by sales (see common-sized values). Second, a moving average of four data periods (quarters) was used to compute the base line for the control chart. Shirland (1993) recommends subgroup sample sizes of four or five data points for such computations. As such, the authors chose to use four points to provide one year of data in the moving average computation. This choice ensured that seasonal variations were considered for each observation. In this case, the first period in which a moving average was computable was for Dec-92 (see moving averages in Table 1). Furthermore, 25 subgroups tend to be considered to be a minimum to set up control charts (Shirland, 1993). In this research, 36 subgroups are reported. Based on those subgroups, a moving standard deviation was computed, that in conjunction with the moving average could be used to compute a z score for each of the ‘‘four-period windows’’ (see moving S.D. in Table 1). Finally, a ‘‘z transformation’’ of the data was used to set the base line to zero and standardize the periodic data (see z score transformation in Table 1). Based on the z score values, control charts were created and analyzed (see Discussion). There are many different rules on which to base the control chart analysis. Within this paper, the authors investigate the use of rules regarding runs in z-transformed account values to identify areas for additional investigation. For example, if the ztransformed value of a specific account has a set of seven positive (or negative) values above (or below) the moving average in seven consecutive time periods, there could be a problem with the underlying process. The probability of seven values in a row being above (or below) the mean is less than 2%. This is the so-called ‘‘rule-of-seven’’ (Shirland, 1993). Through the application of control charts and the appropriate rules to monitor financial accounts (in a manner that the charts and rules have previously been applied in manufacturing environments), the authors believe that it is possible to identify problem areas within the accounts, before the accounts go out of control and potentially irreversible consequences occur.
Mar-92
Common-sized values COGS—industry WorldCom—COGS Bell South—COGS AT&T—COGS
0.554266 0.572806 0.575895 0.557919
Jun-92
Sep-92
Dec-92
Mar-93
Jun-93
Sep-93
Dec-93
614.8397 112.311 2170.10 8788.00 1090.765 197.815 3764.10 15,845.00
538.8757 116.821 2216.10 9023.00 1033.385 205.78 3736.00 16,180.00
618.7917 123.8 2371.20 9713.00 1149.957 213.41 3910.10 17,504.00
623.2446 129.2 2262.00 9517.00 1092.328 219.011 3833.70 15,719.00
641.5731 148.003 2279.60 10,263.00 1132.138 251.514 3906.90 17,337.00
644.2354 158.088 2316.70 10,284.00 1116.206 281.788 4014.90 17,225.00
712.2872 222.546 2436.30 11,571.00 1190.136 392.401 4124.80 19,070.00
0.563678 0.567758 0.576526 0.554623
0.521467 0.567699 0.593175 0.557664
0.5381 0.580104 0.60643 0.554902
0.570566 0.589925 0.590031 0.605446
0.566692 0.588448 0.583481 0.591971
0.577165 0.561018 0.577026 0.597039
0.598492 0.567139 0.590647 0.606765
Moving averages COGS—industry WorldCom—COGS Bell South—COGS AT&T—COGS
0.544378 0.572092 0.588006 0.556277
0.548452 0.576371 0.59154 0.568159
0.549206 0.581544 0.593279 0.577496
0.563131 0.579874 0.589242 0.587339
0.578229 0.576632 0.585296 0.600305
Moving S.D. COGS—industry WorldCom—COGS Bell South—COGS AT&T—COGS
0.018571 0.005854 0.014658 0.001756
0.022776 0.010755 0.012276 0.024896
0.023486 0.010193 0.009652 0.02513
0.017238 0.013294 0.012629 0.022328
0.014184 0.014722 0.006397 0.00703
Z-score transformations COGS—industry WorldCom—COGS Bell South—COGS AT&T—COGS
0.33805 1.368816 1.256839 0.78326
0.97091 1.260149 0.12297 1.497719
0.744516 0.677388 1.0151 0.576025
0.814154 1.41844 0.9673 0.43443
1.42861 0.64481 0.83659 0.918808
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Actual account values COGS—industry 600.6124 WorldCom—COGS 105.252 Bell South—COGS 2153.10 AT&T—COGS 8578.00 Sales—industry 1083.617 WorldCom—sales 183.748 Bell South—sales 3738.70 AT&T—sales 15,375.00
116
Table 1 Two years of values for cost of goods sold for AT&T, Bell South WorldCom, and the industry based on Compustat quarterly account data (March 1992 – December 2001)
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
117
4. Results and discussion The three examples of using control charts to identify potential problem areas are described in this section. First, control charts associated with WorldCom and its competitors and industry are presented. Next, control charts are presented for the data associated with Rite Aid and its competitors and industry. Finally, the results associated with Oxford Health Plans and its competitors and industry are presented in control chart format. The control charts are presented in Figs. 1 through 6. In Figs. 1, 3 and 5, data points that fall under the rule-of-seven are identified with an oval1. Fig. 1 is a control chart demonstrating the behavior of WorldCom’s cost of goods sold from the first quarter of 1992 through the fourth quarter of 2001 (data points identified by diamonds). An industry comparison for the values of cost of goods sold is shown on the chart with squares. An examination of the WorldCom data shows that in Quarters 21 through 31 (Quarter 1, 1997 through Quarter 3, 1999 consisting of 11 data points), the data points are in violation of the rule-of-seven, indicating that there may be some systematic problem with the cost of goods sold during the period. This pattern is consistent with the types of issues that are covered in the allegations made against WorldCom. The company has since restated financial results from 2001 and 2002. Had auditors considered the authors’ proposed method of analysis during 1998, it is likely that the cost of goods sold would have received additional scrutiny during the period under question. As a control, the industry data for the cost of goods sold account are also presented (industry data points are shown as squares). Based on the stated criteria, there do not appear to be any systematic patterns in the industry as a whole. As an additional control, Fig. 2 displays the cost of good sold information for two other companies in WorldCom’s industry, BellSouth (x) and AT&T (n). Analysis of these data also shows no systematic patterns to suggest that the underlying systems used to report cost of goods sold for these companies are out of control. The example of Rite Aid is provided in Fig. 3. Rite Aid’s data are shown using the diamond symbols, while the industry data are identified with squares. Rite Aid’s various problems with inventory prompted its inclusion in this study. In examining the data in Fig. 3, there are two time periods that are identified when applying the rule-of-seven. The first period includes the seven periods from the second quarter of 1996 through the fourth quarter of 1997 (inclusive). The second includes the seven periods from the first quarter of 1998 through the third quarter of 1999 (inclusive). The SEC (2002b) complaint against Rite Aid indicates that problems existed from May 1997 through May 1999, a period that overlaps the time periods where Fig. 3 would support additional investigation. The industry data presented in Fig. 3 do not indicate the existence of a systemic problem within the industry as a whole. 1 The results included in this research are valid even though the charts appear to return to a state of being in control. The primary reason is that when continuously monitoring information is received chronologically, the future points ‘‘going back’’ to control have not happened. Secondly, one cannot assume that the system has returned to control, even if the run has completed. In the current research, a moving average was used for the base line. Over time, such a base line may adjust to the new state, obscuring the underlying change. For this reason, it is important to monitor continuously and consider deviations, rather than waiting to see if a problem resolves itself.
118 R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 1. Control chart—Cost of goods sold of WorldCom and industry quarterly data (March 1992 – December 2001).
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 2. Control chart—Cost of goods sold of BellSouth and AT&T quarterly data (March 1992 – December 2001). 119
120 R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 3. Control chart—Inventory of Rite Aid and industry quarterly data (March 1992 – December 2001).
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 4. Control chart—Inventory of Walgreen and CVS quarterly data (March 1992 – December 2001). 121
122 R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 5. Control chart—Net income of Oxford Health Plan and industry quarterly data (March 1992 – December 2001).
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 6. Control chart—Net income of Aetna and CIGNA quarterly data (March 1992 – December 2001).
123
124
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 4 shows data collected from the two other companies selected from within Rite Aid’s industry, Walgreen’s (x) and CVS (n). The same methods and criteria that were used to evaluate Rite Aid do not provide support for the existence of out-of-control inventory systems at Walgreen’s or CVS. Net income data from Oxford Health Plans (x) are presented, with the corresponding industry data (n), in Fig. 5. Schilit (2002) describes the circumstances surrounding a 1997 New York State Insurance Investigation, ending with a US$3 million fine for Oxford and a mandated US$50 million increase in insurance reserves. In addition, in 1997, the value of the company’s common stock fell 62%. At the heart of the problem was a 1996 computer upgrade that did not work properly, negatively impacting earnings. The results displayed in Fig. 5 indicate the potential for a problem in the nine periods prior to the actual problem (second quarter, 1994 through second quarter 1996, inclusive). It is likely that these results do not identify the actual problem, but may identify systematic issues that precipitated the need for the system upgrades. Once again, the industry data do not provide any support for the existence of an industry-based problem. In the final example, Fig. 6 presents net income data from two members of Oxford’s industry, Aetna and CIGNA. The data provided for these companies do not indicate that a problem exists in their systems or underlying data. For each company that was identified as having prior public disclosures of significant problems within its financial accounting and reporting system, the figures show relevant accounts and how using control charts and the rule-of-seven may help identify problems that might go unnoticed for a significant amount of time, increasing financial damage to the company and ultimately to its investors. Concurrently, industry data were presented, which indicated that the potential problems were not industrywide, but rather company specific. In addition to the industry data, two other companies within each industry were presented to support the validity of the rule to not only identify problems, but also to ‘‘nonidentify’’ companies that do not have problems.
5. Limitations and future research This study is an initial proposal to use control charts to identify problem areas within information produced by an accounting information system. Within the study, the authors demonstrate the usefulness of the control chart, a tool created and refined for use in manufacturing environments but applied to financial accounting information systems by the authors. Because this is an initial study, the authors recognize and acknowledge several significant limitations of the study. Many of these limitations are being addressed in other research currently in process by the authors. One of the main limitations of this research is the lack of true continuous data. Before continuous data are available, it is not possible to comprehensively test and refine the rules for evaluating control charts. Although such refinements are not currently possible, it is important that auditors have a framework for continuous monitoring that includes tools such as control charts. When continuous data are available, much additional testing and refinements will be required. Future research should develop simulations of continuous data, based on actual company data, to begin this refinement process.
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
125
The lack of real continuous data also affects the sample. The sample was selected and used to examine a range of instances, including multiple variables across three industries and nine companies, with three of the companies having known accounting irregularities. Although quarterly data are ‘‘more continuous’’ than annual data are, they are far from continuous in reality. To address this issue will require corporate transparency, a requirement that is complicated by the need for data from companies with irregularities. Typically, one would not expect a company to agree to disclosing data if an irregularity or investigation is known to be in process. In the near future, as technologies, such as XBRL, are implemented, it might be possible to accumulate instances of near-continuous data for a study of active companies. In addition to issues surrounding limited continuous data availability, there are also potential limitations in applying control charts to financial processes. Because this is a new domain for this application, the rules to interpret the charts and identify out-of-control systems may need to be modified. Alternatively, new rules may need to be developed. Other common rules (to manufacturing) that are currently under investigation by the authors include:
If an individual point is above (or below) three standard deviations from the mean, then, the underlying process is potentially out of control. If the values trend in the same direction (increasing or decreasing) for seven periods, then, it is likely that the underlying process is out of control. If the values for two or more periods are greater than two standard deviations (Z = 2) but within the actual control limits, then, it is likely that the underlying process is out of control. If the values for four or more periods are greater than one standard deviation (Z = 1) but within the actual control limits, then, it is likely that the underlying process is out of control. With any control chart rule, one must consider the frequency of reporting, while examining the usefulness of standard control charts when monitoring accounting processes at all reporting frequencies, including continuous reporting. It is likely that companies at different levels of maturity may need to be evaluated by different rules. For example, a start-up company would likely be in a growth mode, making the likelihood of a false out-of-control identification a possibility. There also may need to be variation of rules based upon company size, if it is determined that the size of the firm has an effect on the cost or asset structure of a firm. Future research should be conducted to examine many of the limitations of this paper. Researchers need to examine the effect of control charts on auditors’ abilities to detect irregularities, as well as alternative methods of notification to auditors that irregularities exist. If control charts help auditors identify system irregularities, do they do so better than do the alternative methods? Obviously, fine-tuning the rules to interpret control charts is a necessary component of the stream of research. As stated previously, there are many different types of control charts. Only results based on the original work of Shewhart are presented in this paper. Recent research in manufacturing has suggested the use of fuzzy logic and neural networks to help customize
126
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
the rules associated with control chart interpretation (Guh and Tannock, 1999; Tannock, 2003). Additionally, concerns have been raised when the data are correlated (Wardel et al., 1992) and when they are used to monitor high-quality processes (Xie et al., 2002). However, until continuous or near-continuous data are available, it is unclear which of the many types of control charts or which rules would be most appropriate. There are many more issues regarding the introduction of new tools into the audit process. The current paper was planned to introduce the concept of control charts into continuous monitoring and begin the process of these future investigations.
6. Conclusion In this research, the authors suggest that control charts may provide a way to continuously monitor business and financial processes. Control charts have been successfully used to monitor manufacturing processes and to identify processes when they become out of control. As demonstrated by the provided examples, they may also be beneficial when monitoring financial processes, by helping identify patterns that may indicate a problem with underlying accounting data. Currently, the usefulness of the concept is probably highest for auditors, internal and external, as an additional analytical review procedure. Ultimately, as companies increase the frequency of reporting, the concept also may be beneficial to other decision makers through the identification of systemic problems within an organization.
References American Institute of Certified Public Accountants (AICPA). Report of the Special Committee on Assurance Services, Systems Reliability Assurance Segment; 1997. Available at: http://www.aicpa.org/assurance/scas/ reliab/index.htm. American Institute of Certified Public Accountants. AICPA audit guide: analytical procedures. New York (NY): Author; 2001. Brassard M. The Memory Jogger Plus+: featuring the seven management and planning tools. Methuen (MA): Goal/QPC; 1989. Deming WE. Out of the crisis. Cambridge (MA): MIT Center for Advanced Engineering Study; 1986. Groomer M, Murthy U. Continuous auditing of database applications: an embedded audit module approach. J Inf Syst 1989;3(2):53 – 69. Guh RS, Tannock JDT. A neural network approach to characterize pattern parameters in process control charts. J Intell Manuf 1999;10:449 – 62. Harris RL. Information graphics: a comprehensive illustrated reference. Atlanta (GA): Management Graphics; 1996. Kogan A, Sudit E, Vasarhelyi M. Continuous online auditing: a program of research. J Inf Syst 1999;13(2): 87 – 103. Mauch PD. A basic approach to quality control and SPC. Milwaukee (WI): ASQC Quality Press; 1993. Mulford CW, Comiskey EE. The financial numbers game: detecting creative accounting practices. New York (NY): Wiley; 2002. Reeve JM, Philpot JW. Applications of statistical process control for financial management. J Cost Manage Manuf Ind 1988 (Fall);33 – 40. Schilit H. Financial shenanigans: how to detect accounting gimmicks and fraud in financial reports. 2nd ed. New York (NY): McGraw-Hill; 2002.
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
127
Searcy D, Woodroof J, Behn B. Continuous audit: the motivations, benefits, problems, and challenges identified by partners of a Big 4 accounting firm. In: Sprague Jr R, editor. Proceedings of the 36th Hawaii International Conference on System Sciences. Los Alamitos (CA): IEEE Computer Society Press; 2003. Securities and Exchange Commission (SEC). Re: WorldCom, Inc: Revised Statement Pursuant to Section 21(a)(1) of the Securities Exchange Act of 1934; 2002a. Available at: http://www.sec.gov/news/extra/ wcresponserev.htm. Securities and Exchange Commission (SEC). SEC Announces Fraud Charges Against Former Rite Aid Senior Management; 2002b. Available at: http://www.sec.gov/news/press/2002-92.htm. Shewhart WA. Quality control charts. Bell Syst Tech J 1926 (October);593 – 603. Shirland LE. Statistical quality control with microcomputer applications. New York (NY): Wiley; 1993. Tannock JDT. A fuzzy control charting method for individuals. Int J Prod Res 2003;41(5):1017 – 32. Vasarhelyi MA. Chap. 12: concepts in continuous assurance. In: Arnold V, Sutton SG, editors. Researching accounting as an information system discipline. Sarasota (FL): American Accounting Association; 2002. Vasarhelyi MA, Halper F. The continuous audit of online systems. Audit, J Pract Theory 1991;10(1):110 – 25. Vasarhelyi MA, Halper F, Ezawa K. The continuous process audit system: a Unix-based auditing tool. EDP Audit J 1991;3:85 – 91. Walter RM, Higgins MM, Roth HP. Applications of control charts. CPA J. 1990;90 – 5. Wardel DG, Moskowitz H, Plante RD. Control charts in the presence of data correlation. Manage Sci 1992 (August);38(8):1084 – 105. Welsch GA, Zlatkovich CT, White JA. Intermediate accounting. Homewood (IL): Irwin; 1976. Wheeler DJ. Understanding variation: the key to managing chaos. Knoxville (TN): SPC Press; 1993. Xie M, Goh TN, Kuralmani V. Statistical models and control charts for high-quality processes. Boston (MA): Kluwer Academic Publishing; 2002.
Using control charts to monitor financial reporting of public companies Richard B. Dull a,*, David P. Tegarden b a
School of Accountancy and Legal Studies, Clemson University, 309 Sirrine Hall, Clemson, SC 29634, USA b Pamplin College of Business, Virginia Tech, USA Received 31 March 2003; received in revised form 30 November 2003; accepted 1 January 2004 Available online
Abstract There is currently much being written about increasing the frequency and timeliness of financial reporting. Comments frequently support the desirability and question the feasibility of continuous reporting. Under the current reporting/assurance paradigm, if companies report continuously, auditors must monitor and audit on a continuous basis. Current audit standards generally address the outcomes, rather than the detailed procedures that the auditor must follow to meet the objectives of the audit. The purpose of this paper is to propose and demonstrate a technique for monitoring continuous financial information using control charts of accounting information. In this study, financial data were collected and used to implement control charts. Part of the selected companies had known errors, while others had no known errors. The resulting control charts and common interpretation rules identified potential systematic problems only in the companies with known errors. The authors suggest that when continuous data become available, charts similar with these can be used in conjunction with statistical and analytical techniques to detect signals that financial processes are not in control. Refinements of this technique should assist those internal and external to organizations, who are concerned with the reliability of information produced and reported by the organization. D 2004 Elsevier Inc. All rights reserved. Keywords: Control charts; Financial reporting; Public companies
1. Introduction Over the past two decades, the proliferation of technology has led to the increased availability of financial information. Although the timing of financial reporting has not * Corresponding author. Tel.: +1-864-656-0610; fax: +1-864-656-4892. E-mail address: [email protected] (R.B. Dull). 1467-0895/$ - see front matter D 2004 Elsevier Inc. All rights reserved. doi:10.1016/j.accinf.2004.01.004
110
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
changed significantly, currently, there is much being written about increasing the frequency and timeliness of financial reporting and assurance (Vasarhelyi, 2002). Technology advances have generally increased system capabilities, enabling integrated packages to simplify and accelerate the mechanical process of financial reporting. Generally, people agree that increasing the frequency may be useful but have reservations about the feasibility of the implementation of continuous reporting and assurance. For example, Searcy et al. (2003) report the following quote from a Big 4 audit partner. The process we have is good but is designed for the annual audit. . .We will need the ability to push a button at any point in time and have the system summarize for the engagement team process issues identified to-date that can lead to risk that the financial statements are inaccurate, rather than rely only on a traditional review of workpapers, manual summarizations of issues and follow-up. [We need to be able to] use technology to actually audit as opposed to using technology to automate manual auditing procedures. Although continuous assurance was proposed over a decade ago (Groomer and Murthy, 1989; Vasarhelyi and Halper, 1991; Vasarhelyi et al., 1991), to move assurance from the current periodic model to a continuous process, auditors will need to look to nontraditional tools and monitoring techniques. Furthermore, Kogan et al. (1999) have called for and suggested research to be completed that would support the development of continuous assurance and audit. To adequately monitor on a continuous basis, data must be provided to the auditor on a (near) continuous basis, a state that is not currently available. With the increased use of tools, such as XBRL, the feasibility of (near) continuous data is becoming a reality. Using data from a steady, frequent stream, the current research suggests using control charts to assist auditors and other decision makers in the identification of patterns in the underlying processes that produce financial statements. Control charts have historically been used for monitoring manufacturing processes and identifying when processes become ‘‘out of control’’. Once interesting patterns are identified, decision makers may focus on the processes that generated the patterns. The objective of continuous monitoring is detecting data abnormalities as near as possible to the time of occurrence of the underlying event that generated the data. If an error or irregularity occurs and is detected, the situation can often be corrected, and the effect quickly mitigated. Overall, the purpose for monitoring financial information is to gain confidence that the systems are operating as intended through the ability to identify and resolve errors, irregularities, or inconsistencies. The current research demonstrates a tool, the use of control charts, to detect abnormal patterns in data, providing support for additional analysis and the detection of potential problems in the underlying system. In this paper, the authors propose to use the control chart approach (in conjunction with other analytical techniques) to monitor the financial statements as they are published. As the frequency of the financial statements increases and their timing approaches continuous, the monitoring of the near-continuous statements needs to occur in a near-continuous manner. Control charts typically are not used to monitor inputs (transactions, in the case of accounting information); they are used to monitor the output of a process (account balances).
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
111
The remainder of the paper is organized as follows. In the following section, background literature is provided for the research described in this paper. This background includes material addressing continuous auditing and assurance and an overview of control charts and their use. Section 3 provides a description of the methods used to transform the data for use in control charts. Section 4 discusses the results obtained. The final sections, Sections 5 and 6, respectively, provide descriptions of some of the limitations and future research directions and the conclusions based on this research.
2. Background 2.1. Continuous auditing and assurance As pointed out above, continuous audit and assurance was proposed over a decade ago. Groomer and Murthy (1989) proposed embedding audit modules into a client’s software. By using this method, it would be possible to capture and evaluate relevant information in a more continuous manner. Vasarhelyi and Halper (1991) described an alternative audit approach called the Continuous Process Audit Methodology (CPAM). In this approach, they suggest that ‘‘continuous process auditing can be considered as a meta form of control and can be used in monitoring control. . .by scanning for the occurrence of certain patterns. . .’’ (Vasarhelyi and Halper, 1991, p. 180). Furthermore, they propose that by having online systems monitored in a continuous manner, auditing will become more of an audit by exception instead of a periodic audit; that is, when an exception occurs, an audit will be required based on the nature of the exception. In 1997, a joint committee of the AICPA and CICA issued a report that suggested that CPAs and CAs will need to monitor the functioning of a firm’s systems to assure the reliability of the data provided by the systems (American Institute of Certified Public Accountants [AICPA], 1997). They suggested that the CPA/CA would need to ‘‘either (1) embed some level of monitoring or control in the client’s system or (2) direct regular inquiries into client processing systems/databases’’ (Vasarhelyi, 2002, p. 261). The approach described in the current paper provides the auditor an aid to determine when and where to query a client’s system. As reported by Searcy et al. (2003), a Big 4 audit partner suggests that ‘‘the use of technology tools that assist with auditing through our clients’ systems will become increasingly important’’. They also provide evidence that as continuous auditing of continuous reporting increases, the Big 4 partners believe that the clients’ expectations of an auditor’s ability and responsibility will increase in the areas of (1) reporting ongoing concern problems in a more timely manner, (2) detecting fraud, and (3) determining the reliability of financial information. Based on the above statement, it seems that as the audit profession moves toward a more continuous audit by exception philosophy, the auditor must be able to monitor the financial status of a client firm in a real-time manner. In the next subsection, the use of control charts, as a basis to raise alarms to notify an auditor to further investigate a possible problem within a client’s system, is described.
112
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
2.2. Control charts The control chart, also known as a Shewhart chart, was created in the 1920s by Dr. Walter A. Shewhart of the Bell Telephone Labs (Shewhart, 1926). It was initially created to provide a way to monitor manufacturing processes in a statistically sound manner. The control chart continues to be one of the primary tools used in statistical quality control. Shewhart suggested that control charts have two basic uses: as a judgment and as an operation (Deming, 1986). When used for judgment purposes, observations are reviewed after the fact to determine whether a process was in statistical control or not. When used as an operation, the purpose is to observe whether an ongoing process is still in control or not. From a continuous monitoring perspective, it is the latter use that is proposed. There are many different types of control charts; each type is essentially a run chart using values of observations taken from a manufacturing process and then plotting those observations around the mean. As long as the observations do not stray too far from the mean for too long, the underlying process is considered to be under control (Brassard, 1989; Harris, 1996). Furthermore, experiments conducted by Shewhart demonstrated that the underlying distribution of the data was not relevant when preparing and evaluating control charts (Shirland, 1993). Wheeler (1993, p. 29) notes that ‘‘instead of attempting to attach a meaning to each and every specific value of the time series, the Control Chart Approach concentrates on the behavior of the underlying process’’. Because accounting irregularities are frequently based on systematic issues, rather than on an isolated event or transaction, the control chart approach may be more appropriate than many of the traditional analytical review procedures when attempting to uncover irregularities. In fact, process-based issues may be more difficult to determine using traditional analytical review techniques. Manufacturing processes have been monitored frequently through the use of control charts (Shirland, 1993). If a process is in control, the control chart will appear as if the variation in the control chart is randomly distributed. If a process is out of control, the data will appear to have a pattern or some abnormality. Even if a process is in statistical control, it does not necessarily guarantee that the process will produce a valid or usable product. However, if a process is out of control, the likelihood of the process producing a usable product is low (Shirland, 1993). Wheeler (1993, p. 130) indicates that each signal identified on a control chart provides the decision maker with ‘‘an opportunity to gain more insight into the process’’. Financial accounting information has traditionally been produced annually (and quarterly for public companies). This relatively long period has allowed for the postponement of recording of many transactions, such as inventory corrections, depreciation expense, overhead application and other adjustments. With more frequent (continuous or near-continuous) reporting, the process of recording adjustment transactions would enable contemporaneous reporting with their occurrence, providing a more complete financial picture of the organization on a timely basis. This process will be required to produce continuous financial reports. Metaphorically speaking, financial statement creation parallels a manufacturing process: raw materials (data) are purchased (collected) and processed (classified and recorded), providing the finished goods (financial reports). Real-time financial reporting
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
113
should emulate the results of the manufacturing process. Mauch (1993) indicates that traditional financial measures may not represent the lag between changes in the manufacturing environment and financial reporting. By decreasing the reporting period, this lag is decreased, and in the extreme case of real-time reporting, the financial reports represent a timely representation of the production process. As control charts have been historically used in components of production processes, they should also map to components of real-time financial reports. As control charts are overlaid on the financial reporting process, one should be able to determine if the raw data or financial statement creation process is going out of control. Through the application of this manufacturing metaphor in conjunction with control charts, it is possible to identify potential problem areas within the financial statements. Such data or processing problems could be due to information systems problems, fraud, and/or accounting irregularities. Control charts have been previously suggested for management use within an organization to improve the quality of accounting processes, such as issuance of invoices and preparation of tax returns; such applications would also improve the overall reliability of the system (Walter et al., 1990; Reeve and Philpot, 1988). The current research proposes the use of control charts to raise flags to notify decision makers that additional investigations may be necessary. Because the control charts provide an alert of a potential problem, it is important that they operate at an appropriate sensitivity level. Control charts that are too sensitive will raise flags when not appropriate, causing unnecessary investigations and other problems for decision makers. Control charts with too little sensitivity may lead a decision maker to overlook serious problems that may exist. As such, control charts should be only one of many tools used by decision makers to monitor information identified as ‘‘high risk’’ to the financial health of an organization. 2.3. Analytical review and control charts Traditional auditing literature describes the characteristics of analytical procedures, including the identification, investigation and evaluation of differences between the auditors’ expectations and the recorded values (AICPA, 2001). Traditionally, trend analysis has been used as one of these procedures. Control charts, as proposed in this research, examine the state of control of the underlying system, not the expected value of a specific point, as done in trend analysis. The authors do not suggest that traditional procedures be abandoned, but as reporting becomes increasingly frequent, the auditors’ use of those procedures will likely be modified to reflect the nature of the data. Additionally, new tools will be developed to address new needs. Under the current financial reporting paradigm, control charts would be of limited usefulness because, based on annual data, it would take seven years (using the rule-of-sevens) to determine if a process is out of control. Alternately, in a continuous environment, seven data collection instances may be relatively close together, providing an opportunity to address an out-ofcontrol system issue on a timely basis. Using control charts when evaluating runs should increase in usefulness as reporting periods decrease. When using alternate rules (nonrun) or underlying data, control charts may be beneficial as analytical procedures by auditors of traditional financial information. For example, identifying items that are three or more standard deviations from the mean can
114
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
detect a potential problem, although items with single period changes may be detected by traditional techniques. Additionally, by suggesting control charts for data collection processes, the researches of Walter et al. (1990) and Reeve and Philpot (1988) support the use of the techniques to achieve organizational control goals.
3. Methods To illustrate the application of control charts to the monitoring of financial reporting, the authors developed control charts from certain historical financial data collected from several firms within three industries. The data used to develop the control charts in this paper were collected from Compustat. Due to the increased reporting frequency over annual data, quarterly data were selected. Currently, there is not a common source of data with more frequent periodicity and reliable financial data available for companies and industries. The data selected for this study encompassed the periods from first quarter 1993 through fourth quarter 2002 (10 years). In this paper, the authors demonstrate a method of monitoring financial information, including the detection of anomalies. One of the potential weaknesses of the control chart approach is that control charts are susceptible to two types of errors: Type 1 errors relate to raising false alarms and Type II errors relate to not raising an alarm when one should be raised (Tannock, 2003). As such, for the authors’ purposes, a random sample was not viable. Companies with known anomalies were required to demonstrate whether the control chart approach would detect those anomalies, i.e., avoid Type II errors. The authors selected financial information relating to three companies with a history of specific financial irregularities. To demonstrate the concept, the authors chose companies for which the symptoms of the irregularities differed. For example, companies to be selected should have known problems with that manifest with a balance sheet symptom, an income statement symptom, or a composite symptom. The firms chosen were WorldCom, Rite Aid and Oxford Health Plans. For comparison purposes, the authors also chose two additional firms in each of the corresponding industries of the ‘‘problem’’ firms. For the purpose of this paper, industry is defined as companies with identical SIC codes. The two additional firms that were chosen are not known to have a history of financial irregularities. The comparison firms for WorldCom included AT&T and BellSouth; for Rite Aid, CVS and Walgreen’s; and for Oxford Health Plans, Aetna and CIGNA. The authors are not making a judgment regarding the overall similarities of the firms, other than they share an SIC code and are not known to have material irregularities, i.e., no Type I errors. Selected account balances of the problem firms were also compared with the related account average values for their respective industries. The specific accounts investigated were based on suggestions by Mulford and Comiskey (2002) and Schilit (2002). The accounts used to illustrate the control chart concepts varied by company, based on the type of irregularity that was known to exist within the company. For WorldCom, the irregularity involved improperly classifying cost of sales as long-term assets (Securities and Exchange Commission [SEC], 2002a); the authors selected the cost of goods sold account to use for analysis in the current paper. Rite Aid was charged by the SEC (2002b)
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
115
with a variety of accounting irregularities. Among these irregularities were erroneous inventory markdowns, leading the authors to use the inventory account as the illustrative account. The accounting irregularities of the third company, Oxford Health Plans, were apparently unintentional and based on billing issues related to a computer upgrade (Schilit, 2002). For Oxford Health, the authors looked at net income, an aggregation of the nominal accounts. For this paper, the authors extracted quarterly information for a 10-year time period, providing 40 data points for the preparation of control charts and data analysis. In practice, an individual monitoring the financial statements ideally should have access to the actual values in a more continuous manner. Several procedures were used to standardize the raw extracted data. For demonstration purposes, Table 1 shows only the results from standardization procedures for the first two years (eight quarters) of the cost of goods sold account data for the WorldCom analysis. Table 1 begins with actual values for the cost of goods sold and sales accounts. Following Schilit (2002) and Welsch et al. (1976), the data were ‘‘common sized’’ by dividing the selected accounts by the total sales, for the income statement accounts (cost of goods sold for WorldCom and comparables), or total assets, for the balance sheet accounts (inventory for Rite Aid and comparables), for the same period. In the Table 1 example, cost of goods sold was divided by sales (see common-sized values). Second, a moving average of four data periods (quarters) was used to compute the base line for the control chart. Shirland (1993) recommends subgroup sample sizes of four or five data points for such computations. As such, the authors chose to use four points to provide one year of data in the moving average computation. This choice ensured that seasonal variations were considered for each observation. In this case, the first period in which a moving average was computable was for Dec-92 (see moving averages in Table 1). Furthermore, 25 subgroups tend to be considered to be a minimum to set up control charts (Shirland, 1993). In this research, 36 subgroups are reported. Based on those subgroups, a moving standard deviation was computed, that in conjunction with the moving average could be used to compute a z score for each of the ‘‘four-period windows’’ (see moving S.D. in Table 1). Finally, a ‘‘z transformation’’ of the data was used to set the base line to zero and standardize the periodic data (see z score transformation in Table 1). Based on the z score values, control charts were created and analyzed (see Discussion). There are many different rules on which to base the control chart analysis. Within this paper, the authors investigate the use of rules regarding runs in z-transformed account values to identify areas for additional investigation. For example, if the ztransformed value of a specific account has a set of seven positive (or negative) values above (or below) the moving average in seven consecutive time periods, there could be a problem with the underlying process. The probability of seven values in a row being above (or below) the mean is less than 2%. This is the so-called ‘‘rule-of-seven’’ (Shirland, 1993). Through the application of control charts and the appropriate rules to monitor financial accounts (in a manner that the charts and rules have previously been applied in manufacturing environments), the authors believe that it is possible to identify problem areas within the accounts, before the accounts go out of control and potentially irreversible consequences occur.
Mar-92
Common-sized values COGS—industry WorldCom—COGS Bell South—COGS AT&T—COGS
0.554266 0.572806 0.575895 0.557919
Jun-92
Sep-92
Dec-92
Mar-93
Jun-93
Sep-93
Dec-93
614.8397 112.311 2170.10 8788.00 1090.765 197.815 3764.10 15,845.00
538.8757 116.821 2216.10 9023.00 1033.385 205.78 3736.00 16,180.00
618.7917 123.8 2371.20 9713.00 1149.957 213.41 3910.10 17,504.00
623.2446 129.2 2262.00 9517.00 1092.328 219.011 3833.70 15,719.00
641.5731 148.003 2279.60 10,263.00 1132.138 251.514 3906.90 17,337.00
644.2354 158.088 2316.70 10,284.00 1116.206 281.788 4014.90 17,225.00
712.2872 222.546 2436.30 11,571.00 1190.136 392.401 4124.80 19,070.00
0.563678 0.567758 0.576526 0.554623
0.521467 0.567699 0.593175 0.557664
0.5381 0.580104 0.60643 0.554902
0.570566 0.589925 0.590031 0.605446
0.566692 0.588448 0.583481 0.591971
0.577165 0.561018 0.577026 0.597039
0.598492 0.567139 0.590647 0.606765
Moving averages COGS—industry WorldCom—COGS Bell South—COGS AT&T—COGS
0.544378 0.572092 0.588006 0.556277
0.548452 0.576371 0.59154 0.568159
0.549206 0.581544 0.593279 0.577496
0.563131 0.579874 0.589242 0.587339
0.578229 0.576632 0.585296 0.600305
Moving S.D. COGS—industry WorldCom—COGS Bell South—COGS AT&T—COGS
0.018571 0.005854 0.014658 0.001756
0.022776 0.010755 0.012276 0.024896
0.023486 0.010193 0.009652 0.02513
0.017238 0.013294 0.012629 0.022328
0.014184 0.014722 0.006397 0.00703
Z-score transformations COGS—industry WorldCom—COGS Bell South—COGS AT&T—COGS
0.33805 1.368816 1.256839 0.78326
0.97091 1.260149 0.12297 1.497719
0.744516 0.677388 1.0151 0.576025
0.814154 1.41844 0.9673 0.43443
1.42861 0.64481 0.83659 0.918808
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Actual account values COGS—industry 600.6124 WorldCom—COGS 105.252 Bell South—COGS 2153.10 AT&T—COGS 8578.00 Sales—industry 1083.617 WorldCom—sales 183.748 Bell South—sales 3738.70 AT&T—sales 15,375.00
116
Table 1 Two years of values for cost of goods sold for AT&T, Bell South WorldCom, and the industry based on Compustat quarterly account data (March 1992 – December 2001)
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
117
4. Results and discussion The three examples of using control charts to identify potential problem areas are described in this section. First, control charts associated with WorldCom and its competitors and industry are presented. Next, control charts are presented for the data associated with Rite Aid and its competitors and industry. Finally, the results associated with Oxford Health Plans and its competitors and industry are presented in control chart format. The control charts are presented in Figs. 1 through 6. In Figs. 1, 3 and 5, data points that fall under the rule-of-seven are identified with an oval1. Fig. 1 is a control chart demonstrating the behavior of WorldCom’s cost of goods sold from the first quarter of 1992 through the fourth quarter of 2001 (data points identified by diamonds). An industry comparison for the values of cost of goods sold is shown on the chart with squares. An examination of the WorldCom data shows that in Quarters 21 through 31 (Quarter 1, 1997 through Quarter 3, 1999 consisting of 11 data points), the data points are in violation of the rule-of-seven, indicating that there may be some systematic problem with the cost of goods sold during the period. This pattern is consistent with the types of issues that are covered in the allegations made against WorldCom. The company has since restated financial results from 2001 and 2002. Had auditors considered the authors’ proposed method of analysis during 1998, it is likely that the cost of goods sold would have received additional scrutiny during the period under question. As a control, the industry data for the cost of goods sold account are also presented (industry data points are shown as squares). Based on the stated criteria, there do not appear to be any systematic patterns in the industry as a whole. As an additional control, Fig. 2 displays the cost of good sold information for two other companies in WorldCom’s industry, BellSouth (x) and AT&T (n). Analysis of these data also shows no systematic patterns to suggest that the underlying systems used to report cost of goods sold for these companies are out of control. The example of Rite Aid is provided in Fig. 3. Rite Aid’s data are shown using the diamond symbols, while the industry data are identified with squares. Rite Aid’s various problems with inventory prompted its inclusion in this study. In examining the data in Fig. 3, there are two time periods that are identified when applying the rule-of-seven. The first period includes the seven periods from the second quarter of 1996 through the fourth quarter of 1997 (inclusive). The second includes the seven periods from the first quarter of 1998 through the third quarter of 1999 (inclusive). The SEC (2002b) complaint against Rite Aid indicates that problems existed from May 1997 through May 1999, a period that overlaps the time periods where Fig. 3 would support additional investigation. The industry data presented in Fig. 3 do not indicate the existence of a systemic problem within the industry as a whole. 1 The results included in this research are valid even though the charts appear to return to a state of being in control. The primary reason is that when continuously monitoring information is received chronologically, the future points ‘‘going back’’ to control have not happened. Secondly, one cannot assume that the system has returned to control, even if the run has completed. In the current research, a moving average was used for the base line. Over time, such a base line may adjust to the new state, obscuring the underlying change. For this reason, it is important to monitor continuously and consider deviations, rather than waiting to see if a problem resolves itself.
118 R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 1. Control chart—Cost of goods sold of WorldCom and industry quarterly data (March 1992 – December 2001).
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 2. Control chart—Cost of goods sold of BellSouth and AT&T quarterly data (March 1992 – December 2001). 119
120 R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 3. Control chart—Inventory of Rite Aid and industry quarterly data (March 1992 – December 2001).
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 4. Control chart—Inventory of Walgreen and CVS quarterly data (March 1992 – December 2001). 121
122 R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 5. Control chart—Net income of Oxford Health Plan and industry quarterly data (March 1992 – December 2001).
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 6. Control chart—Net income of Aetna and CIGNA quarterly data (March 1992 – December 2001).
123
124
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
Fig. 4 shows data collected from the two other companies selected from within Rite Aid’s industry, Walgreen’s (x) and CVS (n). The same methods and criteria that were used to evaluate Rite Aid do not provide support for the existence of out-of-control inventory systems at Walgreen’s or CVS. Net income data from Oxford Health Plans (x) are presented, with the corresponding industry data (n), in Fig. 5. Schilit (2002) describes the circumstances surrounding a 1997 New York State Insurance Investigation, ending with a US$3 million fine for Oxford and a mandated US$50 million increase in insurance reserves. In addition, in 1997, the value of the company’s common stock fell 62%. At the heart of the problem was a 1996 computer upgrade that did not work properly, negatively impacting earnings. The results displayed in Fig. 5 indicate the potential for a problem in the nine periods prior to the actual problem (second quarter, 1994 through second quarter 1996, inclusive). It is likely that these results do not identify the actual problem, but may identify systematic issues that precipitated the need for the system upgrades. Once again, the industry data do not provide any support for the existence of an industry-based problem. In the final example, Fig. 6 presents net income data from two members of Oxford’s industry, Aetna and CIGNA. The data provided for these companies do not indicate that a problem exists in their systems or underlying data. For each company that was identified as having prior public disclosures of significant problems within its financial accounting and reporting system, the figures show relevant accounts and how using control charts and the rule-of-seven may help identify problems that might go unnoticed for a significant amount of time, increasing financial damage to the company and ultimately to its investors. Concurrently, industry data were presented, which indicated that the potential problems were not industrywide, but rather company specific. In addition to the industry data, two other companies within each industry were presented to support the validity of the rule to not only identify problems, but also to ‘‘nonidentify’’ companies that do not have problems.
5. Limitations and future research This study is an initial proposal to use control charts to identify problem areas within information produced by an accounting information system. Within the study, the authors demonstrate the usefulness of the control chart, a tool created and refined for use in manufacturing environments but applied to financial accounting information systems by the authors. Because this is an initial study, the authors recognize and acknowledge several significant limitations of the study. Many of these limitations are being addressed in other research currently in process by the authors. One of the main limitations of this research is the lack of true continuous data. Before continuous data are available, it is not possible to comprehensively test and refine the rules for evaluating control charts. Although such refinements are not currently possible, it is important that auditors have a framework for continuous monitoring that includes tools such as control charts. When continuous data are available, much additional testing and refinements will be required. Future research should develop simulations of continuous data, based on actual company data, to begin this refinement process.
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
125
The lack of real continuous data also affects the sample. The sample was selected and used to examine a range of instances, including multiple variables across three industries and nine companies, with three of the companies having known accounting irregularities. Although quarterly data are ‘‘more continuous’’ than annual data are, they are far from continuous in reality. To address this issue will require corporate transparency, a requirement that is complicated by the need for data from companies with irregularities. Typically, one would not expect a company to agree to disclosing data if an irregularity or investigation is known to be in process. In the near future, as technologies, such as XBRL, are implemented, it might be possible to accumulate instances of near-continuous data for a study of active companies. In addition to issues surrounding limited continuous data availability, there are also potential limitations in applying control charts to financial processes. Because this is a new domain for this application, the rules to interpret the charts and identify out-of-control systems may need to be modified. Alternatively, new rules may need to be developed. Other common rules (to manufacturing) that are currently under investigation by the authors include:
If an individual point is above (or below) three standard deviations from the mean, then, the underlying process is potentially out of control. If the values trend in the same direction (increasing or decreasing) for seven periods, then, it is likely that the underlying process is out of control. If the values for two or more periods are greater than two standard deviations (Z = 2) but within the actual control limits, then, it is likely that the underlying process is out of control. If the values for four or more periods are greater than one standard deviation (Z = 1) but within the actual control limits, then, it is likely that the underlying process is out of control. With any control chart rule, one must consider the frequency of reporting, while examining the usefulness of standard control charts when monitoring accounting processes at all reporting frequencies, including continuous reporting. It is likely that companies at different levels of maturity may need to be evaluated by different rules. For example, a start-up company would likely be in a growth mode, making the likelihood of a false out-of-control identification a possibility. There also may need to be variation of rules based upon company size, if it is determined that the size of the firm has an effect on the cost or asset structure of a firm. Future research should be conducted to examine many of the limitations of this paper. Researchers need to examine the effect of control charts on auditors’ abilities to detect irregularities, as well as alternative methods of notification to auditors that irregularities exist. If control charts help auditors identify system irregularities, do they do so better than do the alternative methods? Obviously, fine-tuning the rules to interpret control charts is a necessary component of the stream of research. As stated previously, there are many different types of control charts. Only results based on the original work of Shewhart are presented in this paper. Recent research in manufacturing has suggested the use of fuzzy logic and neural networks to help customize
126
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
the rules associated with control chart interpretation (Guh and Tannock, 1999; Tannock, 2003). Additionally, concerns have been raised when the data are correlated (Wardel et al., 1992) and when they are used to monitor high-quality processes (Xie et al., 2002). However, until continuous or near-continuous data are available, it is unclear which of the many types of control charts or which rules would be most appropriate. There are many more issues regarding the introduction of new tools into the audit process. The current paper was planned to introduce the concept of control charts into continuous monitoring and begin the process of these future investigations.
6. Conclusion In this research, the authors suggest that control charts may provide a way to continuously monitor business and financial processes. Control charts have been successfully used to monitor manufacturing processes and to identify processes when they become out of control. As demonstrated by the provided examples, they may also be beneficial when monitoring financial processes, by helping identify patterns that may indicate a problem with underlying accounting data. Currently, the usefulness of the concept is probably highest for auditors, internal and external, as an additional analytical review procedure. Ultimately, as companies increase the frequency of reporting, the concept also may be beneficial to other decision makers through the identification of systemic problems within an organization.
References American Institute of Certified Public Accountants (AICPA). Report of the Special Committee on Assurance Services, Systems Reliability Assurance Segment; 1997. Available at: http://www.aicpa.org/assurance/scas/ reliab/index.htm. American Institute of Certified Public Accountants. AICPA audit guide: analytical procedures. New York (NY): Author; 2001. Brassard M. The Memory Jogger Plus+: featuring the seven management and planning tools. Methuen (MA): Goal/QPC; 1989. Deming WE. Out of the crisis. Cambridge (MA): MIT Center for Advanced Engineering Study; 1986. Groomer M, Murthy U. Continuous auditing of database applications: an embedded audit module approach. J Inf Syst 1989;3(2):53 – 69. Guh RS, Tannock JDT. A neural network approach to characterize pattern parameters in process control charts. J Intell Manuf 1999;10:449 – 62. Harris RL. Information graphics: a comprehensive illustrated reference. Atlanta (GA): Management Graphics; 1996. Kogan A, Sudit E, Vasarhelyi M. Continuous online auditing: a program of research. J Inf Syst 1999;13(2): 87 – 103. Mauch PD. A basic approach to quality control and SPC. Milwaukee (WI): ASQC Quality Press; 1993. Mulford CW, Comiskey EE. The financial numbers game: detecting creative accounting practices. New York (NY): Wiley; 2002. Reeve JM, Philpot JW. Applications of statistical process control for financial management. J Cost Manage Manuf Ind 1988 (Fall);33 – 40. Schilit H. Financial shenanigans: how to detect accounting gimmicks and fraud in financial reports. 2nd ed. New York (NY): McGraw-Hill; 2002.
R.B. Dull, D.P. Tegarden / Int. J. Account. Inf. Syst. 5 (2004) 109–127
127
Searcy D, Woodroof J, Behn B. Continuous audit: the motivations, benefits, problems, and challenges identified by partners of a Big 4 accounting firm. In: Sprague Jr R, editor. Proceedings of the 36th Hawaii International Conference on System Sciences. Los Alamitos (CA): IEEE Computer Society Press; 2003. Securities and Exchange Commission (SEC). Re: WorldCom, Inc: Revised Statement Pursuant to Section 21(a)(1) of the Securities Exchange Act of 1934; 2002a. Available at: http://www.sec.gov/news/extra/ wcresponserev.htm. Securities and Exchange Commission (SEC). SEC Announces Fraud Charges Against Former Rite Aid Senior Management; 2002b. Available at: http://www.sec.gov/news/press/2002-92.htm. Shewhart WA. Quality control charts. Bell Syst Tech J 1926 (October);593 – 603. Shirland LE. Statistical quality control with microcomputer applications. New York (NY): Wiley; 1993. Tannock JDT. A fuzzy control charting method for individuals. Int J Prod Res 2003;41(5):1017 – 32. Vasarhelyi MA. Chap. 12: concepts in continuous assurance. In: Arnold V, Sutton SG, editors. Researching accounting as an information system discipline. Sarasota (FL): American Accounting Association; 2002. Vasarhelyi MA, Halper F. The continuous audit of online systems. Audit, J Pract Theory 1991;10(1):110 – 25. Vasarhelyi MA, Halper F, Ezawa K. The continuous process audit system: a Unix-based auditing tool. EDP Audit J 1991;3:85 – 91. Walter RM, Higgins MM, Roth HP. Applications of control charts. CPA J. 1990;90 – 5. Wardel DG, Moskowitz H, Plante RD. Control charts in the presence of data correlation. Manage Sci 1992 (August);38(8):1084 – 105. Welsch GA, Zlatkovich CT, White JA. Intermediate accounting. Homewood (IL): Irwin; 1976. Wheeler DJ. Understanding variation: the key to managing chaos. Knoxville (TN): SPC Press; 1993. Xie M, Goh TN, Kuralmani V. Statistical models and control charts for high-quality processes. Boston (MA): Kluwer Academic Publishing; 2002.