- Email: [email protected]
Discussion of supporting continuous monitoring using control charts
International Journal of Accounting Information Systems 5 (2004) 129 – 130
Discussion
Discussion of supporting continuous monitoring using control charts Severin Grabski * Eli Broad Graduate School of Management, Michigan State University, N270 North Business Complex, East Lansing, MI 48824-1122, USA Received 1 January 2004; accepted 1 January 2004
This is a very interesting paper, and it proposes the use of a technique that has been well used and tested within the process control community, that of control charts. An example is provided that demonstrates that this technique would have identified, as ‘‘out of control’’, the reported results for WorldCom, Rite Aid and Oxford Health Plans. The paper contends that while the demonstration was for quarterly data, continuous data streams could be used as in the statistical process control literature to identify potential areas to investigate. The contribution of this paper seems to be a combination of an audit suggestion and a demonstration. The suggestion portion states that audit should make use of techniques that are commonplace in the operations management area, statistical process controls and control charts. The demonstration is provided as a ‘‘proof of concept’’ of the technique. Three concerns result from this paper: first, is the application of a well-documented technique in other areas a significant contribution to the literature; second, is the demonstration appropriate; and third, what is the underlying theory? These questions are answered in reverse order. From a theoretical perspective, the use of control charts is well documented in operations management, but not in this paper, nor does this paper seek to provide any contributions beyond the idea that control charts ‘‘work.’’ The second question is the appropriateness of the proof of concept demonstration. The paper contends that control charts are appropriate for use in continuous monitoring (and that is the manner in which they are often used in production environments). Unfortunately, the paper demonstrates this using quarterly reported data for different account balances (or totals) for three different organizations. That would be a very good demonstration if the objective was to show the use of control charts in the analytical review phase of the audit. To have a proof of concept for the use of control charts in a
* Tel.: +1-517-432-2922; fax: +1-517-432-1101. E-mail address: [email protected] (S. Grabski). 1467-0895/$ - see front matter D 2004 Published by Elsevier Inc. doi:10.1016/j.accinf.2004.01.005
130
Discussion
continuous audit environment, the manuscript needs to both identify and show how a continuous data stream would be identified and analyzed (i.e., What data? What is in control? What is out of control? What is the underlying distributional shape? Is there an alerting facility for internal audit? How would this be embedded in new systems or added into existing ERP (or ERP-type) systems?). The paper must also demonstrate that this technique is ‘‘better’’ (regardless of how better is defined, e.g., less costly, more accurate, etc.) than other techniques that have been proposed in the continuous (and discrete) audit environments. This could be addressed through a simulation in which a known set of errors is seeded and the efficacy of the continuous monitoring system and control charts could be examined. Are control charts better than other techniques in determining that an investigation is needed (such as a consistent drift away from a random error amount that indicated the need to examine a production process that was still ‘‘in control’’ but was tending to go out of control in a production setting)? Are traditional analytical review techniques just as effective as control charts? Is a simple notification system (when a balance/transaction exceeds a certain amount) better than control charts? In conclusion, the ideas presented in this paper are very good, but need to be developed more fully. The use of quarterly financial data is not the same as how one would operationalize a continuous data stream. The identification of the data, its capture, analysis, standard setting, and so forth must be addressed. Even with that done, the authors need to document why this method would be better than any other methods that have been proposed.
Discussion
Discussion of supporting continuous monitoring using control charts Severin Grabski * Eli Broad Graduate School of Management, Michigan State University, N270 North Business Complex, East Lansing, MI 48824-1122, USA Received 1 January 2004; accepted 1 January 2004
This is a very interesting paper, and it proposes the use of a technique that has been well used and tested within the process control community, that of control charts. An example is provided that demonstrates that this technique would have identified, as ‘‘out of control’’, the reported results for WorldCom, Rite Aid and Oxford Health Plans. The paper contends that while the demonstration was for quarterly data, continuous data streams could be used as in the statistical process control literature to identify potential areas to investigate. The contribution of this paper seems to be a combination of an audit suggestion and a demonstration. The suggestion portion states that audit should make use of techniques that are commonplace in the operations management area, statistical process controls and control charts. The demonstration is provided as a ‘‘proof of concept’’ of the technique. Three concerns result from this paper: first, is the application of a well-documented technique in other areas a significant contribution to the literature; second, is the demonstration appropriate; and third, what is the underlying theory? These questions are answered in reverse order. From a theoretical perspective, the use of control charts is well documented in operations management, but not in this paper, nor does this paper seek to provide any contributions beyond the idea that control charts ‘‘work.’’ The second question is the appropriateness of the proof of concept demonstration. The paper contends that control charts are appropriate for use in continuous monitoring (and that is the manner in which they are often used in production environments). Unfortunately, the paper demonstrates this using quarterly reported data for different account balances (or totals) for three different organizations. That would be a very good demonstration if the objective was to show the use of control charts in the analytical review phase of the audit. To have a proof of concept for the use of control charts in a
* Tel.: +1-517-432-2922; fax: +1-517-432-1101. E-mail address: [email protected] (S. Grabski). 1467-0895/$ - see front matter D 2004 Published by Elsevier Inc. doi:10.1016/j.accinf.2004.01.005
130
Discussion
continuous audit environment, the manuscript needs to both identify and show how a continuous data stream would be identified and analyzed (i.e., What data? What is in control? What is out of control? What is the underlying distributional shape? Is there an alerting facility for internal audit? How would this be embedded in new systems or added into existing ERP (or ERP-type) systems?). The paper must also demonstrate that this technique is ‘‘better’’ (regardless of how better is defined, e.g., less costly, more accurate, etc.) than other techniques that have been proposed in the continuous (and discrete) audit environments. This could be addressed through a simulation in which a known set of errors is seeded and the efficacy of the continuous monitoring system and control charts could be examined. Are control charts better than other techniques in determining that an investigation is needed (such as a consistent drift away from a random error amount that indicated the need to examine a production process that was still ‘‘in control’’ but was tending to go out of control in a production setting)? Are traditional analytical review techniques just as effective as control charts? Is a simple notification system (when a balance/transaction exceeds a certain amount) better than control charts? In conclusion, the ideas presented in this paper are very good, but need to be developed more fully. The use of quarterly financial data is not the same as how one would operationalize a continuous data stream. The identification of the data, its capture, analysis, standard setting, and so forth must be addressed. Even with that done, the authors need to document why this method would be better than any other methods that have been proposed.