- Email: [email protected]
A preliminary framework in examining the influence of outcome information on evaluations of auditor decisions
A PRELIMINARY FRAMEWORK IN EXAMINING THE INFLUENCE OF OUTCOME INFORMATION ON EVALUATIONS OF AUDITOR DECISIONS D. Jordan Lowe and Philip M . J . Reckers
ABSTRACT The expectation gap can be considered as a function of two components . The first component represents the difference between ex ante judgments of auditors and nonauditors who do not have outcome knowledge . The second component depicts the difference between nonauditor judgments that are made with and without negative outcome knowledge . Kinney and Nelson (1996) suggest that these two components tend to offset each other resulting in little if any expectation gap . We build upon their study and provide a framework that shows that the effects of these two components is dependent upon the type of judgment elicited. Our preliminary framework suggests that if the judgment involves a likelihood or predictive assessment, then the two expectation gap components may be expected to offset each other. However, if the judgment involves performance evaluation, the two components are more likely to compound each other . An understanding of this framework is important so as not to lead to inappropriate conclusions about the expectation gap . Advances in Accounting, Volume 19, pages 177-187 . Copyright m 2002 by Elsevier Science Ltd. All rights of reproduction in any form reserved . ISBN: 0-7623-0871-0 177
178
D . JORDAN LOWE AND PHILIP M . J . RECKERS
The expectation gap has been attributed to the differences between auditors' ex ante judgments at the time of the audit and the judgments of ex post evaluators who have the benefit of outcome knowledge . The expectation gap can be separated into two components - the differences in perceptions between auditors and nonauditors and the unfavorable consequences of outcome knowledge . Kinney and Nelson (1996) (hereafter KN) suggest that these components individually contribute to an expectation gap but that together their effects are offsetting. In this paper we first review the various aspects of KN and discuss their methodological contributions . Second, we build upon their initial conclusions to show in which contexts their conclusions are likely to hold . Third, we provide additional evidence to show how the effects of these two components are likely to either offset or compound each other depending on the type of judgment elicited. Finally, we propose a framework in which to examine the influence of outcome information on evaluations of auditor decisions .
OVERVIEW OF KN KN designed an experiment to examine a possible expectation gap between auditors' ex ante judgments and ex post judgments of nonauditors (e .g . judges, jurors, regulators) . The authors suggest that the expectation gap may be due to two components . The first component represents the difference between ex ante judgments of auditors and nonauditors who do not have outcome knowledge . The second component represents the difference between nonauditor judgments that are made with and without negative outcome knowledge. The purpose of their study was to disentangle these components to determine whether the expectation gap was primarily due to the : (1) effect of subject group differences on ex ante judgments, or the (2) effect of (negative) outcome knowledge (see Fig . 1) . While they identified some studies that have examined the first component (e .g . Anderson et al ., 1993a, b), and second component (e .g . Lowe & Reckers, 1994 ; Reimers & Butler, 1992), KN represents the first study to examine both components such that auditors' ex ante judgments are compared to nonauditors' ex post judgments . The division of judgments into these components is important in resolving the expectation gap because mitigation procedures would be different depending on which component is the primary contributor to the expectation gap .' KN utilized a litigation case scenario in which subjects were asked to assess the degree to which a potential future loss should be referenced in the auditor's report, as specified by SAS No . 58 . They utilized 70 audit seniors (with 39 months of experience) from a single Big 5 CPA firm and 69 investigators at the General Accounting Office (GAO) . Their results indicate for the first
Evaluations of Auditor Decisions
179
Expectation Gap
Auditors Ex ante Judgments
Component One : Subject Group Differences
Fig . 1 .
Nonauditors Ex ante Judgments
Nonauditors Ex post Judgments
Component Two : Outcome Knowledge Differences
Kinney and Nelson (1996) - The Decomposition of the Expectation Gap .
component that auditors assessed a significantly higher need for report reference than did GAO personnel (see Table 1) . That is, without the benefit of outcome knowledge, auditors were more conservative in their judgments than were n onauditors . KN suggest that auditors are more conservative than nonauditors due to auditors' sensitivity to litigation exposure. For the second component, GAO personnel provided ex post judgments that were significantly greater than their ex ante judgments, indicating the influence of negative outcome knowledge . Of most importance from a legal perspective is the examination of the net effect of these two components . KN found that auditors' judgments made without outcome knowledge were insignificantly different from judgments made by nonauditors with knowledge of a negative outcome . The authors suggest that auditors' ex ante conservatism could offset the effects of negative outcome knowledge on nonauditors and thereby reduce auditors' exposure to the unfavorable consequences of outcome knowledge (e .g . censure, litigation) . KN have made a significant contribution to the understanding of the expectation gap by disentangling the components of subject group and outcome knowledge . Their study should set a methodological standard for future research . We believe that while their results are noteworthy, they may only be generalizable for one type of judgment . That is, ex post evaluators often have to make two types of judgments : (1) likelihood judgments of a decision or an event,
1 80
D. JORDAN LOWE AND PHILIP M. J . RECKERS
Table 1 . Kinney and Nelson (1996) GAO Personnel Likelihood Assessments of SAS No . 58 Compliance . Treatment Meansa, (Std . Dev .), and Cell Sizes Auditors No Outcome 5 .21 (4.12) n=23
(1 )e
GAO Personnel No Outcome
(2)`
2 .45 (5 .95) n=22
GAO Personnel Negative Outcome 5 .13 (3 .82) n=23
Mean assessment of the degree to which proper application of SAS No . 58 requires explanatory paragraph . Subjects provided responses on a 21-point scale . The lower (upper) end of the scale was marked as -10 (+10) with the description for each number being, SAS No . 58 definitely does not (does) require the paragraph to be included in the audit report . I The first component of the expectation gap represents the difference between ex ante judgments of auditors and nonauditors who do not have outcome knowledge . ° The second component of the expectation gap represents the differences between nonauditor judgments that are made with and without negative outcome knowledge .
and (2) direct evaluation judgments of the defendant's performance . While likelihood judgments are assessed in a court of law (and may be fundamental to other judgments), it is the performance evaluation judgments that are often perceived of as being most important .2 The dependent measure that KN utilized most closely resembles a likelihood assessment rather than a performance appraisal (as suggested by an author of KN) .3 Therefore, we believe that their conclusions are appropriate for likelihood judgments but not necessarily for evaluation judgments . The differentiation of judgments is important given that research has shown that the influence of outcome knowledge depends on the type of judgment elicited . That is, negative outcome information has been shown to have the effect of increasing likelihood judgments and decreasing evaluation judgments (Kadous, 1996 ; Lowe, 1992) . Thus, negative outcome information influences these two judgments to move in the opposite direction . This is critical as the direction of this influence may determine the extent of the expectation gap .
COMPOUNDING EFFECTS OF EXPECTATION GAP COMPONENTS We reason that in an audit legal liability context in which performance evaluations are made, the effect of the two components is likely to compound rather than offset each other . Regarding the first component, we would expect
Evaluations of Auditor Decisions
181
that auditors would provide higher ex ante evaluations of the specified auditor than would nonauditors . This assertion is consistent with an extensive body of literature which has demonstrated that observers tend to assign lower evaluations to the actor than actors will to themselves or to their fellow actors (see Watson, 1982 for a review) . The implication here is that the evaluation of a peer is reflective on one's own performance . Consider the case of an auditor and a nonauditor evaluating the performance of a specified auditor . The auditor and the nonauditor would be expected to provide different evaluations due to their differential knowledge and attitudes toward the accounting profession . Lowe and Pany (1993) examined jurors and auditors' attitudes toward the accounting profession . For each question given, jurors had significantly less favorable attitudes toward the accounting profession than did auditors. Jennings et al . (1993) and Lee et al . (1998) found that these differences in attitudes toward the profession were directly related to subsequent evaluations of auditor performance . Other research has also shown that auditors tend to be less critical than nonauditors in their performance evaluations of other auditors . For instance, Arrington et al . (1983) revealed that, compared to small business owners, auditors attributed significantly less responsibility for audit failures to the auditors involved . Lowe (1992) and Anderson et al . (1993b) found that auditors provided significantly higher evaluation judgments of the specified auditor's performance than did respective jurors and judges . For the second component, we would expect negative outcome knowledge to influence nonauditors such that their ex post evaluation judgments would be lower than their ex ante judgments . This is based on theory developed within psychology (Fischhoff, 1975 ; Rachlinski, 1998) as well as numerous studies that have confirmed the robust nature of outcome effects (see Hawkins & Hastie, 1990 for a review) . When outcome knowledge is available before performance is evaluated, the evaluation is conducted from the perspective of hindsight . Evaluators in an ex post position may systematically perceive that there was a predictable sequence of events leading to the obtained outcome (Janoff-Bulman et al ., 1985) . Evaluators may assess whether they would have foreseen the likelihood of an event and consequently project that the decision maker ex ante (foresight) should have foreseen or anticipated what became clear ex post (hindsight) . Research has shown that (negative) outcome information has influenced jurors' evaluations of auditors in their performance of inventory and going concern tasks (e .g . Anderson & Reckers, 1998 ; Kadous, 1996 ; Lowe & Reckers, 1994) . Similar results have been found in related auditor contexts with judges (e .g . Anderson et al ., 1993a ; Jennings et al ., 1998) . Combining these two components, we would expect : (1) auditors to provide higher ex ante evaluation judgments of their peers than nonauditors, and (2)
182
D . JORDAN LOWE AND PHILIP M . J. RECKERS
nonauditors to have higher ex ante judgments than ex post judgments (see Fig . 1) . Thus, rather than having the two components offset each other (as shown in KN), we would expect that these effects would be compounded in an audit legal liability scenario in which performance evaluations take place . Lowe and Reckers (1994) To test our assertions, we examine and extend two prior studies . Lowe and Reckers (1994) was specifically identified by KN as a study that utilized only one subject group (jurors) and therefore was not able to assess both components . This study lacked a cell in which auditors made ex ante evaluation judgments . Utilizing the same case instrument as shown in Lowe and Reckers (1994), we obtained additional data from 20 audit seniors from a Big 5 CPA firm. These auditors had approximately 38 months of experience, or essentially the same experience as the auditor group in KN . With the addition of this cell, we were able to construct and examine the two components identified by KN . In this study, subjects were given a situation in which there was conflicting evidence regarding whether the audit client would be able to continue as a going concern . Subjects were then asked to evaluate the audit partner's judgment that the audit client would continue to exist .' An 11-point scale was used, anchored by 0 (poor judgment) and 10 (very good judgment) . A one-way ANOVA was performed with auditors/no outcome, jurors/no outcome, and jurors/negative outcome serving as the levels of the independent measure . These levels correspond to the design used in KN (see Fig . 1) . As shown in Table 2, the ANOVA was significant (F = 13 .29, p = 0.0001) . One-tailed a priori contrasts were utilized to examine the specific differences between levels . For component one, we found that auditors (6 .20) provided significantly higher (t 81 = 1 .99, p = 0.025) ex ante evaluation judgments than their juror counterparts (4 .91) . Regarding component two, jurors provided significantly lower evaluations (t 81 = 3 .40, p < 0.001) when told of a negative outcome (2 .97) than when they had no outcome knowledge (4 .91) . Finally, when we combined the two components together we found that as expected the effects were compounded such that nonauditors ex post judgments were significantly lower (t81 = 4 .97, p < 0 .0001) than the ex ante judgments made by auditors . Anderson et al . (1997) To provide additional support for our assertions we examined a second study to determine if similar results would prevail . We chose to use Anderson et al . (1997) given that it utilizes a different task, employs another group of
183
Evaluations of Auditor Decisions
Table 2. Lowe and Reckers (1994) Jurors' Evaluation of Auditor Decisions . Panel A . ANOVA Table : Source of Variation
Sum of Squares
Df
Group Error Total
138 .06 420.89 558 .95
2 81 83
Mean Squares 69 .03 5 .20
Significance of F
F 13 .29
0 .0001
Panel B. Treatment Means', (Std. Dev .), and Cell Sizes : Auditors No Outcome 6 .20 (2.44) n=20
(1)e
Jurors No Outcome 4 .91 (2.54) n=32
(2)c
Jurors Negative Outcome 2.97 (1 .86) n=32
Mean assessment of the evaluation of the audit partner's judgment that the audit client would continue to exist . An 11-point scale was used, anchored by 0 (poor judgment) and 10 (very good judgment) . I The first component of the expectation gap represents the difference between ex ante judgments of auditors and nonauditors who do not have outcome knowledge . The second component of the expectation gap represents the differences between non-auditor judgments that are made with and without negative outcome knowledge .
non-auditors (judges), and also examines the influence of outcome information in an audit legal liability scenario . To examine the two components within the KN framework we added another cell that included auditors who did not receive outcome information . Thirty-three audit seniors from the Big 5 CPA firms were enlisted to participate in this study . The average experience level for this group was 34 .5 months which is only slightly lower than that for KN . The case instrument (see Anderson et al ., 1997) provided information which both supported and discounted the existence of inventory obsolescence . Subjects were asked to evaluate the audit partner's decision not to book losses due to possible inventory obsolescence . Responses were based on an 11-point scale, with endpoints of 1 (Very Inappropriate) to 11 (Very Appropriate) . A similar design was utilized with the levels of the independent measure consisting of auditors/no outcome, judges/no outcome, and judges/negative outcome . As indicated by Table 3, the ANOVA results reveal a significant effect (F = 6 .68, p = 0 .0019) . Examining component one, we found that ex ante evaluation judgments were marginally significantly higher (t 103 = 1 .44, p = 0 .077) for auditors (5 .97) than they were for judges (4 .98) . For component two, judges
184
D . JORDAN LOWE AND PHILIP M . J . RECKERS
Table 3. Anderson et al . (1997) Judges' Evaluation of Auditor Decisions . Panel A . ANOVA Table : Source of Variation
Sum of Squares
Group Error Total
Df 2 103 105
120 .13 926 .71 1046 .84
Mean Squares 60.07 8 .99
F
Significance of F
6.68
0 .0019
Panel B. Treatment Means', (Std. Dev .), and Cell Sizes: Auditors No Outcome 5 .97 (3 .14) n=33
(1 )b
Judges No Outcome 4 .98 (2.95) n=44
(2)`
Judges Negative Outcome 3 .21 (2 .91) n=29
'Mean assessment of the evaluation of the audit partner's decision not to book losses due to possible inventory obsolescence . Responses were based on an 11-point scale, with endpoints of -5 (Very Inappropriate) to +5 (Very Appropriate) . For ease of presentation and to be consistent with Anderson et al . this scale was then converted to a scale with endpoints of 1 and 11 . b The first component of the expectation gap represents the difference between ex ante judgments of auditors and nonauditors who do not have outcome knowledge . ` The second component of the expectation gap represents the differences between nonauditor judgments that are made with and without negative outcome knowledge .
provided significantly lower judgments (t 103 = 2.47, p = 0.008) when given a negative outcome (3 .21) than when they had no outcome knowledge (4 .98). When we combine these components we find that as expected nonauditors' ex post judgments were significantly lower (t103 = 3 .62, p < 0 .0001) than auditors' ex ante judgments . These highly significant differences suggest that the effects of these two components are again compounded .
DISCUSSION AND CONCLUSIONS The results of our study (as well as those for KN), provide a preliminary framework in which to understand and examine the influence of outcome information . We conclude that if the judgment involves a likelihood assessment, then the two expectation gap components may be expected to offset each other . This would particularly be the case for audits in which auditors believe that they face relatively high risk of litigation and therefore are more likely to provide conservative ex ante judgments . However, if the judgment involves performance
185
Evaluations of Auditor Decisions
evaluation, the components are more likely to compound each other . The higher ex ante evaluations provided by auditors are thus expected to magnify the effects of outcome knowledge . An understanding of this framework is important so as not to lead to inappropriate conclusions about the expectation gap . For instance, recent research (e .g . Almer, 1999 ; Butler et al ., 2000) has referred to KN in the formulation of their studies . This research has noted that the two components individually demonstrate an expectation gap, but taken together are offsetting . This conclusion may suggest to some that the expectation gap is not a problem as the two components are expected to counteract each other . However, as shown in our study, conclusions regarding the expectation gap may depend on the type of judgment elicited . This knowledge is critical in a court of law as ex post evaluators armed with negative outcome knowledge may be less than generous in their evaluations of auditors .
NOTES 1 . If the expectation gap is found to be caused by differential (ex ante) perceptions of auditors and nonauditors then education measures would be needed in an effort to equate subject attitudes (Anderson et al ., 1993b) . Conversely, if the gap is due to the influence of outcome knowledge then outcome mitigation measures would be needed (see e .g . Anderson et al ., 1997 ; Kennedy, 1995 ; Lowe & Reckers, 1994 ; Wexier & Schopp, 1989) . 2 . For this reason, studies examining the influence of outcome knowledge in audit legal liability scenarios have often utilized performance evaluation judgments (e.g . Anderson et al ., 1993a, b ; Jennings et al ., 1998 ; Wermert, 1998) . 3 . Subjects assessed the degree (or likelihood) to which they believed that a potential future loss should be referenced in the auditor's report . The judgment in KN is characterized as a likelihood judgment given that subjects were asked to provide an assessment of a possible action rather than a direct evaluation of the performance of the specified auditor (Kinney & Nelson, 1996, p . 289) . 4 . In an actual case involving alleged external auditor negligence, the appropriateness of the auditor's decision would be considered in determining whether the auditor was culpable and liable . If found liable (guilty), the court would have to decide what damages should be awarded to the plaintiff . However, in this study (as well as other related studies), the control group does not receive outcome information . Therefore, it was not possible to ask all subjects specific guilt and damage award questions . It was feasible to ask subjects to evaluate the appropriateness of the auditors' decision .
ACKNOWLEDGMENTS The authors wish to acknowledge the helpful comments and suggestions of Mark Nelson, Jane Kennedy, Marlys Lipe, and Peter Booth . We also
186
D . JORDAN LOWE AND PHILIP M . J. RECKERS
appreciate comments from participants at the 1999 Accounting, Behavior, and Organizations Research Conference and the 2000 American Accounting Association Annual Meeting .
REFERENCES Almer, E . D . (1999). The Role of Context in the Expectation Gap : Exploring Going Concern Communications. Working Paper, University of Miami . Anderson, B ., Maletta, M ., & Wright, A. (1998). Perceptions of Auditor Responsibility : Views of the Judiciary and the Profession . The International Journal of Auditing, 2(3), 215-232. Anderson, J. C ., Jennings, M . M ., Lowe, D . J., & Reckers, P . M. J. (1997) . The Mitigation of Hindsight Bias in Judges' Evaluation of Auditor Decisions . Auditing : A Journal of Practice and Theory, 18(2), 20-39 . Anderson, J . C., Jennings, M . M ., & Reckers, P . M. J . (1993a) . The Presence of Hindsight Bias in Peer and Judicial Evaluation in Public Accounting Litigation . Tort and Insurance Law Journal, 28(3), 462-479 . Anderson, J . C ., Lowe, D . J ., & Reckers, P . M . J . (1993b). Evaluation of Auditor Decisions : Hindsight Bias Effects and the Expectation Gap . Journal of Economic Psychology, 14(4), 711-737 . Anderson, J . C., & Reckers, P . M. J . (1998) . Mitigating Hindsight Bias in Jurors' Evaluation of Auditor Decisions : Considering Alternative Outcomes and the Education of Jurors . Advances in Accounting, 16, 221-237 . Arrington, C. E ., Hillison, W., & Williams, P . (1983) . The Psychology of the Expectation Gap: Why is there so Much Dispute About Auditor Responsibility? Accounting and Business Research, 13(52), 243-250. Butler, S . A ., Ward, B . H., & Zimbelman, M . F. (2000) . The Expectation Gap : Auditors' and Investors' Perceptions of Auditors' Fraud Detection Responsibilities . Working Paper, University of Oklahoma. Fischhoff, B . (1975) . Hindsight : Foresight : The Effect of Outcome Knowledge on Judgment Under Uncertainty . Journal of Experimental Psychology : Human Perception and Performance, 1(3), 288-299 . Hawkins, S . A., & Hastie, R. (1990) . Hindsight: Biased Judgments of Past Events After the Outcomes are Known . Psychological Bulletin, 107(3), 311-327 . Janoff-Bulman, R ., Timko, C ., & Carli, L. L . (1985) . Cognitive Biases in Blaming the Victim. Journal of Experimental Social Psychology, 21(2), 161-177 . Jennings, M. M ., Kneer, D . C., & Reckers, P . M. J . (1993). The Significance of Audit Decision Aids and Pre-case Jurists' Attitudes on Perceptions of Audit Firm Culpability and Liability . Contemporary Accounting Research, 9(2), 489-507 . Jennings, M . M ., Lowe, D. J., & Reckers, P. M . J . (1998) . Causality as an Influence on Hindsight Bias : An Empirical Examination of Judges' Evaluations of Professional Audit Judgment . Journal of Accounting and Public Policy, 21, 143-167 . Kadous, K. (1996) . Juror Decision Making in Professional Liability Cases : A Model and Test in the Audit Litigation Setting . Doctoral Dissertation, University of Illinois . Kennedy, J . (1995) . Debiasing the Curse of Knowledge in Audit Judgment . The Accounting Review, 70(2), 249-273 .
Evaluations of Auditor Decisions
187
Kinney, W . R . Jr ., & Nelson, M . W . (1996) . Outcome Information and the Expectation Gap: The Case of Loss Contingencies . Journal of Accounting Research, 34(2), 281-299 . Lee, T., Lowe, D . J ., & Reckers, P . M . J. (1998). The Influence of Moral Development and Audit Expectations in the Mitigation of Hindsight Bias . Advances in Accounting, 16, 239-252 . Lowe, D . J. (1992) . An Empirical Examination of the Hindsight Bias Phenomenon in Evaluation of Auditor Decisions . Doctoral Dissertation, Arizona State University . Lowe, D . J ., & Pany, K . (1993) . Expectations of the Audit Function : A Comparison of Perceptions of Auditors and Jurors . CPA Journal, (August), 58-59. Lowe, D . J ., & Reckers, P . M . J . (1994) . The Effects of Hindsight Bias on Jurors' Evaluations of Auditor Decisions . Decision Sciences, 25(3), 401-426 . Rachlinski, J. J. (1998). A Positive Psychological Theory of Judging in Hindsight . The University of Chicago Law Review, 65(2), 571-625 . Reimers J. L ., & Butler, S . A . (1992) . The Effect of Outcome Knowledge on Auditors' Judgmental Evaluations . Accounting, Organizations and Society, 17(2), 185-194 . Watson, D . (1982). The Actor and the Observer : How are their Perceptions of Causality Divergent? Psychological Bulletin, 92, 683-700 . Wermert, J . G . (1998) . The Role of Outcome Information in the Evaluation of Auditor Performance in Trial Settings : Further Evidence . Working Paper, Drake University . Wexler, D . B ., Schopp, R . F. (1989) . How and When to Correct for Juror Hindsight Bias in Mental Health Malpractice Litigation : Some Preliminary Observations . Behavioral Sciences and the Law, 7(4), 485-504 .
ABSTRACT The expectation gap can be considered as a function of two components . The first component represents the difference between ex ante judgments of auditors and nonauditors who do not have outcome knowledge . The second component depicts the difference between nonauditor judgments that are made with and without negative outcome knowledge . Kinney and Nelson (1996) suggest that these two components tend to offset each other resulting in little if any expectation gap . We build upon their study and provide a framework that shows that the effects of these two components is dependent upon the type of judgment elicited. Our preliminary framework suggests that if the judgment involves a likelihood or predictive assessment, then the two expectation gap components may be expected to offset each other. However, if the judgment involves performance evaluation, the two components are more likely to compound each other . An understanding of this framework is important so as not to lead to inappropriate conclusions about the expectation gap . Advances in Accounting, Volume 19, pages 177-187 . Copyright m 2002 by Elsevier Science Ltd. All rights of reproduction in any form reserved . ISBN: 0-7623-0871-0 177
178
D . JORDAN LOWE AND PHILIP M . J . RECKERS
The expectation gap has been attributed to the differences between auditors' ex ante judgments at the time of the audit and the judgments of ex post evaluators who have the benefit of outcome knowledge . The expectation gap can be separated into two components - the differences in perceptions between auditors and nonauditors and the unfavorable consequences of outcome knowledge . Kinney and Nelson (1996) (hereafter KN) suggest that these components individually contribute to an expectation gap but that together their effects are offsetting. In this paper we first review the various aspects of KN and discuss their methodological contributions . Second, we build upon their initial conclusions to show in which contexts their conclusions are likely to hold . Third, we provide additional evidence to show how the effects of these two components are likely to either offset or compound each other depending on the type of judgment elicited. Finally, we propose a framework in which to examine the influence of outcome information on evaluations of auditor decisions .
OVERVIEW OF KN KN designed an experiment to examine a possible expectation gap between auditors' ex ante judgments and ex post judgments of nonauditors (e .g . judges, jurors, regulators) . The authors suggest that the expectation gap may be due to two components . The first component represents the difference between ex ante judgments of auditors and nonauditors who do not have outcome knowledge . The second component represents the difference between nonauditor judgments that are made with and without negative outcome knowledge. The purpose of their study was to disentangle these components to determine whether the expectation gap was primarily due to the : (1) effect of subject group differences on ex ante judgments, or the (2) effect of (negative) outcome knowledge (see Fig . 1) . While they identified some studies that have examined the first component (e .g . Anderson et al ., 1993a, b), and second component (e .g . Lowe & Reckers, 1994 ; Reimers & Butler, 1992), KN represents the first study to examine both components such that auditors' ex ante judgments are compared to nonauditors' ex post judgments . The division of judgments into these components is important in resolving the expectation gap because mitigation procedures would be different depending on which component is the primary contributor to the expectation gap .' KN utilized a litigation case scenario in which subjects were asked to assess the degree to which a potential future loss should be referenced in the auditor's report, as specified by SAS No . 58 . They utilized 70 audit seniors (with 39 months of experience) from a single Big 5 CPA firm and 69 investigators at the General Accounting Office (GAO) . Their results indicate for the first
Evaluations of Auditor Decisions
179
Expectation Gap
Auditors Ex ante Judgments
Component One : Subject Group Differences
Fig . 1 .
Nonauditors Ex ante Judgments
Nonauditors Ex post Judgments
Component Two : Outcome Knowledge Differences
Kinney and Nelson (1996) - The Decomposition of the Expectation Gap .
component that auditors assessed a significantly higher need for report reference than did GAO personnel (see Table 1) . That is, without the benefit of outcome knowledge, auditors were more conservative in their judgments than were n onauditors . KN suggest that auditors are more conservative than nonauditors due to auditors' sensitivity to litigation exposure. For the second component, GAO personnel provided ex post judgments that were significantly greater than their ex ante judgments, indicating the influence of negative outcome knowledge . Of most importance from a legal perspective is the examination of the net effect of these two components . KN found that auditors' judgments made without outcome knowledge were insignificantly different from judgments made by nonauditors with knowledge of a negative outcome . The authors suggest that auditors' ex ante conservatism could offset the effects of negative outcome knowledge on nonauditors and thereby reduce auditors' exposure to the unfavorable consequences of outcome knowledge (e .g . censure, litigation) . KN have made a significant contribution to the understanding of the expectation gap by disentangling the components of subject group and outcome knowledge . Their study should set a methodological standard for future research . We believe that while their results are noteworthy, they may only be generalizable for one type of judgment . That is, ex post evaluators often have to make two types of judgments : (1) likelihood judgments of a decision or an event,
1 80
D. JORDAN LOWE AND PHILIP M. J . RECKERS
Table 1 . Kinney and Nelson (1996) GAO Personnel Likelihood Assessments of SAS No . 58 Compliance . Treatment Meansa, (Std . Dev .), and Cell Sizes Auditors No Outcome 5 .21 (4.12) n=23
(1 )e
GAO Personnel No Outcome
(2)`
2 .45 (5 .95) n=22
GAO Personnel Negative Outcome 5 .13 (3 .82) n=23
Mean assessment of the degree to which proper application of SAS No . 58 requires explanatory paragraph . Subjects provided responses on a 21-point scale . The lower (upper) end of the scale was marked as -10 (+10) with the description for each number being, SAS No . 58 definitely does not (does) require the paragraph to be included in the audit report . I The first component of the expectation gap represents the difference between ex ante judgments of auditors and nonauditors who do not have outcome knowledge . ° The second component of the expectation gap represents the differences between nonauditor judgments that are made with and without negative outcome knowledge .
and (2) direct evaluation judgments of the defendant's performance . While likelihood judgments are assessed in a court of law (and may be fundamental to other judgments), it is the performance evaluation judgments that are often perceived of as being most important .2 The dependent measure that KN utilized most closely resembles a likelihood assessment rather than a performance appraisal (as suggested by an author of KN) .3 Therefore, we believe that their conclusions are appropriate for likelihood judgments but not necessarily for evaluation judgments . The differentiation of judgments is important given that research has shown that the influence of outcome knowledge depends on the type of judgment elicited . That is, negative outcome information has been shown to have the effect of increasing likelihood judgments and decreasing evaluation judgments (Kadous, 1996 ; Lowe, 1992) . Thus, negative outcome information influences these two judgments to move in the opposite direction . This is critical as the direction of this influence may determine the extent of the expectation gap .
COMPOUNDING EFFECTS OF EXPECTATION GAP COMPONENTS We reason that in an audit legal liability context in which performance evaluations are made, the effect of the two components is likely to compound rather than offset each other . Regarding the first component, we would expect
Evaluations of Auditor Decisions
181
that auditors would provide higher ex ante evaluations of the specified auditor than would nonauditors . This assertion is consistent with an extensive body of literature which has demonstrated that observers tend to assign lower evaluations to the actor than actors will to themselves or to their fellow actors (see Watson, 1982 for a review) . The implication here is that the evaluation of a peer is reflective on one's own performance . Consider the case of an auditor and a nonauditor evaluating the performance of a specified auditor . The auditor and the nonauditor would be expected to provide different evaluations due to their differential knowledge and attitudes toward the accounting profession . Lowe and Pany (1993) examined jurors and auditors' attitudes toward the accounting profession . For each question given, jurors had significantly less favorable attitudes toward the accounting profession than did auditors. Jennings et al . (1993) and Lee et al . (1998) found that these differences in attitudes toward the profession were directly related to subsequent evaluations of auditor performance . Other research has also shown that auditors tend to be less critical than nonauditors in their performance evaluations of other auditors . For instance, Arrington et al . (1983) revealed that, compared to small business owners, auditors attributed significantly less responsibility for audit failures to the auditors involved . Lowe (1992) and Anderson et al . (1993b) found that auditors provided significantly higher evaluation judgments of the specified auditor's performance than did respective jurors and judges . For the second component, we would expect negative outcome knowledge to influence nonauditors such that their ex post evaluation judgments would be lower than their ex ante judgments . This is based on theory developed within psychology (Fischhoff, 1975 ; Rachlinski, 1998) as well as numerous studies that have confirmed the robust nature of outcome effects (see Hawkins & Hastie, 1990 for a review) . When outcome knowledge is available before performance is evaluated, the evaluation is conducted from the perspective of hindsight . Evaluators in an ex post position may systematically perceive that there was a predictable sequence of events leading to the obtained outcome (Janoff-Bulman et al ., 1985) . Evaluators may assess whether they would have foreseen the likelihood of an event and consequently project that the decision maker ex ante (foresight) should have foreseen or anticipated what became clear ex post (hindsight) . Research has shown that (negative) outcome information has influenced jurors' evaluations of auditors in their performance of inventory and going concern tasks (e .g . Anderson & Reckers, 1998 ; Kadous, 1996 ; Lowe & Reckers, 1994) . Similar results have been found in related auditor contexts with judges (e .g . Anderson et al ., 1993a ; Jennings et al ., 1998) . Combining these two components, we would expect : (1) auditors to provide higher ex ante evaluation judgments of their peers than nonauditors, and (2)
182
D . JORDAN LOWE AND PHILIP M . J. RECKERS
nonauditors to have higher ex ante judgments than ex post judgments (see Fig . 1) . Thus, rather than having the two components offset each other (as shown in KN), we would expect that these effects would be compounded in an audit legal liability scenario in which performance evaluations take place . Lowe and Reckers (1994) To test our assertions, we examine and extend two prior studies . Lowe and Reckers (1994) was specifically identified by KN as a study that utilized only one subject group (jurors) and therefore was not able to assess both components . This study lacked a cell in which auditors made ex ante evaluation judgments . Utilizing the same case instrument as shown in Lowe and Reckers (1994), we obtained additional data from 20 audit seniors from a Big 5 CPA firm. These auditors had approximately 38 months of experience, or essentially the same experience as the auditor group in KN . With the addition of this cell, we were able to construct and examine the two components identified by KN . In this study, subjects were given a situation in which there was conflicting evidence regarding whether the audit client would be able to continue as a going concern . Subjects were then asked to evaluate the audit partner's judgment that the audit client would continue to exist .' An 11-point scale was used, anchored by 0 (poor judgment) and 10 (very good judgment) . A one-way ANOVA was performed with auditors/no outcome, jurors/no outcome, and jurors/negative outcome serving as the levels of the independent measure . These levels correspond to the design used in KN (see Fig . 1) . As shown in Table 2, the ANOVA was significant (F = 13 .29, p = 0.0001) . One-tailed a priori contrasts were utilized to examine the specific differences between levels . For component one, we found that auditors (6 .20) provided significantly higher (t 81 = 1 .99, p = 0.025) ex ante evaluation judgments than their juror counterparts (4 .91) . Regarding component two, jurors provided significantly lower evaluations (t 81 = 3 .40, p < 0.001) when told of a negative outcome (2 .97) than when they had no outcome knowledge (4 .91) . Finally, when we combined the two components together we found that as expected the effects were compounded such that nonauditors ex post judgments were significantly lower (t81 = 4 .97, p < 0 .0001) than the ex ante judgments made by auditors . Anderson et al . (1997) To provide additional support for our assertions we examined a second study to determine if similar results would prevail . We chose to use Anderson et al . (1997) given that it utilizes a different task, employs another group of
183
Evaluations of Auditor Decisions
Table 2. Lowe and Reckers (1994) Jurors' Evaluation of Auditor Decisions . Panel A . ANOVA Table : Source of Variation
Sum of Squares
Df
Group Error Total
138 .06 420.89 558 .95
2 81 83
Mean Squares 69 .03 5 .20
Significance of F
F 13 .29
0 .0001
Panel B. Treatment Means', (Std. Dev .), and Cell Sizes : Auditors No Outcome 6 .20 (2.44) n=20
(1)e
Jurors No Outcome 4 .91 (2.54) n=32
(2)c
Jurors Negative Outcome 2.97 (1 .86) n=32
Mean assessment of the evaluation of the audit partner's judgment that the audit client would continue to exist . An 11-point scale was used, anchored by 0 (poor judgment) and 10 (very good judgment) . I The first component of the expectation gap represents the difference between ex ante judgments of auditors and nonauditors who do not have outcome knowledge . The second component of the expectation gap represents the differences between non-auditor judgments that are made with and without negative outcome knowledge .
non-auditors (judges), and also examines the influence of outcome information in an audit legal liability scenario . To examine the two components within the KN framework we added another cell that included auditors who did not receive outcome information . Thirty-three audit seniors from the Big 5 CPA firms were enlisted to participate in this study . The average experience level for this group was 34 .5 months which is only slightly lower than that for KN . The case instrument (see Anderson et al ., 1997) provided information which both supported and discounted the existence of inventory obsolescence . Subjects were asked to evaluate the audit partner's decision not to book losses due to possible inventory obsolescence . Responses were based on an 11-point scale, with endpoints of 1 (Very Inappropriate) to 11 (Very Appropriate) . A similar design was utilized with the levels of the independent measure consisting of auditors/no outcome, judges/no outcome, and judges/negative outcome . As indicated by Table 3, the ANOVA results reveal a significant effect (F = 6 .68, p = 0 .0019) . Examining component one, we found that ex ante evaluation judgments were marginally significantly higher (t 103 = 1 .44, p = 0 .077) for auditors (5 .97) than they were for judges (4 .98) . For component two, judges
184
D . JORDAN LOWE AND PHILIP M . J . RECKERS
Table 3. Anderson et al . (1997) Judges' Evaluation of Auditor Decisions . Panel A . ANOVA Table : Source of Variation
Sum of Squares
Group Error Total
Df 2 103 105
120 .13 926 .71 1046 .84
Mean Squares 60.07 8 .99
F
Significance of F
6.68
0 .0019
Panel B. Treatment Means', (Std. Dev .), and Cell Sizes: Auditors No Outcome 5 .97 (3 .14) n=33
(1 )b
Judges No Outcome 4 .98 (2.95) n=44
(2)`
Judges Negative Outcome 3 .21 (2 .91) n=29
'Mean assessment of the evaluation of the audit partner's decision not to book losses due to possible inventory obsolescence . Responses were based on an 11-point scale, with endpoints of -5 (Very Inappropriate) to +5 (Very Appropriate) . For ease of presentation and to be consistent with Anderson et al . this scale was then converted to a scale with endpoints of 1 and 11 . b The first component of the expectation gap represents the difference between ex ante judgments of auditors and nonauditors who do not have outcome knowledge . ` The second component of the expectation gap represents the differences between nonauditor judgments that are made with and without negative outcome knowledge .
provided significantly lower judgments (t 103 = 2.47, p = 0.008) when given a negative outcome (3 .21) than when they had no outcome knowledge (4 .98). When we combine these components we find that as expected nonauditors' ex post judgments were significantly lower (t103 = 3 .62, p < 0 .0001) than auditors' ex ante judgments . These highly significant differences suggest that the effects of these two components are again compounded .
DISCUSSION AND CONCLUSIONS The results of our study (as well as those for KN), provide a preliminary framework in which to understand and examine the influence of outcome information . We conclude that if the judgment involves a likelihood assessment, then the two expectation gap components may be expected to offset each other . This would particularly be the case for audits in which auditors believe that they face relatively high risk of litigation and therefore are more likely to provide conservative ex ante judgments . However, if the judgment involves performance
185
Evaluations of Auditor Decisions
evaluation, the components are more likely to compound each other . The higher ex ante evaluations provided by auditors are thus expected to magnify the effects of outcome knowledge . An understanding of this framework is important so as not to lead to inappropriate conclusions about the expectation gap . For instance, recent research (e .g . Almer, 1999 ; Butler et al ., 2000) has referred to KN in the formulation of their studies . This research has noted that the two components individually demonstrate an expectation gap, but taken together are offsetting . This conclusion may suggest to some that the expectation gap is not a problem as the two components are expected to counteract each other . However, as shown in our study, conclusions regarding the expectation gap may depend on the type of judgment elicited . This knowledge is critical in a court of law as ex post evaluators armed with negative outcome knowledge may be less than generous in their evaluations of auditors .
NOTES 1 . If the expectation gap is found to be caused by differential (ex ante) perceptions of auditors and nonauditors then education measures would be needed in an effort to equate subject attitudes (Anderson et al ., 1993b) . Conversely, if the gap is due to the influence of outcome knowledge then outcome mitigation measures would be needed (see e .g . Anderson et al ., 1997 ; Kennedy, 1995 ; Lowe & Reckers, 1994 ; Wexier & Schopp, 1989) . 2 . For this reason, studies examining the influence of outcome knowledge in audit legal liability scenarios have often utilized performance evaluation judgments (e.g . Anderson et al ., 1993a, b ; Jennings et al ., 1998 ; Wermert, 1998) . 3 . Subjects assessed the degree (or likelihood) to which they believed that a potential future loss should be referenced in the auditor's report . The judgment in KN is characterized as a likelihood judgment given that subjects were asked to provide an assessment of a possible action rather than a direct evaluation of the performance of the specified auditor (Kinney & Nelson, 1996, p . 289) . 4 . In an actual case involving alleged external auditor negligence, the appropriateness of the auditor's decision would be considered in determining whether the auditor was culpable and liable . If found liable (guilty), the court would have to decide what damages should be awarded to the plaintiff . However, in this study (as well as other related studies), the control group does not receive outcome information . Therefore, it was not possible to ask all subjects specific guilt and damage award questions . It was feasible to ask subjects to evaluate the appropriateness of the auditors' decision .
ACKNOWLEDGMENTS The authors wish to acknowledge the helpful comments and suggestions of Mark Nelson, Jane Kennedy, Marlys Lipe, and Peter Booth . We also
186
D . JORDAN LOWE AND PHILIP M . J. RECKERS
appreciate comments from participants at the 1999 Accounting, Behavior, and Organizations Research Conference and the 2000 American Accounting Association Annual Meeting .
REFERENCES Almer, E . D . (1999). The Role of Context in the Expectation Gap : Exploring Going Concern Communications. Working Paper, University of Miami . Anderson, B ., Maletta, M ., & Wright, A. (1998). Perceptions of Auditor Responsibility : Views of the Judiciary and the Profession . The International Journal of Auditing, 2(3), 215-232. Anderson, J. C ., Jennings, M . M ., Lowe, D . J., & Reckers, P . M. J. (1997) . The Mitigation of Hindsight Bias in Judges' Evaluation of Auditor Decisions . Auditing : A Journal of Practice and Theory, 18(2), 20-39 . Anderson, J . C., Jennings, M . M ., & Reckers, P . M. J . (1993a) . The Presence of Hindsight Bias in Peer and Judicial Evaluation in Public Accounting Litigation . Tort and Insurance Law Journal, 28(3), 462-479 . Anderson, J . C ., Lowe, D . J ., & Reckers, P . M . J . (1993b). Evaluation of Auditor Decisions : Hindsight Bias Effects and the Expectation Gap . Journal of Economic Psychology, 14(4), 711-737 . Anderson, J . C., & Reckers, P . M. J . (1998) . Mitigating Hindsight Bias in Jurors' Evaluation of Auditor Decisions : Considering Alternative Outcomes and the Education of Jurors . Advances in Accounting, 16, 221-237 . Arrington, C. E ., Hillison, W., & Williams, P . (1983) . The Psychology of the Expectation Gap: Why is there so Much Dispute About Auditor Responsibility? Accounting and Business Research, 13(52), 243-250. Butler, S . A ., Ward, B . H., & Zimbelman, M . F. (2000) . The Expectation Gap : Auditors' and Investors' Perceptions of Auditors' Fraud Detection Responsibilities . Working Paper, University of Oklahoma. Fischhoff, B . (1975) . Hindsight : Foresight : The Effect of Outcome Knowledge on Judgment Under Uncertainty . Journal of Experimental Psychology : Human Perception and Performance, 1(3), 288-299 . Hawkins, S . A., & Hastie, R. (1990) . Hindsight: Biased Judgments of Past Events After the Outcomes are Known . Psychological Bulletin, 107(3), 311-327 . Janoff-Bulman, R ., Timko, C ., & Carli, L. L . (1985) . Cognitive Biases in Blaming the Victim. Journal of Experimental Social Psychology, 21(2), 161-177 . Jennings, M. M ., Kneer, D . C., & Reckers, P . M. J . (1993). The Significance of Audit Decision Aids and Pre-case Jurists' Attitudes on Perceptions of Audit Firm Culpability and Liability . Contemporary Accounting Research, 9(2), 489-507 . Jennings, M . M ., Lowe, D. J., & Reckers, P. M . J . (1998) . Causality as an Influence on Hindsight Bias : An Empirical Examination of Judges' Evaluations of Professional Audit Judgment . Journal of Accounting and Public Policy, 21, 143-167 . Kadous, K. (1996) . Juror Decision Making in Professional Liability Cases : A Model and Test in the Audit Litigation Setting . Doctoral Dissertation, University of Illinois . Kennedy, J . (1995) . Debiasing the Curse of Knowledge in Audit Judgment . The Accounting Review, 70(2), 249-273 .
Evaluations of Auditor Decisions
187
Kinney, W . R . Jr ., & Nelson, M . W . (1996) . Outcome Information and the Expectation Gap: The Case of Loss Contingencies . Journal of Accounting Research, 34(2), 281-299 . Lee, T., Lowe, D . J ., & Reckers, P . M . J. (1998). The Influence of Moral Development and Audit Expectations in the Mitigation of Hindsight Bias . Advances in Accounting, 16, 239-252 . Lowe, D . J. (1992) . An Empirical Examination of the Hindsight Bias Phenomenon in Evaluation of Auditor Decisions . Doctoral Dissertation, Arizona State University . Lowe, D . J ., & Pany, K . (1993) . Expectations of the Audit Function : A Comparison of Perceptions of Auditors and Jurors . CPA Journal, (August), 58-59. Lowe, D . J ., & Reckers, P . M . J . (1994) . The Effects of Hindsight Bias on Jurors' Evaluations of Auditor Decisions . Decision Sciences, 25(3), 401-426 . Rachlinski, J. J. (1998). A Positive Psychological Theory of Judging in Hindsight . The University of Chicago Law Review, 65(2), 571-625 . Reimers J. L ., & Butler, S . A . (1992) . The Effect of Outcome Knowledge on Auditors' Judgmental Evaluations . Accounting, Organizations and Society, 17(2), 185-194 . Watson, D . (1982). The Actor and the Observer : How are their Perceptions of Causality Divergent? Psychological Bulletin, 92, 683-700 . Wermert, J . G . (1998) . The Role of Outcome Information in the Evaluation of Auditor Performance in Trial Settings : Further Evidence . Working Paper, Drake University . Wexler, D . B ., Schopp, R . F. (1989) . How and When to Correct for Juror Hindsight Bias in Mental Health Malpractice Litigation : Some Preliminary Observations . Behavioral Sciences and the Law, 7(4), 485-504 .