Modeling the determinants of audit expertise

Pergamon

A¢counttng~ Organizations and Society, Vol. 19, No. 8, pp. 701-716, 1994 Copyright © 1994 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0361--3682/94 $7.00+0.00

0361-3682(93)E0009-6

MODELING THE DETERMINANTS OF AUDIT EXPERTISE*

ROBERT LIBBY Cornell University

and H U N - T O N G TAN

N a n y a n g Technological University (Singapore)

Abstract

lAbby& Daft[AcmTtmttng,O r g a n ~

and Soc/ety( 1993 ) pp. 425-450 ] presented a model of the relations between experience, ability, knowledge, and performance in audit judgment This paper extends the model by developing a framework for predicting the structure of these relations in different judgment settings and provides an initial test of the predictions by analyzing data from Botmer & Lewis's ~otw'nal of Accounting Research (Supplement 1990) pp. 1-20] four ra~k-susing LISRELKey predictions were that problem-solving abilitywould directly affect performance only in unstructured tasks and would indirectly affect performance through its effect on knowledge acquisition where the learning eaqvironment was impoverished. The predicticea were supported in most cases. In addition, the paper provides and tests a basis for predicting the associations between performance on different audit t~ks, Construct measurement problems that need to be addressed in future research are also indicated.

B o n n e r & Lewis ( 1 9 9 0 ) t e s t e d t h e d e g r e e to w h i c h c r o s s - s e c t i o n a l v a r i a t i o n in j u d g m e n t p e r f o r m a n c e in f o u r a u d i t tasks c o u l d b e e x p l a i n e d by measures of knowledge and ability that have b e e n i d e n t i f i e d in t h e p s y c h o l o g y l i t e r a t u r e . T h e y first c o n s t r u c t e d a s e r i e s o f t e s t s for knowledge and ability that they hypothesized w e r e r e l e v a n t t o t h e f o u r tasks. T h e y t h e n b u i l t regression models of the relations of knowledge a n d a b i l i t y t o p e r f o r m a n c e o n t h e f o u r tasks using the test results along with a combination o f serf-reports o f k n o w l e d g e a n d g e n e r a l a n d specific e x p e r i e n c e as p r o x i e s for k n o w l e d g e n o t a s s e s s e d b y t h e tests ( B o n n e r & Lewis, 1990, p. 8). T h e causal m o d e l i m p l i c i t in t h e i r

analysis, in w h i c h a b i l i t y a n d k n o w l e d g e d i r e c t l y affect p e r f o r m a n c e , is p r e s e n t e d in Fig. l ( a ) . M e a s u r e s o f ability, e x p e r i e n c e , k n o w l e d g e , a n d p e r f o r m a n c e w e r e e m p l o y e d in this analysis, b u t t h e n a t u r e o f t h e r e l a t i o n s a m o n g t h e m w a s left as a s u g g e s t e d d i r e c t i o n for f u t u r e r e s e a r c h ( B o n n e r & Lewis, 1990, p. 6; s e e also M a r c h a n t , 1990).

Libby & Luft ( 1 9 9 3 ) suggested a m o r e complete model of these relations where: ( 1 ) there are two classes of inputs, abilities and experiences; ( 2 ) these two inputs result in an internal state of knowledge, which is an intermediate output variable; and ( 3 ) along with the direct effects of abilities, knowledge affects the

* The authors wish to thank Sarah Botmer and Barry Lewis for providing them with the data used in constructing their regression models to form the basis for the analysis presented here. Rick Bagozzi, Sarah Bonnet, Barry Lewis, Joan Luft, and Mark Nelson provided useful comments on prior versions. 701

702

R. LIBBYand HUN-TONG TAN

(a)

Fig.1. Conceptual models; (a) Bormer & Lewis (1990) model; (b) antecedents and consequences of knowledge (Libby & Luft, 1993).

o u t p u t variable, p e r f o r m a n c e ) This s t r u c t u r e is p r e s e n t e d in Fig. l ( b ) . T h e y f u r t h e r s u g g e s t t h a t s i n c e a u d i t j u d g m e n t tasks differ b o t h in opportunities to learn relevant knowledge, and in t h e abilities, k n o w l e d g e , a n d effort n e c e s s a r y for successful task c o m p l e t i o n , a s e p a r a t e v e r s i o n o f Fig. l ( b ) c o u l d b e c o n s t r u c t e d for e a c h task, a n d t h e s t r e n g t h o f t h e f o u r r e l a t i o n s in t h e m o d e l s h o u l d differ a c r o s s tasks. T h e d e g r e e o f s i m i l a r i t y in t h e s t r u c t u r e o f t h e s e r e l a t i o n s a c r o s s tasks s h o u l d also p r o v i d e a basis for s p e c i f y i n g t h e a s s o c i a t i o n s b e t w e e n p e r f o r m a n c e o n different a u d i t tasks.

E m p l o y i n g t h e m o r e c o m p l e t e m o d e l is i m p o r t a n t for t w o reasons. First, t r e a t i n g exp e r i e n c e o n l y as a p r o x y for k n o w l e d g e , as in t h e B o n n e r a n d Lewis analysis, i g n o r e s t h e importance of the knowledge acquisition proc e s s to t h e u n d e r s t a n d i n g o f t h e n a t u r e o f e x p e r t i s e a n d for t h e d e s i g n o f effective training, t a s k - a s s i g n m e n t p r o g r a m s , a n d d e c i s i o n aids. S e c o n d , s i n c e ability c a n affect p e r f o r m a n c e b o t h d i r e c t l y , as in t h e B o n n e r a n d Lewis analysis, a n d i n d i r e c t l y b y affecting k n o w l e d g e acquisition, failure t o m o d e l t h e s e p r o c e s s e s j o i n t l y c a n also l e a d t o e r r o n e o u s c o n c l u s i o n s

t Additional complexities also exist. Effort will affect both learning from experience and the effectiveness of application of knowledge. Performance in one period also affects what an individual experiences the next period. These effort and multiperiod issues are not considered in the current analysis because of a lack of data, but are dicussed as directions for future research at the end of the paper.

THE DETERMINANTSOF AUDIT EXPERTISE concerning the causal relations and to erroneous actions designed to improve performance. Only joint modeling of the determinants of knowledge and performance allows separation o f the effects of ability and experience, which have different implications for the effects of changes in auditor selection policies and/or training. The present study extends Bonner & Lewis ( 1 9 9 0 ) and Libby & Luft ( 1 9 9 3 ) in three key respects. First, w e develop a framework to predict the structure of the relations in Libby and Luft's model which classifies tasks by their degree o f structure and the characteristics o f their related learning environment. Second, w e demonstrate h o w to test these predictions and operationalize the Libby and Luft model using LISREL and the data from Bonner and Lewis's four tasks. We w e r e able to test only three of the four cells in our model since the Bonner & Lewis ( 1 9 9 0 ) data were not collected for the purpose of testing our models. Nevertheless, both the predictive framework and the LISREL analysis provide additional insights into the causal relations underlying Bonner and Lewis's findings and demonstrate the importance of examining the more complete model using p r o p e r analysis. Consistent with our predictions, w e find evidence for an experience-knowledge effect and a knowledge-performance effect across all four tasks, and the effect of ability on knowledge and performance only in certain tasks. Specifically, problem-solving ability was found to he a direct determinant of performance only for the relatively unstructured tasks (ratio analysis and identification of earnings manipulation). Problem-solving ability was also predicted to have an indirect effect on performance through knowledge acquisition only in tasks w h e r e the learning environment was impoverished. This prediction was confirmed in three of the four tasks. The tests of the models provide added insights into the validities of the tests and selfreport measures o f knowledge used in Bonner & Lewis (1990), The results of the tests provide important guidance both for the design of future research and for the development of more valid measures of the constructs o f interest. Our third extension examines whether audit

703

expertise generalizes across different audit tasks. Marchant ( 1 9 9 0 ) discusses available literature that suggests that specific and general expertise are independent of each other. We employ the framework described above to suggest and provide evidence that specification of the associations b e t w e e n performance o n different audit tasks requires consideration of the moderating role o f experience (being more closely related for more experienced auditors) and the degree o f task structure (being more closely related w h e n the tasks are unstructured). This finding is important to the study of expertise in general because the effects of experience and task requirements on these associations have not been investigated in either the auditing or the psychology literature. The findings also have implications for the definition of an "expert auditor", which has been the subject of much current debate (e.g. Abdolmohammadi & Shanteau, 1992; Bonner & Lewis, 1990; Libby, forthcoming). The definition of an expert auditor is important in practice for determining the appropriate individuals to provide policy input on technical issues and to serve as the basis for the development of expert systems. The remainder of the paper is organized as follows. The next section describes the conceptual framework employed in developing the causal models and the results obtained. The development of hypotheses relating to specific and general expertise and the corresponding results is then described. The paper concludes with a discussion of the findings and their implications.

GENERAL MODEL AND TASK CONTINGENCIES A model of performance is one w h e r e experience creates opportunities for the acquisb tion of knowledge, while ability and effort determine the amount of knowledge acquired given that experience. This knowledge acquired, along with ability and effort, then directly affects performance. This performance in one period

704

It. LIBBYand HUN-TONG TAN

in turn affects what an individual experiences in the n e x t period. Although effort and multiperiod issues are important, they are not tested in this paper, owing to data constraints. Rather, w e focus on the general model illustrated in Fig. l ( b ) , w h e r e e x p e r i e n c e and ability are inputs into knowledge, while knowledge and ability impact on performance. Although other dimensions of ability exist and may have different effects on p e r f o r m a n c e and knowledge, w e focus on problem-solving abilities in o u r model as such abilities have b e e n s h o w n to be important determinants of job performance. Meta-analyses of studies of the determinants of performance by Schmidt et aL (1986) provide s o m e support for this general model for non-accounting tasks. However, this general model may not b e applicable to all tasks. We argue that the effects of problem-solving ability on knowledge acquired and on performance are contingent on the nature of the taslc Specifically, the more impoverished the learning environment related to the task, the more important is the effect of problem-solving ability on knowledge acquired. The richness of the learning environment is defined h e r e to be a function of both the complexity of the knowledge necessary to c o m p l e t e the task and the m a n n e r in which the knowledge is conveyed to the auditors in question. A learning environm e n t is m o r e impoverished to the degree that the relevant knowledge is c o m p l e x and/or not made available in a structured form through instruction and professional guidance. Since both attributes contribute to the richness of the learning environment, b o t h a learning environm e n t involving m o r e c o m p l e x knowledge available in structured form and one involving simpler knowledge available only in unstructured form would be classified as having an intermediate level of richness. Bonner & Pennington (1991 ) document dramatic differences across audit tasks in the n u m b e r of well-structured learning opportunities made available to auditors. 2

Ackerman ( 1 9 8 8 ) has d o c u m e n t e d the importance o f cognitive abilities for the learning of declarative knowledge that occurs for novel tasks. We also p r o p o s e that, the m o r e unstructured a task is, the m o r e important is the direct influence o f problem-solving ability on per. formance ( e.g. see Simon, 1979; Lesgold, 1984 ). Consistent with Einhorn (1976), Abdolmohammadi ( 1991 ), and Bonner & Pennington ( 1991 ), to the degree that the task is characterized by the need to define the problem, generate alternative solutions, search for information from disparate sources, make c o m p l e x computations, and e m p l o y forward and backward reasoning as a basis for selection a m o n g alternatives, it is categorized as being less structured. Hunter ( 1 9 8 3 ) p e r f o r m e d metaanalyses of 14 studies, and found that the effect of ability on p e r f o r m a n c e was greater in the case of civilian data than in the case of military data. This was attributable to the greater emphasis on adhering to standard operating p r o c e d u r e in the military service, c o m p a r e d to the emphasis on creativity in the civilian sector (see discussion in Schmidt et aL, 1986). To the extent that the military tasks are m o r e structured than the civilian tasks examined b y Hunter (1983), the study provides s o m e support for the importance o f problem-solving ability for less structured tasks. This framework can b e applied to the four tasks examined by Bonner and Lewis: internal control evaluation, ratio analysis, determination of earnings manipulation, and accounting for financial instruments. In the next section, each of the Bonner & Lewis ( 1 9 9 0 ) tasks is classified b y task requirements and learning e n v i r o n m e n t and this classification is used as a basis for the causal m o d e l p r o p o s e d for each of the tasks.

Task-specific models As m e n t i o n e d earlier, the basic m o d e l considers e x p e r i e n c e and ability to be inputs to the

2 Similarly,Bonnet & Pennington ( 1991 ) classifiedknowledge as structured if it is well organized and reasonably extensive, and impoverished to the degree it is either non-extensive or poorly organized.

THE DETERMINANTSOF AUDIT EXPERTISE

705

Task Requircrnents Structured

Rich

Unstructured

InternalControl

Lmmmg Environrmmt Impoverished

Financial Instruments

Earnings Manipulation Ratio Analysis

Fig. 2. Task classification.

acquisition of knowledge, while knowledge itself and ability affect performance. The classification o f tasks developed b e l o w is summarized in Fig. 2 and the predicted models for each of the tasks are s h o w n in the left panel of Fi~ 3. In the discussion that follows, w e focus on modifications to the basic model in Fig. l ( b ) , if any, for each of the tasks, using the framework in Fig. 2. The classifications o f the four learning environments w e r e m a d e in light o f the fact that nearly all of the participating auditors had three or m o r e years of audit e x p e r i e n c e with a Big 6 firm. Internal control model The internal control evaluation tasks requires the detection o f errors arising from deficiencies in the internal control system, and identification of substantive tests that can b e used to d e t e c t these errors. Since the knowledge necessary for the p e r f o r m a n c e of this task is only m o d e r a t e l y c o m p l e x and is the subject o f a large n u m b e r of well-structured learning opportunities during the first three years o f an auditor's career (Bonner & Pennington, 1991), this learning environment is classified as rich for the current subjects. 3 In this task, the p r o b l e m is well defined, the alternative solutions

and relevant information are well specified, and no computations or backward reasoning are required. The internal control evaluation task therefore appears quite structured. Hence, no effects of problem-solving ability on knowledge or p e r f o r m a n c e are predicted.

Ratio analysis and earnings manipulation model The ratio analysis task requires the detection o f a single low-frequency accounting e r r o r that can explain the pattern o f ratio changes, and the earnings manipulation task requires indentification of a reason for irregularities in the accounts. The c o m p l e t i o n of the ratio analysis task requires knowledge of the effects o f accounting errors on financial statem e n t accounts, the definitions of financial ratios, and the identity of low-frequency errors. T h e r e are numerous structured opportunities to acquire the first t w o knowledge elements from texts and training and thus the learning e n v i r o n m e n t for these elements is rich. However, auditors have little opportunity to acquire knowledge of lowfrequency errors from either structured sources or e x p e r i e n c e (see Bonner & Pennington, 1991). Since this third knowledge element is necessary for task completion, w e classify the

3 The classificationsare supported by the fact that mean scores on the knowledge tests related to the taskswith impoverished learning environments were much lower than those with rich learning environments.

706

R. LIBBY and HUN-TONG TAN

(a) ImenmlControl Ev#,mfioa Task ActualModel Obtained 0.193"**

.~ 0.106"*

(b) Ratio Analysis Task

l ~ i c ~ ! Qm~! Nodel

Acm~ M o ~ Obmiu~

(c) Earnings M~ipel~akm Task Act~l Model Obmine~

(d) FinancialInslrementsTask Predicted Casual Model

Actnal Model Obtained

* p < 0 . 1 ; ** p <0.05; *** p <0.01 Fig. 3, Models of performance of audit tasks: standardized (unstandardized) coefficients.

THE DETERMINANTS OF AUDIT EXPERTISE l e a r n i n g e n v i r o n m e n t as i m p o v e r i s h e d . W i t h r e s p e c t t o t h e e a r n i n g s m a n i p u l a t i o n task, t h e s u r v e y b y B o n n e r & P e n n i n g t o n ( 1991 ) s h o w e d that only very experienced auditors perform the task o f d e t e c t i n g e a r n i n g s m a n i p u l a t i o n , a n d even these auditors do not receive any form of f o r m a l i n s t r u c t i o n p r i o r t o p e r f o r m i n g t h e task. Furthermore, fraudulent reporting occurrences are rare (National Commission on Fraudulent Financial R e p o r t i n g , 1987). H e n c e , t h e l e a r n i n g e n v i r o n m e n t r e l a t e d to this task is also classified as i m p o v e r i s h e d . 4 T h e r a t i o analysis a n d e a r n i n g s m a n i p u l a t i o n tasks a r e u n s t r u c t u r e d b e c a u s e t h e y r e q u i r e g e n e r a t i o n o f a l t e r n a t i v e solutions, s e a r c h for information from disparate sources, and computations and forward and backward reasoning as a basis for s e l e c t i o n a m o n g alternatives. Since b o t h t h e r a t i o analysis a n d t h e e a r n i n g s m a n i p u l a t i o n tasks a r e c o n s i d e r e d t o b e r e l a t i v e l y u n s t r u c t u r e d tasks w i t h i m p o v e r i s h e d l e a r n i n g e n v i r o n m e n t s , t h e p r o p o s e d m o d e l for t h e s e tasks i n c l u d e s b o t h a link b e t w e e n a b i l i t y a n d k n o w l e d g e a c q u i r e d a n d a d i r e c t link b e t w e e n a b i l i t y a n d p e r f o r m a n c e [ s e e Fig. 3 ( b ) a n d ( c ) ] . F i n a n c i a l i n s t r u m e n t s m o d e l T h e financial i n s t r u m e n t s task [ s e e Fig. 3 ( d ) ] r e q u i r e s identification and specification of journal entries r e l a t e d t o an i n t e r e s t r a t e swap. K n o w l e d g e r e q u i r e d for s u c h financial t r a n s a c t i o n s m a y b e considered specialized and complex, since these t r a n s a c t i o n s a r e n o t c o m m o n l y e n c o u n t e r e d . As a c o n s e q u e n c e , t h e financial i n s t r u m e n t s task l e a r n i n g e n v i r o n m e n t is i m p o v e r i s h e d for t h e participating subject group, and thus ability s h o u l d a i d in its acquisition. H o w e v e r , t h e task

707

itself d o e s n o t r e q u i r e p r o b l e m definition, s e a r c h for i n f o r m a t i o n f r o m d i s p a r a t e s o u r c e s , o r b a c k w a r d r e a s o n i n g as a basis for s e l e c t i o n a m o n g alternatives, a n d is t h u s s t r u c t u r e d . P e r f o r m a n c e o n this task w o u l d s e e m t o d e p e n d primarily on whether the requisite knowledge is available, a n d t h u s g r e a t e r p r o b l e m - s o l v i n g skill s h o u l d n o t h a v e a d i r e c t effect o n p e r f o r m ance. In a d d i t i o n t o g e n e r a l a u d i t e x p e r i e n c e , a link is also p o s i t e d b e t w e e n specific e x p e r i e n c e in t h e financial i n s t r u m e n t s task a n d k n o w l e d g e a c q u i r e d . This l i n k e x i s t s as t h e k n o w l e d g e r e q u i r e d for t h e task is s p e c i a l i z e d , a n d a c q u i r e d in p a r t t h r o u g h s p e c i a l t r a i n i n g a n d e x p o s u r e t o financial i n s t r u m e n t t r a n s a c t i o n s . Results a n d discussion T h e causal m o d e l s w e r e t e s t e d w i t h s t r u c t u r a l e q u a t i o n s u s i n g t h e LISREL VII p r o g r a m . In this a p p r o a c h , a m e a s u r e m e n t m o d e l is s p e c i f i e d t h a t relates the observed variables to the latent variables, in a d d i t i o n to a s t r u c t u r a l m o d e l t h a t relates the latent variables to each other. Both the measurement model and the structural m o d e l a r e s i m u l t a n e o u s l y e s t i m a t e d b y LISREL VII ( J o r e s k o g & S o r b o m , 1 9 8 9 ) 5 F o r s o m e o f t h e v a r i a b l e s s u c h as specific e x p e r i e n c e , t w o m e a s u r e d ( o r m a n i f e s t ) v a r i a b l e s w e r e u s e d as i n d i c a t o r s o f l a t e n t variables. This r e d u c e s t h e effect o f r a n d o m a n d m e a s u r e m e n t errors, a n d s t r u c t u r a l coefficients o b t a i n e d a r e less b i a s e d than those obtained using manifest variables alone. F a c t o r l o a d i n g s a n d s t r u c t u r a l coeffic i e n t s a r e o b t a i n e d u s i n g t h e m a x i m u m likelihood estimation method. Estimation involves finding v a l u e s o f t h e coefficients t h a t p r o d u c e

4 Bonner & Pennington (1991) classify the knowledge necessary to assess theprobabllity of management fraud to be structured. However, the earnings manipulation task of concern in the study by Bonner & Lewis (1990) was that of determining the mason for the earnings manipulation. Given the infrequent exposure to earnings manipulation and the paucity of instruction available, we believe that the learning environment related to this task is impoverished. 5 A major advantage of LISRELis that it produces goodness of fit indices that indicate the extent to which the hypothesized model could have produced the observed dataL One goodness of fit statistic is the overall chi-square statistic. It takes on values from zero to infinity, representing a perfect fit and lack of fit, respectively. However, the chi-square statistic is sensitive to sample size and violations of the assumption of multivariate normality (Bentler, 1983; Joreskog & Sorbom, 1989), leading to rejections of the model even when the fit is reasonable. Consequently, it is useful to look at other indices as well. These include the adjusted goodness of fit index (AGFI) and the root-mean-square residual (RMR). AGFIvalues in excess of 0.90 and RMR values of less than 0.10 are generally regarded as indications of good fit.

708

R. LIBBY and HUN-TONG TAN

an estimated covariance matrix that is as close as possible to the sample covariance structure o f the manifest variables. Internal control m o d e l The first model tested was the internal control evaluation m o d e l Experience was operationalized as months of audit experience (MOEXP) while performance was operationalized as the scores obtained for the internal control evaluation task (Task 1 ). We initially operationalized knowledge as a latent variable comprising two manifest variables: objective test of internal control knowledge (AUDCTL) and self-rated ability at internal control evaluation (ICEVAL). The standardized factor loadings of 0.313 and 0.160 were low, and way below the benchmark of 0.60 suggested by Bagozzi & Yi (1988), indicating that the convergent validity of the knowledge construct was questionable. We next constructed a knowledge composite, made out of the sum o f the standardized objective knowledge test scores (AUDCTL) and the self-ratings (ICEVAL). This composite should not b e interpreted as a scale in the psychometric sense, since the components w e r e nearly distinct from one another. Rather, w e wanted a broad construct that captured the distinctive elements of both knowledge measures, in the absence of criteria to determine which measure was the more valid (see Van de Ven & Ferry, 1980, for a discussion o f the scope o f a construct). The fit of the model using this knowledge composite was good: X2(2) = 3.78, p = 0.151, AGFI = 0.969, RMR = 0.050. All the theoretical links proposed also achieved statistical significance ( p < 0.05 and p < 0.01, respectively). The variances explained in the knowledge construct and performance construct w e r e 6.4% and 1.1%, respectively. This model is shown in the right panel of Fig. 3(a). 6

To determine w h e t h e r ability affected knowledge and/or performance, w e also examined a model which included a link from ability to knowledge and to performance, w h e r e ability was operationalized by scores o n a problemsolving test. Neither the ability-knowledge link ( p = 0.413), the ability-performance link ( p = 0.212), nor the incremental fit, X2a(2) = 1.20 ( p > 0.50) was significant, supporting our contentions regarding the effect of ability on knowledge and performance in structured tasks with rich learning e n v i r o m e n t s . Further analysis suggested that each o f the knowledge measures appeared to be measuring different aspects of knowledge. When knowledge was measured by the objective knowledge test, only the experience-knowledge link achieved statistical significance ( p < 0.10). With the selfratings (ICEVAL) as a measure of knowledge, both the experience-knowledge link (p < 0.01 ) and the knowledge-performance link (p < 0.05) were significant. The lack of a k n o w l e d g e performance link w h e n using the objective knowledge test scores calls into question its validity as a measure of knowledge necessary for performance of this particular task. The selfratings had predictive validity, though they may be capturing auditors' perceptions of their selfefficacies at the task.7 While not conclusive, the fact that the self-ratings w e r e also related to experience suggests that they capture some portion of relevant knowledge. Ratio analysis m o d e l The ratio analysis model was examined next_ Knowledge for this task was initially assessed by the combination o f the objective test of ratio analysis knowledge (AUDAPS) and self-rated ability at ratio analysis (RAEVAL). One factor loading was low (0.167) while the other loading was moderate (0.405),

6 Both unstandardized and standardized coefficients are reported. In both instances, the indicators retain their original scales. However, w h e n the coefficients are standardized, each construct has unit variance. Thus, the standardized coefficients should be interpreted as the n u m b e r of standard deviation (S.D.) changes in one construct resulting from a 1 S.D. change in another construct. Standardized coefficients have the advantage of ease of interpretation of effect sizes. Those that are near O suggest small effects, while those that are close to I suggest large effects. 7 Bandura ( 1 9 8 6 ) has demonstrated that serf-efficacy belief is an important determinant of behavior and is related to actual experiences.

THE DETERMINANTSOF AUDIT EXPERTISE again implying the lack o f convergent validity for the knowledge construct. A knowledge c o m p o s i t e c o m p o s e d of the sum o f the t w o variables was again used as a measure o f knowledge. An excellent fit was obtained: Z2(4) = 0.18, p = 0,996, AGFI = 0.999 and RMR = 0.007. The predicted links w e r e significant, with the exception o f the ability-knowledge link [see Fig. 3(b)]. The m o d e l explained 1.7% of the variance in the knowledge construct, and 24.5% o f the variance in the p e r f o r m a n c e construct. As in the internal control evaluation task, the individual knowledge meastwes appeared to b e capturing different attributes o f knowledge. W h e n knowledge was Ol~.xationalized as selfrated ability at ratio analysis (RAEVAL), the hypothesized links achieved statistical significance, with the exception o f the abilityknowledge link. This again suggests that the self-ratings captured a portion o f knowledge acquired through experience. W h e n the m o d e l obtained using the objective test o f ratio analysis knowledge (AUDAPS) was used, both the ability-knowledge and e x p e r i e n c e - k n o w l e d g e finks w e r e not significant. This suggests the possibility that the tests captured t e x t b o o k knowledge acquired during schooling. However, the low percentage o f variance explained in the knowledge construct suggests that the above results and lack of a significmat ability-knowledge link may b e caused by m e a s u r e m e n t error in the knowledge variables. Earnings manipulation model W e n e x t examined the causal m o d e l for the earnings manipulation task. Knowledge was operationalized as general business knowledge. All the posited links achieved statistical significance, and the fit indices w e r e satisfactory, X2(1) -0.90, p = 0.925, adjusted goodness o f fit index = 0.995, RMR = 0.O15. The m o d e l is presented

~,J.

.

.

.

.

.

.

,.

.

.

.

.

.

.

.

.

.

709

in Fig. 3(c). The percentages of variance in the knowledge and performance constructs explained b y the m o d e l w e r e 12.7% and 26.9%, respectively. F/nanaa/i~~ m o d e l The final model tested was that for the financial instruments task. Knowledge was operationalized as the test score on the financial instruments test. The causal m o d e l for the financial instruments task differed from the o t h e r models in one particular aspect: in addition to ability and general audit experience, specific e x p e r i e n c e in financial instnm~ents was also posited to determine knowledge. Specific experience was operationalized as the combination of t w o measures: p e r c e n t a g e of time spent o n financial institution clients (FIN) and e x p e r i e n c e o n interest rate swaps (IRS). The factor loadings w e r e satisfactory. Standardized factor loadings of 0.453 and O.831 w e r e obtained.S The chi-square statistic was significant Z2(9) = 117.44, p = O.000; the adjusted goodness of fit index was 0.884, while the RMR residual was 0.139. These indices indicate a relatively p o o r fit. Examination of the modification indices indicates that the fit o f the m o d e l could b e substantially improved b y estimating the link b e t w e e n performance and specific experience. We tested a variant o f the model originally proposed, w h e r e specific e x p e r i e n c e also directly affects performance [see Fig. 3(d)]. The fit of this alternative m o d e l was satisfactory: Z2(8) = 14.46, p = O.O71, adjusted goodness o f fit = 0.948, RMR = 0.038. The chi-square difference test indicated that the addition of the specific e x p e r i e n c e - p e r f o r m a n c e link i m p r o v e d the fit o f the m o d e l [X2d(1) = 102.98, p < O.001]. 9 T h e m o d e l explained 13.1% of the variance in the knowledge construct, and 56.8% of the variance in the p e r f o r m a n c e construct. T h e p r e s e n c e o f this link suggests that the

.

8 Although the factor loading of 0.453 was somewhat below the 0.60 criterion, it was reasonably dose, Separate models were run with the individual indicators, and results were similar to those using multiple indicators. 9 Addition of a direct general experience-~rformance link to the other three task models yielded marginallysignificant (at>< 0A ) improvement in fit for the internal control task and non-significantimprovement (p > 0.5 and p > 0.25) for the ratio analysis and earnings n~nipul~ion tasks.

710

R. LIBBYand HUN-TONG TAN

knowledge test may have failed to capture important elements of the knowledge necessary for the successful task completion. This is particularly likely since one c o m p o n e n t of specific experience measured was the amount of direct training on interest rate swaps ( t h e task of interest). It also presents the possibility that e x p e r i e n c e may directly affect performance due to the acquisition of methods, techniques and p s y c h o m o t o r skills, that are independent of increases in job knowledge (Schmidt et al., 1986). More e x p e r i e n c e d auditors may have learned to perform the tasks faster and m o r e accurately. In addition, the tasks with direct effects of experience on performance turned out to be structured tasks (internal control evaluation task and the financial instruments task), w h e r e ability did not affect performance. It m a y be that these tasks are susceptible to automaticity effects arising from experience, since problem-solving skills are not required for their comp!etiorL Summary. Overall, the results w e r e consistent with our theoretical models, with the exception of the ratio analysis task. The general links w h e r e experience affects knowledge acquired, which in turn affects task performance, w e r e supported for all four tasks. Also, as expected, the nature of the other specific links was found to b e contingent on the task. Ability had a direct impact o n knowledge acquired in the earnings manipulation and financial instruments tasks w h e r e the learning environment was impoverished. In the internal control evaluation task, owing to the relatively simple nature of that knowledge and the effects of prior exposure through classroom training, no such relation was in evidence. In the ratio analysis task, the e x p e c t e d relation was not in evidence, possibly because of validity problems with the knowledge measures. In addition, for unstructured tasks such as ratio analysis and detection o f earnings manipulation, ability had a direct effect on performance. For structured tasks such

as internal control evaluation and accounting for financial instruments, ability had no direct impact on performance.

RELA~ON BETWEEN GENERAL AND SPECIFIC EXPERTISE The low correlations a m o n g the performance statistics for the four audit tasks reported b y Bonner & Lewis ( 1 9 9 0 ) suggest a lack of generality of audit expertise across tasks. Marchant (1990, p. 23) argued that "the auditor is m o r e likely to gain expertise in a specific subdomain without gaining exposure to auditing in general", and that distinguishing b e t w e e n general and specific expertise is important: This claim draws support from a study by McGraw & Pinney ( 1 9 9 0 ) which found that general and specific expertise in politics operate independently. The framework presented in Fig. 1( b ) suggests that consideration of the relations among experience, ability, knowledge, and performance may indicate w h e n general or cross-task expertise will m o r e likely be observed. The framework suggests that performance is determined by two inputs, e x p e r i e n c e and ability, the effects of which are mediated b y knowledge. Given this structure, and assuming h o m o g e n e i t y of experiences across auditors, the degree of cross-task association in performance will b e determined by the similarity in the manner in which experience and ability affect p e r f o r m a n c e of the two tasks. Further, if betterperforming auditors are retained in the firm, the association b e t w e e n e x p e r i e n c e level observed and performance will increase. To the degree that experiences are not h o m o g e n e o u s across auditors, the relation b e t w e e n performance on two tasks will be stronger for m o r e e x p e r i e n c e d auditors because their larger samples of experiences are likely to be m o r e similar, l° As a consequence, w e suggest that cross-task associations in performance will be larger the m o r e

to There is less measurement error in months of experience as a proxy for actual experience for more experienced auditors.

THE DETERMINANTSOF AUDIT EXPERTISE similar their k n o w l e d g e a n d ability requirements, a n d t h e m o r e e x p e r i e n c e d the auditors in question.t t B o n n e r & Lewis ( 1 9 9 0 ) suggest that t h e k n o w l e d g e n e c e s s a r y for p e r f o r m a n c e o f the four tasks differs, so that n o association b a s e d o n k n o w l e d g e similarity is p r e d i c t e d . Both t h e ratio analysis task a n d t h e e a r n i n g s m a n i p u l a t i o n task are classified as relatively u n s t r u c t u r e d tasks w h e r e p r o b l e m - s o l v i n g abilities d i r e c t l y affect p e r f o r m a n c e . Since ability is a n i n n a t e a t t r i b u t e that r e m a i n s c o n s t a n t o v e r time, w e e x p e c t an association i n the p e r f o r m a n c e of these tasks for b o t h less a n d m o r e e x p e r i e n c e d auditors. Alternatively, for s t r u c t u r e d tasks, k n o w l e d g e is posited to be the only direct determinant of p e r f o r m a n c e ; h o w e v e r , k n o w l e d g e o f different tasks c a n vary for a p a r t i c u l a r i n d i v i d u a l a n d is m o r e likely t o vary for less e x p e r i e n c e d auditors. H e n c e , for less e x p e r i e n c e d auditors, n o associat i o n is e x p e c t e d b e t w e e n the p e r f o r m a n c e of v a r i o u s pairs o f s t r u c t u r e d tasks a n d b e t w e e n s t r u c t u r e d a n d u n s t r u c t u r e d tasks. I n contrast, m o r e e x p e r i e n c e d auditors are e x p o s e d t o a larger sample of tasks, w h i c h r e d u c e s differences i n e x p e r i e n c e s across auditors, a n d will i n c r e a s e the c o n t r i b u t i o n of ability to p e r f o r m a n c e . C o n s i s t e n t w i t h this a r g u m e n t , it is e x p e c t e d that e x p e r i e n c e d a u d i t o r s will b e e x p e r t s at m o r e tasks, a n d that t h e association b e t w e e n p e r f o r m a n c e o n v a r i o u s tasks will b e s t r o n g e r as e x p e r i e n c e increases.

711

Results and discussion Definition o f task-specific expertise. A key issue i n t h e analysis is t h e d e t e r m i n a t i o n o f w h a t constitutes expert performance. Performance scores w e r e s t a n d a r d i z e d a n d e x p e r t s w e r e identified as t h o s e auditors w h o h a d scores that w e r e at least as high as t h o s e i n t h e t o p few decries. A cut-off level of 30%, 20%, a n d 10% p r o d u c e d similar results. H e n c e , results are r e p o r t e d o n l y for e x p e r t p e r f o r m a n c e d e f i n e d t o b e the scores o b t a i n e d b y t h e t o p 20% o f t h e participants. ~2

Role o f task structure and experience. The p r e d i c t i o n is that for less e x p e r i e n c e d auditors, a r e l a t i o n exists b e t w e e n e x p e r t i s e i n t h e ratio analysis task a n d the e a r n i n g s m a n i p u l a t i o n task b e c a u s e of their c o m m o n r e l i a n c e o n p r o b l e m - s o l v i n g ability, b u t n o t b e t w e e n o t h e r tasks. Chi-square tests w e r e c o n d u c t e d t o test the association b e t w e e n different pairs o f a u d i t tasks. A c o n d i t i o n a l p r o p o r t i o n , defined as the p r o p o r t i o n o f a u d i t o r s w h o are e x p e r t s i n o n e task g i v e n that t h e y are e x p e r t s i n a n o t h e r task, was also c o m p u t e d ) 3 The conditional p r o p o r t i o n is e x p e c t e d to b e higher for m o r e e x p e r i e n c e d a u d i t o r s t h a n for less e x p e r i e n c e d auditors. Results are p r e s e n t e d i n T a b l e 1. C o n s i s t e n t w i t h the p r e d i c t i o n s , for b o t h m a n a g e r s a n d seniors, chi-square tests indicated an association b e t w e e n e x p e r t p e r f o r m a n c e i n t h e ratio analysis task a n d the e a r n i n g s m a n i p u l a t i o n task ( p < 0.01). For the seniors, this w a s the o n l y pair of tasks

,l These effects can still be in evidence even though Bonner & Lewis (1990) found no differences in the performance regression models across experience groups. 12All auditors performing at a level greater than or equal to the 20th percentile individual were included as experts. As a consequence, the number of auditors judged expert for each task varies. In addition, differences in difficultylevel across tasks can make the 20% criterion problematic to interpret; however, this concern is mitigated by the fact that different criteria (e.g. cut-off of 30%, 20%, 10% ) produced similar results. t3 The conditional proportion indicates, among the auditors who were experts in one task such as ratio analysis, what proportion of these auditors were also experts at a second task such as internal control. This was computed in the following way: For each task, a contingencytable was obtained, showing the number of auditors who were expert in one, the other or both tasks. The conditional proportion of auditors who were experts in a specific task, given that they were experts in a second task, would be obtained by the formula: Proportion (Task I experts/Fask 2 experts) -Number of auditors whowere both Task 1 and Task 2 experts Number of auditors who were Task 2 experts

712

R. HBBY and HUN-TONG TAN

Table 1. Chi-square tests of association between expertise in various tasks by position (top 20% of participants) Position

Task combination

Seniors Conditional proportion

Ratio Analysis/Int Control

Managers Conditional proportion

Overall Conditional proportion

8/63#

10/25

18/88

0.13

0.40""

0.20

Int Control/Ratio Analysis

8/29* 0.28

10/16 0.63

18/45 0.40

Earnings Manip/Int Control

10/63 0.16 10/37 0.27

12/25 0.48 12/23 0.52

22/88 0.25 22/60 0.37

10/63 0.16 10/29 0.35

13/25 0.52** 13/22 0.59

23/88 0.26 23/51 0.45

11/29 0.38** 11/37 0.30

9/16 0.56* 9/23 0.39

20/45 0.44*** 20/60 0.33

5/29 0.17 5/29 O.17

7/16 0.44 7/22 0.32

12/45 0.27 12/51 0.24

6/37 O.16 6/29 0.21

10/23 0.43 10/22 0.45

16/60 0.27 16/51 0.31

Int Control/Earnings Manip Fin Instru/Int Control lnt Control/Fin lnstru Earnings Manip/Ratio Analysis Ratio Analysis/Earnings Manip Fin Insmt/Ratio Analysis Ratio Analysis/Fin Instru Fin Instru/Earnings Manip Earnings Manip/Fin lnstru

# The numerator indicates the number of auditors who were experts in the ratio analysis and internal control evaluation tasks. The denominator indicates the number of auditors who were experts in the internal control evaluation task. Figures below the fractions are decimals. * The numerator indicates the number of auditors who were experts in the ratio analysis and internal control evaluation tasks. The denominator indicates the number of auditors who were experts in the ratio analysis task. *p < 0.10; **p < 0.05; ***p < 0.01.

w h e r e there was any association in performance. For the managers, there was also evidence of an association b e t w e e n ratio analysis and internal control evaluation ( p < 0.05), and financial instruments and internal control evaluation ( p < O.05). Comparisons w e r e also made b e t w e e n the conditional proportions at both e x p e r i e n c e levels. As Table 1 shows, managers had consistently higher conditional proportions c o m p a r e d to the seniors across the various task combinations.

DISCUSSION Two important findings in Bonner & Lewis ( 1 9 9 0 ) were that abilities and knowledge are determinants of audit performance, and that these factors differentially affect performance in different audit tasks. The paper extends Bonner & Lewis ( 1 9 9 0 ) by jointly modeling the determinants of knowledge and the determinants of performance, and specifying the nature of the causal model as a function of audit task.

THE DETERMINANTSOF AUDITEXPERTISE As noted earlier, this extension is important since ability can affect performance both directly and indirectly by affecting knowledge acquisition. Failure to model these processes jointly can lead to erroneous conclusions concerning the causal relations and erroneous actions designed to improve performance. This was most evident in the financial instruments task where mental ability played a significant role in the performance model only through the indirect link. Failure to capture this indirect effect would result in focusing only on possible changes in training or task-assignment programs while ignoring potential effects of improved selection on performance of this task. Modeling the relations in this fashion made evident the need to classify tasks both by their degree of structure, and by the characteristics of the related learning environment. More generally, treating experience only as a proxy for knowledge ignores the importance of the knowledge acquisition process to understanding the nature of expertise, and understates the importance of studying experience effects. Even if knowledgeperformance relations have been demonstrated, without understanding the knowledge acquisition process, the implications for improving the auditor selection and development process would not be clear. An understanding of the complete causal model is necessary to infer the implications of findings for decision improvement. Hogarth (1993) makes a similar point that one cannot determine how to improve performance without understanding the process. In summary, what have we learned beyond the results documented in Bonner & Lewis (1990)? Bonner & Lewis (1990) were, in a sense, demonstrating the limitations of using general experience to explain expert performance, by testing the direct effects of experience, ability, and knowledge on performance across all four tasks. They also did not perform crosstask analyses. In comparison, we first developed a general causal model of performance determinants and then modified this model by developing a framework for classifying the knowledge and performance elements for each task to predict the effects of ability on the different corn-

713

ponents, and to predict cross-task associations in performance. Concerning the effects of experience, we found that experience had direct effects on performance only for structured tasks such as the internal control evaluation task and the financial instruments task, which suggests the possibility of automaticity developing from experience. Moreover, for all tasks, experience had an impact on the development ofknowledge relevant to task performance. Also, ability directly affected performance only for unstructured tasks (ratio analysis task and earnings manipulation task); ability had an indirect effect on performance mediated by knowledge for the earnings manipulation task and the financial instruments task. Finally, the validity of the knowledge construct was tested where there were multiple indicators. Analyses showed that the convergent validity of the construct was lacking. The objective knowledge measures in the internal control task and the ratio analysis task were found to lack predictive validity. The paper also extends the expertise literature by examining the relation between expertise in different audit tasks. Recent research has focused on the importance of task-specific expertise. However, no research has investigated the nature of the relation between general and specific expertise and any moderating variables. An important contribution this study makes is demonstrating that general and specific expertise are not independent as suggested by prior research in psychology, but that the relation is conditional on the experience level and task demands. Expertise in a particular task was generally associated with expertise in another task for managers, but not for seniors. For the latter group, expert performance was associated only for tasks which place similar demands on the auditor's abilities, in this case requiring problem-solving skills. To the degree that these findings generalize to a wider array of audit tasks, they suggest that identification of general technical experts to serve in policymaking positions, or as a basis for expert system development, may not be as difficult as prior

714

R. LIBBYand HUN-TONG TAN

studies emphasizing the task-specificity of audit expertise m a y imply.

Limitations and future research directions Several limitations of the p a p e r must be noted. Most concerns result from the fact that Bonner and Lewis did not gather their data for the purpose of testing our models. As a consequence, the effect of invalid measures may have resulted in equivocal results with respect to s o m e of the causal models, in particular for the ratio analysis task. Task selection presents the possibility of other validity problems. First, concerning the knowledge measures, item analyses o f the Bonner and Lewis data, which would allow assessment of the types of knowledge measured b y the different tests, would be an important addition to the literature because different types of knowledge m a y vary in their sensitivity to different types of experiences and abilities. Further, validity p r o b l e m s with the knowledge tests forced reliance on the self-ratings of knowledge, which themselves may be capturing other constructs. As a consequence, validation of the knowledge measures should be a p r i m e c o n c e r n in future research. Second, the types of abilities necessary for the completion of various audit tasks are not well specified. Only general problem-solving ability was assessed b y Bonner and Lewis. The development of m o r e refined and pertinent ability measures would also benefit future research. Third, inasmuch as Bonner and Lewis did not design their tests and tasks to determine the validity of our classification framework in Fig. 2 or our predictions concerning specific and general expertise, additional relations not uncovered here may b e in evidence w h e n m o r e appropriate tasks and measures are designed. Of related concern is the fact that w e w e r e only able to test three of the four cells of o u r model. Fourth, objective criteria w e r e used to determine successful performance. To the degree that these p e r f o r m a n c e criteria may not b e representative of what is actually important in practice, important abilities and knowledge elements may be omitted from the models. The low levels o f average performance on s o m e tasks support

this possibility. Fifth, the generalizability of the conclusions made regarding the relation between different types of task-specific expertise is a function of h o w representative the tasks examined in this study are to the repertoire of tasks auditors actually encounter. For example, according to Frensch & Sternberg (1989, p. 163), in some tasks, experts may actually be o u t p e r f o r m e d by non-experts owing to a reduction in their flexibility, which is "the ability to change one's m o d e or direction of thinking as a function of changing task or situational constraints". In all four of the Bonner and Lewis tasks, auditors w e r e performing familiar tasks for which there w e r e no novel or n e w task. situational constraints. As a consequence, it is not surprising that diminishing returns to experience w e r e not observed in our analysis. We believe that o u r study provides a m o d e l and demonstrates a data analysis m e t h o d that can guide future research, as well as additional insights into the relations represented in the Bonner and Lewis data and the strengths and weaknesses of the measures employed. Of particular importance is the need for future research to e m p l o y validated constructs. The Bonner and Lewis study involved a massive datagathering effort. As such, it probably will not be practical to deal with all or m o s t of the data problems described above in a single future study. Examining m o r e complete models over multiple tasks would place too great a burden o n subject and author resources in a single study. Instead, w e r e c o m m e n d that future studies first focus on individual problems or small related groups of the p r o b l e m s noted above. While w e r e c o m m e n d considering use of m o r e c o m p l e x assessments of knowledge content and structure such as those requiring recall or sorting data, their use will further exacerbate the large demands placed on subjects. Further, a proliferation of tests of ability, knowledge, and performance may be an impedim e n t to progress because of loss of continuity and comparability. Cross-task studies may need to focus on differences in only a subset of the relations presented in our models, while using b o t h old and new tests to maintain

THE DETERMINANTS OF AUDIT EXPERTISE

comparability, and placing reasonable demands on subjects. Finally, as noted earlier, like any models, those proposed here are still incomplete. Particularly important omissions include the effect of effort and multiperiod effects on learning and performance. Effort (along with abilities) also determines the degree to which people acquire knowledge from experiences and the degree to which available knowledge and abilities are brought to bear on the task. Recent papers such as Libby

715

& Lipe ( 1 9 9 2 ) have begun to evaluate these relations. Further, h o w well one performs in one period affects assignments in future periods, and as a consequence the types of future learning opportunities available. The effect of differential assignment patterns on learning and subsequent performance is a particularly important direction for future work since this omitted variable could determine some portion of the relations between performance on novice tasks and tasks normally reserved for more experienced auditors.

BIBLIOGRAPHY Abdolmohammadi, M.J., A Taxonomy of Audit Task Complexity and Experience Requirements in Auditing: Implications for Audit Research and Practice, Working paper (1991). Abdolmohammadi, M.J. & Shanteau, J., Personal Attributes of Expert Auditors, Organizational Behavior and Human DeciMon Proceeses (1992) pp. 158-172. Ackerman, P. L, Determinants of Individual Differences During Skill Acquisition,Journal of Expertmental P$ychology: General (1988) pp. 288-318. Bagozzi, R. P. & Yi, Y., On the Evaluation of Structural Equation Models, Journal of the Academy of Marketing Science (1988) pp. 74-94. Bandura, A., Social Foundations of Thought and Actiom A Social Cognitive Theory (Englewood Cliffs, NJ: Prentice-HaU, 1986). Bentler, M., Some Contributions of Efficient Statistics in Structural Models: Specification and Estimation of Moment Structures, Psychometrlca (1983) Pp. 493-517. Bonner, S. E. & Lewis, B. L, Determinants of Auditor Expertise, Journal of Accounting Research (Supplement 1990)pp. 1-20. Bonner, S. & Pennington, N., Cognitive Processes and Knowledge as Determinants of Auditor Expertise, Journal of Accounting Literature (1991) pp. 1-50. Einhorn, H. J., A Synthesis: Accounting and Behavioral Science, Journal of Accounting Research (Supplement 1976) pp. 196-206. FrensclL P. A., & Sternberg, R. J., Expertise and Intelligent Thinking: When is it Worse to Know Better?, in Sternberg, R. ( ed. ),Advapxes in theP$3~hology ofHuman lnteltigen~, VoL 5, pp. 157-188 ( Hillsdale, NJ: Erlbaum, 1989). Hogarth, R. M., Accounting for Decisions and Decisions for Accounting, Accountin~ Organizations and Soc/ety (July 1993) pp. 407-424. Hunter, J. E., A Causal Analysis of Cognitive Ability, Job Knowledge, Job Performance, and Supervisory Ratings, in Landy, F., Zedeck, S. & Cleveland, J. (eds), Performance Measurement and Theory, pp. 257-266 (Hillsdale, NJ: Erlbaum, 1983). Joreskog, K. G. & Sorbom, D., L/SREL 7: A Guide to theProgram andApptications (SPSS Inc., 1989). Lesgold, A., Acquiring Expertise, in Anderson, J. & Kossyln, S. (eds), Tutorials in Learning andMemory: Essays in Honor of Gordon Bower (New York: W. H. Freeman & Co., 1984). Libby, R., The Role of Knowledge and Memory in Audit Judgment, in Ashton, R. H. & Ashton, A. H. (eds), Judgment and Decision Making Research in Accounting and Auditing (forthcoming). Libby, R. & Lipe, M. G., Incentive Effects and the Cognitive Processes Involved in Accounting Judgments, Journal of Accounting Research (Autumn 1992) pp. 249-273. Libby, R. & Luf~,J., Determinants of Judgment Performance in Accounting Settings: Ability, Knowledge, Motivation, and Environment, Accountin~ Organizations and Society (July 1993) pp. 425-450. Marchant, G., Discussion of Determinants of Auditor Expertise, Journal of Accounting Research (Supplement 1990) pp. 21-28.

716

R. LIBBY and HUN-TONG TAN McGraw, K.M. & Plnney, N., The Effect of General and Domain Specific Expertise on Political Memory and Judgment, S o c ~ Cogn~t/on (Sprl~H 1990) pp. 9-30. National Commission on Fraudulent Financial Reporting (1987~ Schmldt, F. L, Hunter, J. E. & Outerbridge, A. N., Impact of Job Experience and Ability on Job Knowledge, Work Sample Performance, and Supervisory Ratings ofJob PerformanceJournalofApplied Psychology (1986) pp. 432-439. Simon, H., Information Processing Models of Cognition, Annual R~wiew of Psychology (1979) pp. 363-396. Van de Ven, A. H. & Ferry, D. L., Measuring and Assessing Organizations (New York: Wiley, 1980).