- Email: [email protected]
The effectiveness of interactive professional learning experiences as a pedagogical tool: Evidence from an audit setting
J. of Acc. Ed. 30 (2012) 163–172
Contents lists available at SciVerse ScienceDirect
J. of Acc. Ed. journal homepage: www.elsevier.com/locate/jaccedu
Main article
The effectiveness of interactive professional learning experiences as a pedagogical tool: Evidence from an audit setting Maria H. Sanchez a,⇑, Christopher P. Agoglia b,1, Kevin F. Brown c,2 a
Department of Accounting, Rider University, 2083 Lawrenceville Road, Lawrenceville, NJ 08648, USA Department of Accounting & Information Systems, Isenberg School of Business, University of Massachusetts Amherst, 121 Presidents Drive, Amherst, MA 01003, USA c Department of Accountancy, Raj Soin College of Business, Wright State University, 3640 Colonel Glenn Highway, Dayton, OH 45435, USA b
a r t i c l e
i n f o
Article history: Available online 26 August 2012 Keywords: Interactive professional learning experience Auditing education Professional supervisory setting Control environment Fraud risk assessment
a b s t r a c t This paper describes an interactive professional learning experience (IPLE) and provides guidance for implementing an IPLE in an audit classroom. The IPLE described in this paper exposes students to a realistic practice environment within the classroom by bringing practitioners together with students in a professional supervisory setting. Practitioners review students’ work and then meet with students one-on-one to provide feedback on their work. We also document evidence of the pedagogical value of an IPLE by using a between-subjects experimental design in which learning outcomes for participants are compared to a control group that received the same instructions and completed the same written assignment, but did not participate in the professional interaction. In addition, pre- and posttests of students’ audit knowledge allowed for a within-subjects self-assessment of knowledge acquisition. The results strongly suggest that participation in the IPLE improves students’ performance on a skills test of relevant audit material and increases their self-perceptions of knowledge gained. In addition, results indicate that both students and audit professionals consider the IPLE a positive professional learning experience. Ó 2012 Elsevier Ltd. All rights reserved.
⇑ Corresponding author. Tel.: +1 609 895 5582; fax: +1 609 896 5304. E-mail addresses: [email protected] (M.H. Sanchez), [email protected] (C.P. Agoglia), [email protected] (K.F. Brown). 1 Tel.: +1 413 545 5582. 2 Tel.: +1 937 775 3138. 0748-5751/$ - see front matter Ó 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jaccedu.2012.08.004
164
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
1. Introduction Meeting with supervisors who inspect and evaluate one’s work is common practice in the auditing profession. This meeting with its accompanying professional interaction presents a unique learning environment in which advice, insights, and knowledge can be shared. This kind of hierarchical professional interaction is not something to which students are traditionally exposed.3 However, it is possible for instructors to bring this experience to the classroom, in turn helping to make the real-life work environment more tangible and improve student learning. The purpose of this paper is to show how an interactive professional learning experience (IPLE) can help improve student learning. In the audit function, the interaction between supervisor and subordinate plays a critical role in ensuring audit quality, so much so that the process is formalized into the supervision process described in the First Standard of Fieldwork of Generally Accepted Auditing Standards (GAASs) and implicitly recognized in the First General Standard of GAAS, (AICPA, 2011a, 2011b, AU 150.02).4 While supervisor and subordinate interaction plays a critical role in the audit, the real-world interaction entailed in this process is difficult for students to appreciate. Agrawal and Siegel (1991) show that a lack of real-world experience often impairs the ability of students to understand basic concepts and procedures, particularly with regard to complex subjects such as auditing. The Advisory Committee on the Auditing Profession, in its Final Report to the Department of the Treasury (ACAP), notes that often new accounting faculty have little professional experience and that it is difficult for academics to acquire ‘‘practice-oriented’’ knowledge. ACAP supports more interaction between faculty and the profession (ACAP, 2008, VI, pp. 19–22). In addition, a recent report from The Pathways Commission (2012, p. 56) strongly supports the involvement of practitioners in the classroom. Incorporating interaction with professionals in a realistic context may improve knowledge acquisition while providing students with an interesting perspective on practice not readily available in the traditional classroom setting. This paper describes an IPLE and also provides guidance and support for instructors interested in using an IPLE in their classes. The IPLE examined in this study involved a professional review of students’ written submissions for a risk assessment task. In order to create a realistic environment, auditors from Big-Four public accounting firms reviewed students’ memos documenting the control environment and fraud risk assessments for a hypothetical company. After reviewing each student’s memo, the practitioners prepared review notes and later met one-on-one with each student to review his/her work. This meeting allowed students to interact with, and learn from, their reviewers in a professional manner and setting. These one-on-one meetings provided students with insights into what it is like to be a member of an actual audit team. The face-to-face interaction also allowed students to discuss the merits of their work and ask questions of their reviewers. Reviewers suggested ways for students to improve their memos, which ranged from alternate problem-solving approaches and evidence interpretation to more effective written presentation of their work. Perhaps the most common feedback from the professionals was that the students needed to document their conclusions more clearly and succinctly. We tested effectiveness of the IPLE for improving learning by comparing learning outcomes for students participating in the IPLE with students completing the same course material and writing assignment without professional interaction. Further evidence of effectiveness was obtained by performing knowledge assessments of students prior to and after completion of the IPLE. The results of this study strongly suggest that completing the IPLE improves students’ performance on a skills test of the relevant audit material. Also, students’ self-perceptions of their knowledge of the subject matter increase after the IPLE. In addition, results indicate that students consider the IPLE a positive interactive experience. The audit professionals participating in this IPLE perceived the experience to be effective for teaching the material and conveying an understanding of audit practice. The practitioners also appeared to find the IPLE useful, indicating that the IPLE provided them with insights into students’ interpersonal skills and technical abilities.
3 PricewaterhouseCoopers (2003, pp. 5–8) refers to such deficiencies in a critique of accounting education strongly advocating student interaction with professionals. 4 Formal statement of the 10 GAAS is dropped in AICPA’s (new) clarified standards that are applicable for audits of financial statements for years ending after December 15, 2012. Presentation of equivalent responsibilities can be found in sections AU-C 200, AU-C 220 and AU-C 300 of the clarified standards.
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
165
The remainder of this study is organized as follows. The next section discusses IPLE implementation. This is followed by a description of the experimental designs used to measure the efficacy of an IPLE. The final section offers a summary of the study and a discussion of the potential benefits and opportunities resulting from IPLE implementation. 2. IPLE implementation 2.1. How an IPLE works In order to convey an experience of interaction comparable to actual practice, an IPLE requires students to complete a realistic task. For our IPLE implementation, the realistic task included a written assignment. Once completed, students’ written assignments were submitted to professionals for review. In the audit setting that serves as the backdrop for our study, professional auditors met with the students to discuss their work in a manner similar to the review process in practice. This oneon-one interaction between the professionals and students attempted to replicate the richness of the ‘‘on-the-job’’ learning environment of practice. The IPLE provides students with the opportunity to have their work critiqued by professionals who are focusing, not only on the line of reasoning students present, but also on how clearly and concisely they communicate their findings. The IPLE highlights that the scarcity of time for review during actual audit engagements necessitates such clarity and brevity. With this one-on-one professional critique, students receive feedback in an interactive setting which allows them to ask for the professionals’ advice regarding improvement of their work, thought processes, and/or writing style. In this way, students have an opportunity to benefit from the competence of the professionals, just as staff members would learn from their supervisors.5 Finally, the IPLE gives students a realistic sense of accountability to the professionals whom they have the chance to impress with their written work and/or their performance during the one-on-one interactions. The IPLE we used required students to evaluate the control environment and write a memo regarding the assessment of fraud risk for a hypothetical company.6 Instructors informed students that their risk-assessment memos would be reviewed by audit professionals from Big Four public accounting firms. The professionals reviewed the students’ memos as they would review the work of their firms’ staff accountants and prepared review comments. Nine audit professionals from two Big-Four public accounting firms served as the reviewers for our IPLE. The reviewers’ job titles ranged from audit senior to senior manager and all had extensive supervisory experience. Approximately 2 weeks after students submitted their memos, the practitioners met individually with each student whose work they reviewed to discuss their review comments. These one-on-one meetings, scheduled during class times, typically lasted between 10 and 20 min each. These discussions provided feedback to the students on the form and content of their memos, the appropriateness of their assessments, and the clarity of their written communications. The review comments were only discussed with the individual students and were not shared with the instructors. We told the professionals in advance that the only person who would see their review notes would be the individual student. We felt that this would encourage the professionals to be ‘‘blunt’’ with the students, since the review notes would not adversely affect the students’ grades. In addition to the opportunity to discuss the reviewers’ comments, students were able to inquire about the nature of the review process in practice. The IPLE concluded with the audit professionals conducting a question and answer session with the respective classes as a whole. 2.2. Suggestions for IPLE implementation in your classroom While the implementation of an IPLE may appear daunting, instructors may find that integrating an IPLE into the course is much less difficult than they might anticipate. In preparation for an IPLE, we 5 This meets one of the requirements of AACSB standards in that students are actively participating in the learning process (AACSB, 2012, Standard 14). 6 The professionals who reviewed the memos noted to us that the task was realistic and that similar memos would be used in their firms.
166
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
recommend meeting with representatives of the public accounting firms participating in the experience. Firm representatives typically see the potential value of an IPLE and are eager to participate. Coordination is a key factor in successfully implementing the IPLE. We found it useful to go through accounting firms’ campus recruiters and have the recruiters select the professionals from their respective firms for us. Due to the demanding schedules of the professionals, the date for their one-on-one meetings with students must be determined well in advance. Further, to accommodate for unanticipated changes in their schedules, it may be advisable to enlist more professionals than the class size would appear to indicate. For example, while four reviewers could comfortably conduct 30 one-onone meetings in a 2-h time period (averaging approximately 15 min), the absence of even one of the reviewers might cause significant disruptions to the review process. Providing the professionals with an adequate amount of time for their review of the students’ work prior to the date of the one-on-one meetings should make the professionals’ participation less onerous. We suggest that the IPLE be held approximately 2 weeks after the memos are collected from students and distributed to the professionals. This way, the case is still fresh in the students’ minds and yet the professionals have adequate time to prepare review comments. It may also be helpful to send a reminder email to students the day before the IPLE, encouraging them to review the case and their memos to refresh their memories so that they can benefit more from the professional interaction. An early consideration for instructors is choosing a case or task for the IPLE. Given the vast and varied number of cases available in textbooks, casebooks and the accounting literature, it can be a difficult choice. We used a case that dealt with assessing the control environment of a hypothetical company.7 It worked well to choose an unstructured task which required students to exercise professional judgment. We felt that it was better for the important part of the task to be documenting and explaining how/why the student came to his or her conclusion. This allowed for a good dialogue between the students and professionals. In fact, one of the biggest challenges for us with IPLE implementation was to keep the individual meetings on schedule. We found that the professionals and students enjoyed speaking with one another and often spent more than the suggested 10–20 min without realizing it. 3. Effectiveness of an IPLE We used an experimental design to test the effectiveness of our IPLE. One-hundred students served as participants for this study. Sixty-two students in two undergraduate auditing classes from different universities took part in the IPLE. In order to help determine the effectiveness of the IPLE, a third undergraduate auditing class with 38 students (from one of the two universities) served as a control group, receiving the same instructions and completing the same case assignment as the IPLE groups.8 3.1. Experimental designs Given that learning takes place with our IPLE over an extended period, an experimental design was used to evaluate the experience. Similar to Ashbaugh et al. (2002), we utilize both between-subjects and within-subjects research designs to mitigate the weaknesses of a field experiment (Cook & Campbell, 1979). Random assignment was not feasible given the implementation of the IPLE in a classroom setting. Additional assessment was performed to evaluate IPLE participants’ perceptions of their experience. 3.1.1. Between-subjects assessment Pre- and posttest assessments were administered to the students participating respectively in the IPLE and control groups (see Fig. 1). First, both groups of students were given a traditional lecture 7 The case we used was Dickinson Technologies (Agoglia, Brown, & Hanno, 2003). We chose this case because the answers were not ‘‘black and white,’’ and the students have to carefully scrutinize the materials before making a risk assessment based on professional judgment. 8 A potential threat to the validity of the experiment is that the control group is only from one university. However, conclusions presented in the study do not change if either of the two classes participating in the IPLE are excluded from the analyses. Consistent with prior studies, students were assigned to treatment and control groups by class, rather than by random assignment (e.g., Ashbaugh, Johnstone, & Warfield, 2002; Cleaveland & Larkins, 2004; Hite & Parry, 1994; Mohrweis, 1991). We have used a pretest/posttest design to help control for systematic differences between the classes.
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
167
IPLE Group: Lecture followed by pretest assessment
Writing assignment including IPLE
Posttest assessment
Writing assignment without IPLE
Posttest assessment
Control Group: Lecture followed by pretest assessment
Fig. 1. Between-subjects experimental design.
presentation covering the evaluation of the control environment and assessment of fraud risk. Following the lecture, both groups were asked to complete an instrument containing a skills test and a selfassessment of their knowledge of the audit material which served as a pretest. The skills test consisted of 15 relevant multiple-choice questions about the control environment and fraud risk. The questions were adapted from CPA examination review questions.9 The knowledge self-assessment questions asked students to rate the strength of their understanding of relevant aspects of the audit material as well as their understanding of the review process in auditing and the interaction between audit team members. Immediately following the pretest, both the IPLE and the control groups were given the same assignment to prepare professional memos evaluating the control environment and assess the fraud risk of a hypothetical company. At the end of the IPLE, the IPLE group completed an instrument containing questions regarding the effectiveness of the IPLE as a tool for learning the material, demographic questions,10 and the skills test and knowledge self-assessment questions from the pretest. This assessment served as a posttest. The control group responded to a similar posttest instrument containing these assessments after completion of the professional memo assignment. The responses of the IPLE and control groups were compared to determine if students’ performance on the skills test and knowledge perceptions differed between the two groups. 3.1.2. Within-subjects assessment IPLE group participants’ pre- and posttest responses were compared to determine whether students’ skills test performance and knowledge self-assessments improved after participating in the IPLE. No mention of the pretest or posttest assessments was made prior to their administration and no feedback regarding the skills test questions was provided prior to the posttest. 4. Results 4.1. Student performance on knowledge-based skills test We expected that participation in an IPLE would improve student learning of the relevant audit material, in our case, evaluation of the control environment and assessment of fraud risk. Pretest and posttest results on the knowledge-based skills test are shown in Table 1.11 The scores indicate the percentage of correct responses for the 15 multiple-choice questions included on the tests. Analysis of within-IPLE-group results (Panel A) indicates that participants’ scores on the post-IPLE skills test are 9 This is similar to a study by Mohrweis (1991) in which improvement in student writing skills was measured with pre- and posttests consisting of GMAT multiple choice questions. 10 The following demographic variables of participants were measured: gender, work experience and previous academic performance. 11 In order to determine if any of the demographic variables influenced the results reported in this study, separate analyses were performed which included the demographic variables as covariates. These analyses yielded results consistent with those presented in the text.
168
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
Table 1 Students’ performance on skills test (percentage of correct skills test responses). Treatment group Panel A IPLE (n = 62) Control (n = 38)
Mean (SD) Mean (SD)
Timing of measurement Panel B Pretest Posttest
Mean (SD) Mean (SD)
Pretest
Posttest
t Statistica
p Value
50.64 (14.81) 48.07 (12.61)
57.85 (13.75) 47.89 (14.36)
4.010
0.001
0.078
0.938
IPLE n = 62
Control n = 38
t Statistica
p Value
50.64 (14.81) 57.85 (13.75)
48.07 (12.61) 47.89 (14.36)
0.892
0.375
3.455
0.001
a Independent-samples t-test conducted. All tests are two-tailed. The Dunn–Bonferroni Method was utilized to control for family-wise error, with the family defined as all planned comparisons. This adjusts the significance level for all t-tests herein to alpha = 0.002. While this adjustment is overly conservative, the results are robust as to this choice of technique for control of Type I error.
significantly improved over their pretest scores, up 7.21% points (t = 4.010, p < 0.001). In contrast, the control group did not demonstrate significant improvement over a comparable time period (t = 0.078, p = 0.938). The between-subjects results (Panel B) indicate that, while the mean pretest scores for the IPLE and control groups were not significantly different (means = 50.64 and 48.07, respectively; p = 0.375), the IPLE group did perform significantly better than the control group for the posttest administration of the skills test (means = 57.85 and 47.89, respectively; p < 0.001). In addition, these changes in pre- and posttest scores were significantly higher (indicating greater improvement) for the IPLE group than for the control group (t = 2.549, p = 0.012). Overall, these results support the expectation that an IPLE can improve student learning. 4.2. Student perceptions We expected that participation in an IPLE would improve students’ self-perceptions of their knowledge of relevant audit material. Table 2 shows the results regarding student self-perceptions of their knowledge. Student perceptions were measured with five questions relating to their knowledge of the subject matter. The between-subjects results reveal that the IPLE group’s mean responses to all five perception-of-knowledge questions were significantly higher than those of the control group for the posttest administration of the instrument (all p’s < 0.001). Also, it is interesting to note that the IPLE participants’ perceived greater posttest understanding of these concepts occurred even though their pretest responses indicated that they were starting at a deficit with respect to the control group. Moreover, analysis of within-subjects (IPLE group) results indicates that participants’ mean responses to all five perception-of-knowledge questions were significantly higher in the posttest administration than in the pretest administration (all p’s < .001).12 These results provide additional support for the expectation that an IPLE can improve student learning. Further assessment of students’ perceptions of their knowledge included responses to four additional questions gathered from the IPLE group only. Table 3 shows results for these additional assessments. IPLE participants feel that the experience was an effective way for them to gain an understanding of the material. The IPLE participants also feel that the experience was an effective way to help them understand the review and supervision process, as well as to make abstract concepts from the textbook more concrete. During the posttest administration of the assessment instrument, students also were asked to respond to several items regarding their perceptions of the experience of participating in an IPLE. These results show that students viewed the IPLE experience favorably. Participation in the IPLE ap12 Again, no significant improvement occurred for the control group after completion of the same material and written assignment, but without IPLE.
169
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172 Table 2 Students’ self-perceptions of their knowledge. Perceptions of knowledgea Understanding of Control environment (PK1)
Timing of measurement Pretest Posttest
Fraud risk (PK2)
Pretest Posttest
Control environment assessment in practice (PK3)
Pretest Posttest
Fraud risk assessment in practice (PK4)
Pretest Posttest
Review process in practice (PK5)
Pretest Posttest
IPLE n = 62
Control n = 38
t Statisticb
p Value
Mean (SD) Mean (SD)
4.79 (1.55) 7.24 (1.07)
5.34 (1.65) 5.89 (1.59)
1.688
0.095
5.071
<0.001
Mean (SD) Mean (SD)
4.94 (1.64) 7.10 (1.13)
6.05 (1.94) 5.95 (1.52)
3.081
0.003
4.322
<0.001
Mean (SD) Mean (SD)
4.48 (1.74) 6.92 (1.08)
5.53 (1.77) 5.71 (1.52)
2.887
0.005
4.644
<0.001
Mean (SD) Mean (SD)
4.37 (1.60) 7.00 (1.06)
5.95 (1.80) 5.87 (1.58)
4.556
<0.001
4.294
<0.001
Mean (SD) Mean (SD)
4.87 (1.86) 6.89 (1.23)
5.79 (1.85) 5.82 (1.64)
2.403
0.018
3.716
<0.001
a PK1 through PK5 were measured by asking students to rate their perceptions of knowledge (e.g., for Control environment, ‘‘My understanding of the control environment component of internal control is. . .’’). Responses were indicated on nine-point scales with endpoints labeled ‘‘weak’’ and ‘‘strong’’ (coded 1 and 9, respectively). b Independent-samples t-test conducted. All tests are two-tailed.
Table 3 IPLE participants’ perceptions of their knowledge: additional post-IPLE perceptions. Mean (SD) n = 62
t Statisticb
p Value
Effective for understanding control environment and fraud risk (PK6)
7.65 (1.19)
17.522
<0.001
Effective for understanding review and supervision process (PK7)
7.29 (1.34)
13.505
<0.001
Made abstract concepts more concrete (PK8)
7.44 (1.14)
16.826
<0.001
Learned more using approach (PK9)
7.90 (1.08)
21.127
<0.001
Perceptions of knowledge
a
a PK6 and PK7 were measured by asking students to rate the effectiveness of the IPLE (e.g., ‘‘Is this interactive learning experience an effective way for students to gain an understanding of control environment and fraud risk assessment?’’). Responses were indicated on nine-point scales with endpoints labeled ‘‘not at all effective’’ and ‘‘very effective’’ (coded 1 and 9, respectively). PK8 and PK9 asked them to indicate their agreement with statements regarding the IPLE (e.g., ‘‘This interactive learning experience helped me put the abstract concepts of the text into a more concrete setting’’). Students recorded their responses on nine-point scales with endpoints labeled ‘‘strongly disagree’’ and ‘‘strongly agree’’ (coded 1 and 9, respectively). b One-sample t-test conducted using the scale midpoint of 5 as the test value. All tests are two-tailed.
pears to have stimulated interest in auditing and in auditing careers. Other responses indicate that the participants valued the experience as a networking opportunity and that the experience improved students’ perceptions of the Big-Four firms participating in the IPLE implementation (see Table 4).
170
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
Table 4 Students’ perceptions of IPLE. Perceptions of the experiencea Panel A: IPLE and control groups’ posttest assessment Interest in auditing stimulated (PE1) Interest in pursuing auditing career (PE2)
Perceptions of the experience
c
Panel B: IPLE participants’ perceptions post-IPLE assessment Effective for getting exposure to potential employers/ professionals (PE3)
IPLE n = 62
Control n = 38
t Statistic b
p Value
Mean (SD)
6.74 (1.34)
5.37 (1.85)
4.290
<0.001
Mean (SD)
6.95 (1.42)
5.34 (2.18)
4.470
<0.001
Mean (SD) n = 62
t Statistic
p Value
7.74
18.412
<0.001
14.304
<0.001
d
(1.17) Impression of participating accounting firm (PE4)
7.58 (1.42)
a PE1 asked IPLE participants to report their agreement with a statement indicating the experience stimulated their interest in auditing. Students in the control group were asked to respond to a similar statement regarding the material they were presented on control environment and fraud risk. Responses were indicated on nine-point scales with endpoints labeled ‘‘strongly disagree’’ and ‘‘strongly agree’’ (coded 1 and 9, respectively). PE2 asked students to rate their interest in pursuing a career in auditing after completing the IPLE (or, for students in the control group, after completing the material presented on control environment and fraud risk). Responses were indicated on nine-point scales with endpoints labeled ‘‘less interested’’ and ‘‘more interested’’ (coded 1 and 9, respectively). b Independent-samples t-test conducted. All tests are two-tailed. c PE3 asked IPLE participants to rate the effectiveness of the experience for getting exposure to potential employers and professionals in the accounting field. Responses were indicated on nine-point scales with endpoints labeled ‘‘not at all effective’’ and ‘‘very effective’’ (coded 1 and 9, respectively). PE4 asked IPLE participants to characterize their impressions of the participating accounting firm. Responses were indicated on nine-point scales with endpoints labeled ‘‘more negative’’ and ‘‘more positive’’ (coded 1 and 9, respectively). d One-sample t-test conducted using the scale midpoint of 5 as the test value. All tests are two-tailed.
4.3. Audit professionals’ perceptions In addition to assessing student perceptions of this IPLE, we also surveyed the audit professionals who performed the reviews to assess their perceptions of the experience. Table 5 presents the results of the practitioner survey. The professionals’ responses suggest that they viewed the IPLE as an effective method of relating the audit material and that they view an IPLE as effective in conveying knowledge of audit decision making as well as review and supervision. The responses also suggest that the practitioners believed the IPLE to be useful in assessing students’ interpersonal skills and technical abilities.
5. Summary and discussion This study describes how to implement and documents the pedagogical value of an interactive professional learning experience, or IPLE. Our IPLE engages practitioners in the learning process by bringing them together with students in a professional supervisory setting. Practitioners review students’ work and then meet with students one-on-one to provide feedback on their work. During these interactions, students also have the opportunity to ask the professionals questions relating to the specific topical material and about the practice environment in general. The efficacy of an IPLE was evaluated and the results indicate that performance on the skills test improved for those students participating in the IPLE. Further, skills test performance for students who completed the IPLE was superior to that of the control group. Additionally, students’ self-perceptions of their knowledge increased significantly after the IPLE and were significantly higher than those
171
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172 Table 5 Practitioner reviewers’ perceptions of efficacy of IPLE. Practitioner reviewers’ perceptionsa
b
Mean (SD) n=9
t Statistic
6.89 (1.17)
4.857
0.001
In making abstract concepts more concrete (RP2)
7.56 (1.33)
5.750
<0.001
In teaching subject matter (RP3)
7.22 (1.39)
4.781
0.001
For understanding practice decisions about CE and FR assessment (RP4)
6.89 (1.17)
4.857
0.001
For understanding review and supervision process (RP5)
7.11 (0.93)
6.825
<0.001
For assessing interpersonal skills (RP6)
7.44 (0.88)
8.315
<0.001
For assessing technical abilities (RP7)
7.00 (1.50)
4.000
0.004
Is IPLE effective For understanding CE and FR assessment (RP1)
p Value
a RP1 through RP7 were measured by asking reviewers to rate the efficacy of the IPLE (e.g., for RP1, ‘‘Is this interactive learning experience an effective way for students to gain an understanding of control environment and fraud risk assessment?’’). Responses were indicated on nine-point scales with endpoints labeled ‘‘not at all effective’’ and ‘‘very effective’’ (coded 1 and 9, respectively). b One-sample t-test conducted using the scale midpoint of 5 as the test value. All tests are two-tailed.
of the control group. Students who participated in the IPLE indicated that they had a better understanding of fraud risk assessment, as well as a better understanding of the review process in auditing. The students’ perceptions of the experience itself also were positive, as they viewed the experience as an effective means to get exposure to potential employers and professionals. The perceptions of the audit professionals confirm student perceptions about the effectiveness of the IPLE. The professionals indicated that the IPLE seemed an effective way to teach the material and to convey knowledge of audit practice. Further, their responses indicated the potential for IPLE to provide them with insights into students’ interpersonal skills and technical abilities. As the results of this study strongly suggest, an IPLE may provide learning opportunities, particularly for courses and topics in which classroom presentation is challenging (e.g., for abstract concepts and areas in which qualitative interpretation of information is required). However, the IPLE described in this study also presents potential benefits beyond mastery of a particular subject matter. The IPLE provides an occasion to create and enhance a unique relationship which links professionals, students, and academic programs in precisely the type of mutually beneficial endeavor called for in recent appraisals of accounting education (e.g., AAA, 2003; ACAP, 2008; Albrecht & Sack, 2000; PwC, 2003; The Pathways Commission, 2012). Along with potential benefits for professionals and students, programs incorporating such experiences in their curricula may gain from adopting this innovation. The close collaboration with professionals to implement such experiences may help ensure the relevance of course content and enable more effective infusion of professional identity and knowledge into the classroom. Similarly, an IPLE may provide a venue for the program to showcase its innovative pedagogy to the professional community. It is interesting to note that the professionals who served as reviewers in this study expressed a more positive view of the respective departments/programs after participating in the IPLE. Also, such experiences may foster closer contact with alumni and other professionals who might not otherwise have occasion to become involved with the program. While on-campus recruiting efforts may involve only one or two individuals from a particular firm, implementing an IPLE would likely involve the participation of additional professionals, further expanding the program’s chances to build relationships in the professional community.
172
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
In addition to using the IPLE in an auditing context, this innovative approach is appropriate for use in other accounting courses as well. For example, financial accounting instructors could have students assess the quality of an accounting estimate or the choice of an appropriate accounting method. An IPLE might involve a systems analysis in an accounting information systems course. Similarly, an IPLE could be developed for relevant settings in tax or managerial accounting courses. While professionals from public accounting firms can serve in audit supervisory roles, accountants working in industry can provide valuable perspectives for an IPLE in these other accounting contexts. While the IPLE described in this study occurred in a higher-level, undergraduate accounting course, including this experience in earlier accounting courses may offer distinct advantages as well. Using an IPLE in the first corporate financial reporting course (e.g., Intermediate I) could provide students with helpful insights at an earlier stage in their career discernment process. Also, providing networking opportunities to students earlier in their coursework may lead to future internships and subsequent placement opportunities. In fact, earlier introduction may further enhance the appeal of participating in the IPLE for public accounting firms. While graduate students may not need the same career direction as undergraduates, graduate curricula also might be enhanced by the practical experience inherent in an IPLE. Thus, given its broad potential for application, the IPLE may prove to be an effective and interesting way of enhancing the learning experience of students and getting professionals more engaged in accounting education. References Advisory Committee on the Auditing Profession (ACAP) (2008). Final report to the Department of the Treasury. Washington, DC: The US Department of the Treasury. Agoglia, C. P., Brown, K. F., & Hanno, D. M. (2003). Dickinson Technologies, Inc.: Assessing control environment and fraud risk. Issues in Accounting Education, 18(1), 71–78. Agrawal, S. P., & Siegel, P. H. (1991). Introduction to accounting: An experiment in experiential learning theory. Southwest Business Review, 1, 99–118. Albrecht, W. S., & Sack, R. J. (2000). Accounting education: Charting the course through a perilous future. Accounting education series (Vol. 16). Sarasota, FL: American Accounting Association. American Accounting Association (2003). Response by the federation of schools of accountancy to the pricewaterhousecoopers position on accounting education. Accounting Programs Leadership Group Newsletter, 27(2), 4–6. American Institute of Certified Public Accountants (AICPA) (2011a). Codification of auditing standards and procedures. New York, NY: AICPA. American Institute of Certified Public Accountants (AICPA) (2011b). Statement on auditing standards number 122: Clarification and recodification. New York, NY: AICPA. Ashbaugh, H., Johnstone, K. M., & Warfield, T. D. (2002). Outcome assessment of a writing-skill improvement initiative: Results and methodological implications. Issues in Accounting Education, 17(2), 123–148. Association to Advance Collegiate Schools of Business International (AACSB) (2012). Eligibility procedures and accreditation standards for business accreditation. Tampa, FL: AACSB. Cleaveland, M. C., & Larkins, E. R. (2004). Web-based practice and feedback improve tax students’ written communication skills. Journal of Accounting Education, 22, 211–228. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin. Hite, P. A., & Parry, R. W. (1994). A study in the effectiveness of writing exercises as elaboration techniques for teaching tax. Journal of the American Taxation Association, 16(1), 172–186. Mohrweis, L. C. (1991). The impact of writing assignments on accounting students’ writing skills. Journal of Accounting Education, 9, 309–325. PricewaterhouseCoopers (PwC) (2003). Educating for the public trust: The PricewaterhouseCoopers position on accounting education. New York, NY: PricewaterhouseCoopers LLP. The Pathways Commission (2012). The Pathways Commission on higher education: Charting a national strategy for the next generation of accountants. Sarasota, FL: American Accounting Association.
Contents lists available at SciVerse ScienceDirect
J. of Acc. Ed. journal homepage: www.elsevier.com/locate/jaccedu
Main article
The effectiveness of interactive professional learning experiences as a pedagogical tool: Evidence from an audit setting Maria H. Sanchez a,⇑, Christopher P. Agoglia b,1, Kevin F. Brown c,2 a
Department of Accounting, Rider University, 2083 Lawrenceville Road, Lawrenceville, NJ 08648, USA Department of Accounting & Information Systems, Isenberg School of Business, University of Massachusetts Amherst, 121 Presidents Drive, Amherst, MA 01003, USA c Department of Accountancy, Raj Soin College of Business, Wright State University, 3640 Colonel Glenn Highway, Dayton, OH 45435, USA b
a r t i c l e
i n f o
Article history: Available online 26 August 2012 Keywords: Interactive professional learning experience Auditing education Professional supervisory setting Control environment Fraud risk assessment
a b s t r a c t This paper describes an interactive professional learning experience (IPLE) and provides guidance for implementing an IPLE in an audit classroom. The IPLE described in this paper exposes students to a realistic practice environment within the classroom by bringing practitioners together with students in a professional supervisory setting. Practitioners review students’ work and then meet with students one-on-one to provide feedback on their work. We also document evidence of the pedagogical value of an IPLE by using a between-subjects experimental design in which learning outcomes for participants are compared to a control group that received the same instructions and completed the same written assignment, but did not participate in the professional interaction. In addition, pre- and posttests of students’ audit knowledge allowed for a within-subjects self-assessment of knowledge acquisition. The results strongly suggest that participation in the IPLE improves students’ performance on a skills test of relevant audit material and increases their self-perceptions of knowledge gained. In addition, results indicate that both students and audit professionals consider the IPLE a positive professional learning experience. Ó 2012 Elsevier Ltd. All rights reserved.
⇑ Corresponding author. Tel.: +1 609 895 5582; fax: +1 609 896 5304. E-mail addresses: [email protected] (M.H. Sanchez), [email protected] (C.P. Agoglia), [email protected] (K.F. Brown). 1 Tel.: +1 413 545 5582. 2 Tel.: +1 937 775 3138. 0748-5751/$ - see front matter Ó 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.jaccedu.2012.08.004
164
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
1. Introduction Meeting with supervisors who inspect and evaluate one’s work is common practice in the auditing profession. This meeting with its accompanying professional interaction presents a unique learning environment in which advice, insights, and knowledge can be shared. This kind of hierarchical professional interaction is not something to which students are traditionally exposed.3 However, it is possible for instructors to bring this experience to the classroom, in turn helping to make the real-life work environment more tangible and improve student learning. The purpose of this paper is to show how an interactive professional learning experience (IPLE) can help improve student learning. In the audit function, the interaction between supervisor and subordinate plays a critical role in ensuring audit quality, so much so that the process is formalized into the supervision process described in the First Standard of Fieldwork of Generally Accepted Auditing Standards (GAASs) and implicitly recognized in the First General Standard of GAAS, (AICPA, 2011a, 2011b, AU 150.02).4 While supervisor and subordinate interaction plays a critical role in the audit, the real-world interaction entailed in this process is difficult for students to appreciate. Agrawal and Siegel (1991) show that a lack of real-world experience often impairs the ability of students to understand basic concepts and procedures, particularly with regard to complex subjects such as auditing. The Advisory Committee on the Auditing Profession, in its Final Report to the Department of the Treasury (ACAP), notes that often new accounting faculty have little professional experience and that it is difficult for academics to acquire ‘‘practice-oriented’’ knowledge. ACAP supports more interaction between faculty and the profession (ACAP, 2008, VI, pp. 19–22). In addition, a recent report from The Pathways Commission (2012, p. 56) strongly supports the involvement of practitioners in the classroom. Incorporating interaction with professionals in a realistic context may improve knowledge acquisition while providing students with an interesting perspective on practice not readily available in the traditional classroom setting. This paper describes an IPLE and also provides guidance and support for instructors interested in using an IPLE in their classes. The IPLE examined in this study involved a professional review of students’ written submissions for a risk assessment task. In order to create a realistic environment, auditors from Big-Four public accounting firms reviewed students’ memos documenting the control environment and fraud risk assessments for a hypothetical company. After reviewing each student’s memo, the practitioners prepared review notes and later met one-on-one with each student to review his/her work. This meeting allowed students to interact with, and learn from, their reviewers in a professional manner and setting. These one-on-one meetings provided students with insights into what it is like to be a member of an actual audit team. The face-to-face interaction also allowed students to discuss the merits of their work and ask questions of their reviewers. Reviewers suggested ways for students to improve their memos, which ranged from alternate problem-solving approaches and evidence interpretation to more effective written presentation of their work. Perhaps the most common feedback from the professionals was that the students needed to document their conclusions more clearly and succinctly. We tested effectiveness of the IPLE for improving learning by comparing learning outcomes for students participating in the IPLE with students completing the same course material and writing assignment without professional interaction. Further evidence of effectiveness was obtained by performing knowledge assessments of students prior to and after completion of the IPLE. The results of this study strongly suggest that completing the IPLE improves students’ performance on a skills test of the relevant audit material. Also, students’ self-perceptions of their knowledge of the subject matter increase after the IPLE. In addition, results indicate that students consider the IPLE a positive interactive experience. The audit professionals participating in this IPLE perceived the experience to be effective for teaching the material and conveying an understanding of audit practice. The practitioners also appeared to find the IPLE useful, indicating that the IPLE provided them with insights into students’ interpersonal skills and technical abilities.
3 PricewaterhouseCoopers (2003, pp. 5–8) refers to such deficiencies in a critique of accounting education strongly advocating student interaction with professionals. 4 Formal statement of the 10 GAAS is dropped in AICPA’s (new) clarified standards that are applicable for audits of financial statements for years ending after December 15, 2012. Presentation of equivalent responsibilities can be found in sections AU-C 200, AU-C 220 and AU-C 300 of the clarified standards.
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
165
The remainder of this study is organized as follows. The next section discusses IPLE implementation. This is followed by a description of the experimental designs used to measure the efficacy of an IPLE. The final section offers a summary of the study and a discussion of the potential benefits and opportunities resulting from IPLE implementation. 2. IPLE implementation 2.1. How an IPLE works In order to convey an experience of interaction comparable to actual practice, an IPLE requires students to complete a realistic task. For our IPLE implementation, the realistic task included a written assignment. Once completed, students’ written assignments were submitted to professionals for review. In the audit setting that serves as the backdrop for our study, professional auditors met with the students to discuss their work in a manner similar to the review process in practice. This oneon-one interaction between the professionals and students attempted to replicate the richness of the ‘‘on-the-job’’ learning environment of practice. The IPLE provides students with the opportunity to have their work critiqued by professionals who are focusing, not only on the line of reasoning students present, but also on how clearly and concisely they communicate their findings. The IPLE highlights that the scarcity of time for review during actual audit engagements necessitates such clarity and brevity. With this one-on-one professional critique, students receive feedback in an interactive setting which allows them to ask for the professionals’ advice regarding improvement of their work, thought processes, and/or writing style. In this way, students have an opportunity to benefit from the competence of the professionals, just as staff members would learn from their supervisors.5 Finally, the IPLE gives students a realistic sense of accountability to the professionals whom they have the chance to impress with their written work and/or their performance during the one-on-one interactions. The IPLE we used required students to evaluate the control environment and write a memo regarding the assessment of fraud risk for a hypothetical company.6 Instructors informed students that their risk-assessment memos would be reviewed by audit professionals from Big Four public accounting firms. The professionals reviewed the students’ memos as they would review the work of their firms’ staff accountants and prepared review comments. Nine audit professionals from two Big-Four public accounting firms served as the reviewers for our IPLE. The reviewers’ job titles ranged from audit senior to senior manager and all had extensive supervisory experience. Approximately 2 weeks after students submitted their memos, the practitioners met individually with each student whose work they reviewed to discuss their review comments. These one-on-one meetings, scheduled during class times, typically lasted between 10 and 20 min each. These discussions provided feedback to the students on the form and content of their memos, the appropriateness of their assessments, and the clarity of their written communications. The review comments were only discussed with the individual students and were not shared with the instructors. We told the professionals in advance that the only person who would see their review notes would be the individual student. We felt that this would encourage the professionals to be ‘‘blunt’’ with the students, since the review notes would not adversely affect the students’ grades. In addition to the opportunity to discuss the reviewers’ comments, students were able to inquire about the nature of the review process in practice. The IPLE concluded with the audit professionals conducting a question and answer session with the respective classes as a whole. 2.2. Suggestions for IPLE implementation in your classroom While the implementation of an IPLE may appear daunting, instructors may find that integrating an IPLE into the course is much less difficult than they might anticipate. In preparation for an IPLE, we 5 This meets one of the requirements of AACSB standards in that students are actively participating in the learning process (AACSB, 2012, Standard 14). 6 The professionals who reviewed the memos noted to us that the task was realistic and that similar memos would be used in their firms.
166
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
recommend meeting with representatives of the public accounting firms participating in the experience. Firm representatives typically see the potential value of an IPLE and are eager to participate. Coordination is a key factor in successfully implementing the IPLE. We found it useful to go through accounting firms’ campus recruiters and have the recruiters select the professionals from their respective firms for us. Due to the demanding schedules of the professionals, the date for their one-on-one meetings with students must be determined well in advance. Further, to accommodate for unanticipated changes in their schedules, it may be advisable to enlist more professionals than the class size would appear to indicate. For example, while four reviewers could comfortably conduct 30 one-onone meetings in a 2-h time period (averaging approximately 15 min), the absence of even one of the reviewers might cause significant disruptions to the review process. Providing the professionals with an adequate amount of time for their review of the students’ work prior to the date of the one-on-one meetings should make the professionals’ participation less onerous. We suggest that the IPLE be held approximately 2 weeks after the memos are collected from students and distributed to the professionals. This way, the case is still fresh in the students’ minds and yet the professionals have adequate time to prepare review comments. It may also be helpful to send a reminder email to students the day before the IPLE, encouraging them to review the case and their memos to refresh their memories so that they can benefit more from the professional interaction. An early consideration for instructors is choosing a case or task for the IPLE. Given the vast and varied number of cases available in textbooks, casebooks and the accounting literature, it can be a difficult choice. We used a case that dealt with assessing the control environment of a hypothetical company.7 It worked well to choose an unstructured task which required students to exercise professional judgment. We felt that it was better for the important part of the task to be documenting and explaining how/why the student came to his or her conclusion. This allowed for a good dialogue between the students and professionals. In fact, one of the biggest challenges for us with IPLE implementation was to keep the individual meetings on schedule. We found that the professionals and students enjoyed speaking with one another and often spent more than the suggested 10–20 min without realizing it. 3. Effectiveness of an IPLE We used an experimental design to test the effectiveness of our IPLE. One-hundred students served as participants for this study. Sixty-two students in two undergraduate auditing classes from different universities took part in the IPLE. In order to help determine the effectiveness of the IPLE, a third undergraduate auditing class with 38 students (from one of the two universities) served as a control group, receiving the same instructions and completing the same case assignment as the IPLE groups.8 3.1. Experimental designs Given that learning takes place with our IPLE over an extended period, an experimental design was used to evaluate the experience. Similar to Ashbaugh et al. (2002), we utilize both between-subjects and within-subjects research designs to mitigate the weaknesses of a field experiment (Cook & Campbell, 1979). Random assignment was not feasible given the implementation of the IPLE in a classroom setting. Additional assessment was performed to evaluate IPLE participants’ perceptions of their experience. 3.1.1. Between-subjects assessment Pre- and posttest assessments were administered to the students participating respectively in the IPLE and control groups (see Fig. 1). First, both groups of students were given a traditional lecture 7 The case we used was Dickinson Technologies (Agoglia, Brown, & Hanno, 2003). We chose this case because the answers were not ‘‘black and white,’’ and the students have to carefully scrutinize the materials before making a risk assessment based on professional judgment. 8 A potential threat to the validity of the experiment is that the control group is only from one university. However, conclusions presented in the study do not change if either of the two classes participating in the IPLE are excluded from the analyses. Consistent with prior studies, students were assigned to treatment and control groups by class, rather than by random assignment (e.g., Ashbaugh, Johnstone, & Warfield, 2002; Cleaveland & Larkins, 2004; Hite & Parry, 1994; Mohrweis, 1991). We have used a pretest/posttest design to help control for systematic differences between the classes.
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
167
IPLE Group: Lecture followed by pretest assessment
Writing assignment including IPLE
Posttest assessment
Writing assignment without IPLE
Posttest assessment
Control Group: Lecture followed by pretest assessment
Fig. 1. Between-subjects experimental design.
presentation covering the evaluation of the control environment and assessment of fraud risk. Following the lecture, both groups were asked to complete an instrument containing a skills test and a selfassessment of their knowledge of the audit material which served as a pretest. The skills test consisted of 15 relevant multiple-choice questions about the control environment and fraud risk. The questions were adapted from CPA examination review questions.9 The knowledge self-assessment questions asked students to rate the strength of their understanding of relevant aspects of the audit material as well as their understanding of the review process in auditing and the interaction between audit team members. Immediately following the pretest, both the IPLE and the control groups were given the same assignment to prepare professional memos evaluating the control environment and assess the fraud risk of a hypothetical company. At the end of the IPLE, the IPLE group completed an instrument containing questions regarding the effectiveness of the IPLE as a tool for learning the material, demographic questions,10 and the skills test and knowledge self-assessment questions from the pretest. This assessment served as a posttest. The control group responded to a similar posttest instrument containing these assessments after completion of the professional memo assignment. The responses of the IPLE and control groups were compared to determine if students’ performance on the skills test and knowledge perceptions differed between the two groups. 3.1.2. Within-subjects assessment IPLE group participants’ pre- and posttest responses were compared to determine whether students’ skills test performance and knowledge self-assessments improved after participating in the IPLE. No mention of the pretest or posttest assessments was made prior to their administration and no feedback regarding the skills test questions was provided prior to the posttest. 4. Results 4.1. Student performance on knowledge-based skills test We expected that participation in an IPLE would improve student learning of the relevant audit material, in our case, evaluation of the control environment and assessment of fraud risk. Pretest and posttest results on the knowledge-based skills test are shown in Table 1.11 The scores indicate the percentage of correct responses for the 15 multiple-choice questions included on the tests. Analysis of within-IPLE-group results (Panel A) indicates that participants’ scores on the post-IPLE skills test are 9 This is similar to a study by Mohrweis (1991) in which improvement in student writing skills was measured with pre- and posttests consisting of GMAT multiple choice questions. 10 The following demographic variables of participants were measured: gender, work experience and previous academic performance. 11 In order to determine if any of the demographic variables influenced the results reported in this study, separate analyses were performed which included the demographic variables as covariates. These analyses yielded results consistent with those presented in the text.
168
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
Table 1 Students’ performance on skills test (percentage of correct skills test responses). Treatment group Panel A IPLE (n = 62) Control (n = 38)
Mean (SD) Mean (SD)
Timing of measurement Panel B Pretest Posttest
Mean (SD) Mean (SD)
Pretest
Posttest
t Statistica
p Value
50.64 (14.81) 48.07 (12.61)
57.85 (13.75) 47.89 (14.36)
4.010
0.001
0.078
0.938
IPLE n = 62
Control n = 38
t Statistica
p Value
50.64 (14.81) 57.85 (13.75)
48.07 (12.61) 47.89 (14.36)
0.892
0.375
3.455
0.001
a Independent-samples t-test conducted. All tests are two-tailed. The Dunn–Bonferroni Method was utilized to control for family-wise error, with the family defined as all planned comparisons. This adjusts the significance level for all t-tests herein to alpha = 0.002. While this adjustment is overly conservative, the results are robust as to this choice of technique for control of Type I error.
significantly improved over their pretest scores, up 7.21% points (t = 4.010, p < 0.001). In contrast, the control group did not demonstrate significant improvement over a comparable time period (t = 0.078, p = 0.938). The between-subjects results (Panel B) indicate that, while the mean pretest scores for the IPLE and control groups were not significantly different (means = 50.64 and 48.07, respectively; p = 0.375), the IPLE group did perform significantly better than the control group for the posttest administration of the skills test (means = 57.85 and 47.89, respectively; p < 0.001). In addition, these changes in pre- and posttest scores were significantly higher (indicating greater improvement) for the IPLE group than for the control group (t = 2.549, p = 0.012). Overall, these results support the expectation that an IPLE can improve student learning. 4.2. Student perceptions We expected that participation in an IPLE would improve students’ self-perceptions of their knowledge of relevant audit material. Table 2 shows the results regarding student self-perceptions of their knowledge. Student perceptions were measured with five questions relating to their knowledge of the subject matter. The between-subjects results reveal that the IPLE group’s mean responses to all five perception-of-knowledge questions were significantly higher than those of the control group for the posttest administration of the instrument (all p’s < 0.001). Also, it is interesting to note that the IPLE participants’ perceived greater posttest understanding of these concepts occurred even though their pretest responses indicated that they were starting at a deficit with respect to the control group. Moreover, analysis of within-subjects (IPLE group) results indicates that participants’ mean responses to all five perception-of-knowledge questions were significantly higher in the posttest administration than in the pretest administration (all p’s < .001).12 These results provide additional support for the expectation that an IPLE can improve student learning. Further assessment of students’ perceptions of their knowledge included responses to four additional questions gathered from the IPLE group only. Table 3 shows results for these additional assessments. IPLE participants feel that the experience was an effective way for them to gain an understanding of the material. The IPLE participants also feel that the experience was an effective way to help them understand the review and supervision process, as well as to make abstract concepts from the textbook more concrete. During the posttest administration of the assessment instrument, students also were asked to respond to several items regarding their perceptions of the experience of participating in an IPLE. These results show that students viewed the IPLE experience favorably. Participation in the IPLE ap12 Again, no significant improvement occurred for the control group after completion of the same material and written assignment, but without IPLE.
169
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172 Table 2 Students’ self-perceptions of their knowledge. Perceptions of knowledgea Understanding of Control environment (PK1)
Timing of measurement Pretest Posttest
Fraud risk (PK2)
Pretest Posttest
Control environment assessment in practice (PK3)
Pretest Posttest
Fraud risk assessment in practice (PK4)
Pretest Posttest
Review process in practice (PK5)
Pretest Posttest
IPLE n = 62
Control n = 38
t Statisticb
p Value
Mean (SD) Mean (SD)
4.79 (1.55) 7.24 (1.07)
5.34 (1.65) 5.89 (1.59)
1.688
0.095
5.071
<0.001
Mean (SD) Mean (SD)
4.94 (1.64) 7.10 (1.13)
6.05 (1.94) 5.95 (1.52)
3.081
0.003
4.322
<0.001
Mean (SD) Mean (SD)
4.48 (1.74) 6.92 (1.08)
5.53 (1.77) 5.71 (1.52)
2.887
0.005
4.644
<0.001
Mean (SD) Mean (SD)
4.37 (1.60) 7.00 (1.06)
5.95 (1.80) 5.87 (1.58)
4.556
<0.001
4.294
<0.001
Mean (SD) Mean (SD)
4.87 (1.86) 6.89 (1.23)
5.79 (1.85) 5.82 (1.64)
2.403
0.018
3.716
<0.001
a PK1 through PK5 were measured by asking students to rate their perceptions of knowledge (e.g., for Control environment, ‘‘My understanding of the control environment component of internal control is. . .’’). Responses were indicated on nine-point scales with endpoints labeled ‘‘weak’’ and ‘‘strong’’ (coded 1 and 9, respectively). b Independent-samples t-test conducted. All tests are two-tailed.
Table 3 IPLE participants’ perceptions of their knowledge: additional post-IPLE perceptions. Mean (SD) n = 62
t Statisticb
p Value
Effective for understanding control environment and fraud risk (PK6)
7.65 (1.19)
17.522
<0.001
Effective for understanding review and supervision process (PK7)
7.29 (1.34)
13.505
<0.001
Made abstract concepts more concrete (PK8)
7.44 (1.14)
16.826
<0.001
Learned more using approach (PK9)
7.90 (1.08)
21.127
<0.001
Perceptions of knowledge
a
a PK6 and PK7 were measured by asking students to rate the effectiveness of the IPLE (e.g., ‘‘Is this interactive learning experience an effective way for students to gain an understanding of control environment and fraud risk assessment?’’). Responses were indicated on nine-point scales with endpoints labeled ‘‘not at all effective’’ and ‘‘very effective’’ (coded 1 and 9, respectively). PK8 and PK9 asked them to indicate their agreement with statements regarding the IPLE (e.g., ‘‘This interactive learning experience helped me put the abstract concepts of the text into a more concrete setting’’). Students recorded their responses on nine-point scales with endpoints labeled ‘‘strongly disagree’’ and ‘‘strongly agree’’ (coded 1 and 9, respectively). b One-sample t-test conducted using the scale midpoint of 5 as the test value. All tests are two-tailed.
pears to have stimulated interest in auditing and in auditing careers. Other responses indicate that the participants valued the experience as a networking opportunity and that the experience improved students’ perceptions of the Big-Four firms participating in the IPLE implementation (see Table 4).
170
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
Table 4 Students’ perceptions of IPLE. Perceptions of the experiencea Panel A: IPLE and control groups’ posttest assessment Interest in auditing stimulated (PE1) Interest in pursuing auditing career (PE2)
Perceptions of the experience
c
Panel B: IPLE participants’ perceptions post-IPLE assessment Effective for getting exposure to potential employers/ professionals (PE3)
IPLE n = 62
Control n = 38
t Statistic b
p Value
Mean (SD)
6.74 (1.34)
5.37 (1.85)
4.290
<0.001
Mean (SD)
6.95 (1.42)
5.34 (2.18)
4.470
<0.001
Mean (SD) n = 62
t Statistic
p Value
7.74
18.412
<0.001
14.304
<0.001
d
(1.17) Impression of participating accounting firm (PE4)
7.58 (1.42)
a PE1 asked IPLE participants to report their agreement with a statement indicating the experience stimulated their interest in auditing. Students in the control group were asked to respond to a similar statement regarding the material they were presented on control environment and fraud risk. Responses were indicated on nine-point scales with endpoints labeled ‘‘strongly disagree’’ and ‘‘strongly agree’’ (coded 1 and 9, respectively). PE2 asked students to rate their interest in pursuing a career in auditing after completing the IPLE (or, for students in the control group, after completing the material presented on control environment and fraud risk). Responses were indicated on nine-point scales with endpoints labeled ‘‘less interested’’ and ‘‘more interested’’ (coded 1 and 9, respectively). b Independent-samples t-test conducted. All tests are two-tailed. c PE3 asked IPLE participants to rate the effectiveness of the experience for getting exposure to potential employers and professionals in the accounting field. Responses were indicated on nine-point scales with endpoints labeled ‘‘not at all effective’’ and ‘‘very effective’’ (coded 1 and 9, respectively). PE4 asked IPLE participants to characterize their impressions of the participating accounting firm. Responses were indicated on nine-point scales with endpoints labeled ‘‘more negative’’ and ‘‘more positive’’ (coded 1 and 9, respectively). d One-sample t-test conducted using the scale midpoint of 5 as the test value. All tests are two-tailed.
4.3. Audit professionals’ perceptions In addition to assessing student perceptions of this IPLE, we also surveyed the audit professionals who performed the reviews to assess their perceptions of the experience. Table 5 presents the results of the practitioner survey. The professionals’ responses suggest that they viewed the IPLE as an effective method of relating the audit material and that they view an IPLE as effective in conveying knowledge of audit decision making as well as review and supervision. The responses also suggest that the practitioners believed the IPLE to be useful in assessing students’ interpersonal skills and technical abilities.
5. Summary and discussion This study describes how to implement and documents the pedagogical value of an interactive professional learning experience, or IPLE. Our IPLE engages practitioners in the learning process by bringing them together with students in a professional supervisory setting. Practitioners review students’ work and then meet with students one-on-one to provide feedback on their work. During these interactions, students also have the opportunity to ask the professionals questions relating to the specific topical material and about the practice environment in general. The efficacy of an IPLE was evaluated and the results indicate that performance on the skills test improved for those students participating in the IPLE. Further, skills test performance for students who completed the IPLE was superior to that of the control group. Additionally, students’ self-perceptions of their knowledge increased significantly after the IPLE and were significantly higher than those
171
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172 Table 5 Practitioner reviewers’ perceptions of efficacy of IPLE. Practitioner reviewers’ perceptionsa
b
Mean (SD) n=9
t Statistic
6.89 (1.17)
4.857
0.001
In making abstract concepts more concrete (RP2)
7.56 (1.33)
5.750
<0.001
In teaching subject matter (RP3)
7.22 (1.39)
4.781
0.001
For understanding practice decisions about CE and FR assessment (RP4)
6.89 (1.17)
4.857
0.001
For understanding review and supervision process (RP5)
7.11 (0.93)
6.825
<0.001
For assessing interpersonal skills (RP6)
7.44 (0.88)
8.315
<0.001
For assessing technical abilities (RP7)
7.00 (1.50)
4.000
0.004
Is IPLE effective For understanding CE and FR assessment (RP1)
p Value
a RP1 through RP7 were measured by asking reviewers to rate the efficacy of the IPLE (e.g., for RP1, ‘‘Is this interactive learning experience an effective way for students to gain an understanding of control environment and fraud risk assessment?’’). Responses were indicated on nine-point scales with endpoints labeled ‘‘not at all effective’’ and ‘‘very effective’’ (coded 1 and 9, respectively). b One-sample t-test conducted using the scale midpoint of 5 as the test value. All tests are two-tailed.
of the control group. Students who participated in the IPLE indicated that they had a better understanding of fraud risk assessment, as well as a better understanding of the review process in auditing. The students’ perceptions of the experience itself also were positive, as they viewed the experience as an effective means to get exposure to potential employers and professionals. The perceptions of the audit professionals confirm student perceptions about the effectiveness of the IPLE. The professionals indicated that the IPLE seemed an effective way to teach the material and to convey knowledge of audit practice. Further, their responses indicated the potential for IPLE to provide them with insights into students’ interpersonal skills and technical abilities. As the results of this study strongly suggest, an IPLE may provide learning opportunities, particularly for courses and topics in which classroom presentation is challenging (e.g., for abstract concepts and areas in which qualitative interpretation of information is required). However, the IPLE described in this study also presents potential benefits beyond mastery of a particular subject matter. The IPLE provides an occasion to create and enhance a unique relationship which links professionals, students, and academic programs in precisely the type of mutually beneficial endeavor called for in recent appraisals of accounting education (e.g., AAA, 2003; ACAP, 2008; Albrecht & Sack, 2000; PwC, 2003; The Pathways Commission, 2012). Along with potential benefits for professionals and students, programs incorporating such experiences in their curricula may gain from adopting this innovation. The close collaboration with professionals to implement such experiences may help ensure the relevance of course content and enable more effective infusion of professional identity and knowledge into the classroom. Similarly, an IPLE may provide a venue for the program to showcase its innovative pedagogy to the professional community. It is interesting to note that the professionals who served as reviewers in this study expressed a more positive view of the respective departments/programs after participating in the IPLE. Also, such experiences may foster closer contact with alumni and other professionals who might not otherwise have occasion to become involved with the program. While on-campus recruiting efforts may involve only one or two individuals from a particular firm, implementing an IPLE would likely involve the participation of additional professionals, further expanding the program’s chances to build relationships in the professional community.
172
M.H. Sanchez et al. / J. of Acc. Ed. 30 (2012) 163–172
In addition to using the IPLE in an auditing context, this innovative approach is appropriate for use in other accounting courses as well. For example, financial accounting instructors could have students assess the quality of an accounting estimate or the choice of an appropriate accounting method. An IPLE might involve a systems analysis in an accounting information systems course. Similarly, an IPLE could be developed for relevant settings in tax or managerial accounting courses. While professionals from public accounting firms can serve in audit supervisory roles, accountants working in industry can provide valuable perspectives for an IPLE in these other accounting contexts. While the IPLE described in this study occurred in a higher-level, undergraduate accounting course, including this experience in earlier accounting courses may offer distinct advantages as well. Using an IPLE in the first corporate financial reporting course (e.g., Intermediate I) could provide students with helpful insights at an earlier stage in their career discernment process. Also, providing networking opportunities to students earlier in their coursework may lead to future internships and subsequent placement opportunities. In fact, earlier introduction may further enhance the appeal of participating in the IPLE for public accounting firms. While graduate students may not need the same career direction as undergraduates, graduate curricula also might be enhanced by the practical experience inherent in an IPLE. Thus, given its broad potential for application, the IPLE may prove to be an effective and interesting way of enhancing the learning experience of students and getting professionals more engaged in accounting education. References Advisory Committee on the Auditing Profession (ACAP) (2008). Final report to the Department of the Treasury. Washington, DC: The US Department of the Treasury. Agoglia, C. P., Brown, K. F., & Hanno, D. M. (2003). Dickinson Technologies, Inc.: Assessing control environment and fraud risk. Issues in Accounting Education, 18(1), 71–78. Agrawal, S. P., & Siegel, P. H. (1991). Introduction to accounting: An experiment in experiential learning theory. Southwest Business Review, 1, 99–118. Albrecht, W. S., & Sack, R. J. (2000). Accounting education: Charting the course through a perilous future. Accounting education series (Vol. 16). Sarasota, FL: American Accounting Association. American Accounting Association (2003). Response by the federation of schools of accountancy to the pricewaterhousecoopers position on accounting education. Accounting Programs Leadership Group Newsletter, 27(2), 4–6. American Institute of Certified Public Accountants (AICPA) (2011a). Codification of auditing standards and procedures. New York, NY: AICPA. American Institute of Certified Public Accountants (AICPA) (2011b). Statement on auditing standards number 122: Clarification and recodification. New York, NY: AICPA. Ashbaugh, H., Johnstone, K. M., & Warfield, T. D. (2002). Outcome assessment of a writing-skill improvement initiative: Results and methodological implications. Issues in Accounting Education, 17(2), 123–148. Association to Advance Collegiate Schools of Business International (AACSB) (2012). Eligibility procedures and accreditation standards for business accreditation. Tampa, FL: AACSB. Cleaveland, M. C., & Larkins, E. R. (2004). Web-based practice and feedback improve tax students’ written communication skills. Journal of Accounting Education, 22, 211–228. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin. Hite, P. A., & Parry, R. W. (1994). A study in the effectiveness of writing exercises as elaboration techniques for teaching tax. Journal of the American Taxation Association, 16(1), 172–186. Mohrweis, L. C. (1991). The impact of writing assignments on accounting students’ writing skills. Journal of Accounting Education, 9, 309–325. PricewaterhouseCoopers (PwC) (2003). Educating for the public trust: The PricewaterhouseCoopers position on accounting education. New York, NY: PricewaterhouseCoopers LLP. The Pathways Commission (2012). The Pathways Commission on higher education: Charting a national strategy for the next generation of accountants. Sarasota, FL: American Accounting Association.