The influence of scope and timing of reliability assurance in B2B e-commerce

International Journal of Accounting Information Systems 7 (2006) 115 – 129

The influence of scope and timing of reliability assurance in B2B e-commerce Elaine G. Mauldin a,⁎, Andreas I. Nicolaou b,1 , Stacy E. Kovar c,2 a b

School of Accountancy, 331 Cornell Hall, University of Missouri-Columbia, Columbia, MO 65211, United States Department of Accounting and MIS, Bowling Green State University, Bowling Green, OH 43403, United States c Department of Accounting, Calvin 109, Kansas State University, Manhattan, KS 66503, United States Received 8 May 2005; received in revised form 20 September 2005; accepted 26 September 2005

Abstract This study investigates potential demand for third-party assurance reports in business-to-business electronic commerce (B2B e-commerce). We experimentally analyze 95 purchasing professionals' decisions to recommend using a B2B exchange. The experiment is a 2 × 2 between-participants design varying the scope and timing of an assurance report with an additional control condition of no assurance. The results suggest that purchasing professionals are more likely to recommend use of the exchange when general assurance over the reliability of the exchange's system is present than when specific assurance over the reliability of transaction information is present. Purchasing professionals are also more likely to recommend using the exchange when the assurance report is continuous than when it is static, issued at a point in time. However, the results also suggest that participants are less likely to recommend using the exchange when specific information assurance or static assurance is present than when assurance is not present at all. Further, other factors besides assurance, especially trust in the trading partner and propensity to trust, have a stronger influence on the decision to use a B2B exchange than the presence of either continuous or systems assurance. Potential implications for practice include redesigning existing assurance services to provide continuous assurance, de-emphasizing formalized reports and considering assurance services over other factors, such as web site design. © 2006 Elsevier Inc. All rights reserved. Keywords: Assurance services; Continuous audit; E-commerce

⁎ Corresponding author. Tel.: +1 573 884 0933; fax: +1 573 882 2437. E-mail addresses: [email protected] (E.G. Mauldin), [email protected] (A.I. Nicolaou), [email protected] (S.E. Kovar). 1 Tel.: +1 419 372 2932; fax: +1 419 372 2875. 2 Tel.: +1 785 532 6083; fax: +1 785 532 5959. 1467-0895/$ - see front matter © 2006 Elsevier Inc. All rights reserved. doi:10.1016/j.accinf.2005.09.002

116

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

1. Introduction The purpose of this study is to experimentally explore the potential demand for assurance reports beyond financial statements using transaction cost economic theory. In spite of earlier unsuccessful attempts at promoting assurance reports in business-to-consumer (B2C) e-commerce, the AICPA/CICA has recently focused significant efforts on continuing to develop assurance related to systems reliability through their newly combined Trust Services Principles and Criteria (AICPA, 2003). Consistent with transaction cost economic theory, we suggest that to be more successful, the scope and timing of future assurance reports may need to be targeted to address the specific transaction cost concerns in a given business environment. We test this theory by comparing the efficacy of different assurance reports on purchasing professionals' decisions to recommend using a B2B e-commerce exchange (also referred to as an e-hub), where buyers and sellers exchange information and execute spot market transactions for critical, hard to locate, high-value manufacturing goods in real time. In addition to being an unproven target market for systems reliability assurance, B2B exchanges are characterized by information risk concerns that can be addressed by assurance mechanisms (Kalyvas, 2002). Continuous flows of information, remoteness of buyers and sellers, constantly changing environments and high demand for reliable information and efficient exchange mechanisms give rise to high coordination and motivation costs (Garicano and Kaplan, 2001). Third-party assurance may be used to reduce these transaction costs. First, we vary the scope of the assurance report, comparing general assurance over systems reliability to more specific assurance over the underlying information in the system, suggesting that these assurances best address coordination and motivation costs respectively. The AICPA/ CICA Trust Services Principles and Criteria are an example of general systems reliability assurance. The Trust Services criteria outline principles for a broad examination of management's assertions related to systems reliability, based on information availability, security, maintainability and confidentiality. Mixed results in past B2C research related to WebTrust, a general systems reliability precursor to Trust Services (for example, Kaplan and Nieschwietz, 2003; Kovar et al., 2000; Lala et al., 2003; Mauldin and Arunachalam, 2002), suggest that a general assurance report may not always be the most effective alternative. Research by Boritz and Hunton (2002) suggests a greater value for more specific assurances, implying that users seek out assurance related to their most salient concerns. Examples of more specific assurance might include assurances whose scope covers only privacy policies (Greenstein and Hunton, 2002), or information pertinent to debt covenants associated with loans (Woodroof and Searcy, 2001). In the B2B exchange setting, the specific information expected to be most salient to buyers on the exchange concerns suppliers' ability to deliver the quantity of goods ordered to specification and on time. Thus, we compare the effect of an assurance report addressing exchange management's assertions about controls over specific supplier information to the effect of a general systems reliability assurance report. Evidence about decision-makers' preference for general assurance about systems reliability versus specific assurance about information reliability should enhance our understanding of the demand for assurance services. Second, we vary the timing of the assurance report. Current instantiations of Trust Services assurance reports remain static, issued on a periodic basis. However, Vasarhelyi (2002) suggests that a series of audit risks has emerged that cannot be properly addressed by traditional, periodic assurance. One example he gives is e-business, because the processes are highly automated and essentially faceless. Pathak et al. (2005) also suggest that monitoring must be continuous to be effective in complex processing systems where transaction volume is high, such as in e-

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

117

commerce. We add that continuous monitoring is important when the nature and associated risks from motivation and coordination costs are constantly changing because of dynamic systems and information. Therefore, we also offer evidence about decision-makers' preference for continuous versus static assurance, again to further enhance our understanding of the demand for assurance services. We conducted a two-by-two, between-participants experiment, manipulating the scope (general systems reliability or specific information reliability) and timing (continuous or static) of the assurance report. Professionals involved in business purchasing activities participated in an online experiment using a case about a fictitious B2B exchange used for spot purchasing of critical, well-defined raw materials being evaluated for use in a business setting. The dependent variable was a rating of the likelihood the participant would recommend the exchange to their firm. We find that participants were more likely to recommend use of the exchange when general systems reliability assurance was present than when specific information reliability assurance was present and when continuous assurance was present than when static assurance was present. In addition, we find that participants were less likely to recommend using the exchange when specific information assurance or static assurance was present than when assurance was not present at all. Further, we find that a number of other factors, most notably trust in the trading partner and propensity to trust, have a stronger influence on the decision to use a B2B exchange than the presence of either continuous or systems assurance. The study contributes to the assurance service, continuous audit and B2B literatures. First, we contribute to the assurance service literature by examining different types of reports, providing insight into the efficacy of general purpose reports compared to more specific targeted reports. Bedard et al. (2005) suggest more research is needed in the specific context of systems assurance, including how users' expectations about systems assurance influence their actions. Second, we contribute to the continuous audit literature by demonstrating clear preference for continuous versus static assurance in a B2B setting. The AICPA/CICA calls for research to determine circumstances where users perceive continuous audit by an objective third party to enhance decision making by improving the reliability of information. Finally, we contribute to B2B research by examining how one form of monitoring, assurance services, may impact potential use of exchanges. In addition, by providing ex ante evidence about the value of different assurance service characteristics compared to other e-commerce characteristics, the study may also assist practice in developing services. The remainder of this paper is organized as follows. Section 2 presents background information and develops hypotheses. Section 3 describes the research design and participant demographics and Section 4 presents the empirical findings. Section 5 provides a discussion and opportunities for future research. 2. Background and hypotheses B2B encompasses a wide variety of environments through which businesses transact with one another. The B2B exchange utilizes portals maintained by intermediaries to provide a structured marketplace for buyers and sellers (Kaplan and Sawhney, 2000). A major goal of B2B exchanges is aggregation, bringing together many sellers and buyers, making their products/demand available in one place and providing commonly needed services. A second goal of B2B exchanges is matching, bringing together sellers with immediate availability of product with buyers with immediate need (Kaplan and Sawhney, 2000). In addition to aggregation and matching services, B2B exchanges may provide buyer/seller certification, fulfillment, inspection, receivables and financing (Kaplan and Sawhney, 2000). We examine a B2B exchange focused on the spot

118

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

sourcing of expensive and critical manufacturing inputs, products that are needed in a short time frame, but that can be explained with generic descriptions. Thus, the primary function of the hub in our experiment was matching of buyers and suppliers with corresponding needs and availability. This scenario was created to create a clear need for both general systems reliability assurance, as well as specific assurance over the matching information provided by suppliers through the exchange, without introducing biases related to preexisting relationships between the buyer and supplier. To understand assurance needs related to B2B exchanges, we use economic theory related to transaction costs (Milgrom and Roberts, 1992). Garicano and Kaplan (2001) suggest that the success of business relationships in B2B depends on the ability of technology to reduce transaction costs, both coordination costs and motivation costs. B2B exchanges seek to reduce coordination costs by providing an efficient system to match buyers and sellers and to allow them to transact, directly reducing the cost of an activity already being conducted through process improvements and marketplace benefits. Motivation costs are associated with informational asymmetries and imperfect commitment (Milgrom and Roberts, 1992). Though e-commerce is often believed to accentuate motivation costs, B2B exchanges seek to reduce such costs by providing supplier screening and standardized processes with real-time information updates and an electronic trail (Garicano and Kaplan, 2001), making it difficult for buyers and sellers to misrepresent their capabilities. The desire to reduce coordination costs and motivation costs each provide a potential need for assurance. To reduce coordination costs, the reliability of the system provided for conducting transactions must be high to assure that desired savings from more efficient transactions will be realized and will not be offset by losses incurred due to lack of availability and/or security of systems as well as inaccurate transaction processing. Trust Services assurance development stems directly from this type of need, because it is analogous to, and a direct extension of, traditional audit service demand derived from information risk. Assurance services, however, could also be developed over reductions in motivation costs. In a B2B exchange, the reliability of information provided by the exchange about seller capabilities is vital, because it helps assure that the parties to the exchange have more complete and accurate information about one another's capabilities, thus reducing motivation costs. While systems reliability, as a precursor to information reliability, may address motivation costs somewhat, assurance specifically targeted at the underlying information is more likely to curb intentional misrepresentation by sellers. Garicano and Kaplan (2001) study a B2B exchange in the auto industry and find that the greatest benefits were in business process improvements (coordination costs). However, the exchange took measures to reduce motivation costs and, as a result, there was little evidence that motivation costs were higher than in the physical world. Garicano and Kaplan (2001) conclude that reduced coordination costs are the first-order benefit over reduced motivation costs. Systems reliability assurance, which addresses the availability, security, integrity, and confidentiality of transactions conducted at a B2B exchange, addresses coordination costs by assuring that the exchange does indeed provide a reliable vehicle for conducting transactions. Assurance specifically over supplier's delivery, volume and specification performance addresses motivation costs by assuring that the information provided by suppliers is reliable. Thus, information assurance may also be a measure used to reduce motivation costs. The tension in this study is that it is not entirely clear whether general or specific assurance is more important. On the one hand, the buyer may be able to rely on longstanding relationships with the exchange provider for assurance about general systems reliability, while demanding

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

119

more assurance about the information provided by suppliers because it is more dynamic and less reliable, given the potential motivation of suppliers to misrepresent their capabilities. On the other hand, because systems reliability assurance addresses coordination costs and is a prerequisite for information reliability, systems reliability assurance may sufficiently increase B2B exchange participants' confidence in the reliability of the information utilized by the exchange such that there may be higher demand for systems assurance. This prediction is supported by Boritz and Hunton (2002), who find significantly stronger results for the influence of availability and security assurances over the operations of an application software provider than for integrity and maintainability, suggesting that systems-related assurances will be more important than data-related assurances. Thus, our first hypothesis suggests: H1. General systems reliability assurance will be valued more that specific information reliability assurance. Continuous auditing is defined as a methodology that enables auditors to provide written assurance on information either simultaneously, or a short period of time after, the occurrence of events underlying the information (AICPA, 1999). While still in the early stages of development and rarely implemented in practice at the present time, continuous audit techniques have been developed to analyze and audit the real-time flow of information directed towards financial reporting (Murthy and Groomer, 2004; Alles et al., 2002; Woodroof and Searcy, 2001) or to internal controls (Murthy and Groomer, 2004). For example, Woodroof and Searcy (2001) develop a working prototype of a continuous audit of debt covenants for financial reporting purposes. One of the reasons for the slow growth of continuous audit is the slow growth of continuous reporting. Because e-commerce currently relies on real-time information flows, we suggest that continuous auditing may be demanded to support the need for assurance over potentially constantly changing information. In particular, in e-commerce and especially spot purchasing situations like the one examined in this paper, the level of risk from both coordination and motivation is constantly changing as both the number of suppliers and individual supplier information changes. Rational economic parties are expected to re-evaluate their level of risk whenever using the exchange, indicating that continuous assurance will be particularly valued. H2. Continuous assurance will be valued more than static assurance. However, continuous assurance may not be equally necessary for all e-commerce information. Systems, once developed with appropriate controls, may be quite stable. Additionally, in a B2B exchange, the relationship between buyer and exchange is fixed, reducing the risk associated with coordination costs. Periodic, or static, assurance that the controls are continuing to function may be sufficient. On the other hand, in a B2B exchange with spot purchasing, the buyer/seller relationship is fluid, resulting in a greater ongoing change to the risk situation in relation to motivation costs. Thus, real time information about the information provided by sellers may be needed to address this heightened risk. Hence, real-time or continuous assurance may be more important for specific information than for general systems information. H3. Continuous assurance will be valued more for specific information reliability assurance than for general systems reliability assurance. 3. Methods An online experiment was designed to examine the two variables described above. In the experimental task, participants were provided with a scenario describing them as a decision-

120

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

Table 1 Profile of 95 purchasing participants' background and attitudes

Panel A: Participants' background Age Years of purchasing work experience Percent of purchases conducted electronically (beyond e-mail) Gender Male Female Degree in accounting Yes No CPA Yes No Panel B: Attitudes (answers on various 7-point scales) Design of MetalMarket's web sitea Propensity to trustb Trust in suppliers in the scenarioc Complexity of technologyd a b c d

Mean

Std. Dev.

Minimum

Maximum

42.42 9.80 45.05

8.90 8.56 32.65

21 0 0

65 40 100

Count 58 37 19 76 4 91

5.04 3.72 4.33 4.13

1.23 1.39 1.55 1.48

1.67 1 1 1

7 7 7 7

Average of questions regarding the attractiveness, ease of navigation, and response time for the web site. Average of four questions regarding individual propensity to trust in generic situations. Propensity to trust suppliers in this specific purchasing scenario. Rating of the complexity of process and technology in the specific B2B exchange context.

maker evaluating a B2B exchange as a possible means for increasing the efficiency of sourcing critical, hard-to-locate inputs. A fictitious B2B exchange, MetalMarkets, was created for the experiment. As part of the scenario, participants were provided with instructions to go to the MetalMarkets web site and complete two large purchases of goods as a means for evaluating the site. In the process of completing the transactions, participants were exposed to one of five possible experimental conditions. The first four cross the scope of the report (either over the reliability of MetalMarkets' system or the reliability of information about supplier capabilities) and the timing of the report (either static or continuous). A fifth, control, condition provides no form of assurance and is intended to provide a baseline of attitudes toward the fictitious B2B exchange. The reports in the scenario were based on a Trust Services report issued by Ernst and Young displayed on the Covisant web site during 2003–2004. 3.1. Participant demographics Participants were 95 adults between 21 and 65 years of age. Participants were either individuals with current or past direct purchasing experience or managers with authority over purchasing-related activities. Participants were contacted through local chapters of the Institute for Supply Management in several cities located throughout the mid-west, or through students enrolled in one authors' executive MBA course. Though a few of the participants were actually students in the executive MBA course, most were other individuals employed by the same organizations as the MBA students.

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

121

Panel A of Table 1 provides demographic data on these individuals. Their mean age was just over 42 and 37 were females. On average, they possessed 9.8 years of purchasing experience, with an average of 45% of purchases conducted electronically.3 Only 19 participants had an accounting degree and 4 were CPAs. Thus, as a group, they possess the experience necessary to act as reasonable proxies for the actual purchasing managers who would make decisions about purchasing from an electronic exchange. Additional analyses (not shown) controlled for these demographic factors and none were found significant except purchasing experience. We include purchasing experience in our multivariate analyses. 3.2. Dependent variable After completing the two large purchase transactions, participants were asked to assess the likelihood of the following statement being true “I would recommend that my company enter into a membership agreement with MetalMarkets to purchase large, critical raw materials.” This question was assessed using a 7-point Likert scale.4 3.3. Independent variables and manipulation checks Two different variables were manipulated in the experiment—the scope of the assurance and its timing. Assurance scope was manipulated by varying the assurance report viewed by the subject in terms of whether it addressed management's assertions about the “reliability of the system for transacting with suppliers (i.e., the systems information availability, security, integrity, and confidentiality)” or management's assertions about controls over screening of the “reliability of suppliers (i.e. whether they deliver goods on time, in the right quantities, and to specification).” In reference to the actual activities conducted as part of the audit, the systems report referred to procedures related to general systems controls, while the information assurance referred to tests of screening controls as well as comparison of all suppliers' actual performance with performance listed on the e-hub. Assurance timing was manipulated by varying whether the assurance report was stated as of the current date and specified continuous assurance or whether it was stated as of the end of the previous quarter and specified assurance over a fixed three-month period. Appendix A provides samples of the continuous audit report covering systems reliability and the static audit report covering information reliability to give an idea of the wording differences. In addition to the four report conditions, in a control condition participants received neither report. For participants who received the reports, three manipulation check questions were used. Participants were asked their level of agreement, using 7-point Likert scales, regarding whether (1) an independent accountants' report verifying the reliability (i.e., high information availability, security, integrity, and confidentiality) of MetalMarkets e-hub was provided on the web site, (2) an 3 As a sensitivity analysis, we analyzed the data after deleting nine participants without any actual purchasing experience. The results (not shown) are substantially the same as that shown in Table 2. In particular, Scope and Timing remained significant at p = 0.05 and p = 0.04, respectively. 4 Participants also assessed two other questions, for which the responses were highly correlated. (1) Purchasing large, critical raw materials through MetalMarkets e-hub will provide my company with reliable suppliers (i.e. suppliers who deliver goods on time, in the right quantities, and to specification). (2) Purchasing large, critical raw materials through MetalMarkets e-hub will provide my company with a reliable system for transacting with suppliers (i.e., a system with high information availability, security, integrity, and confidentiality). Results for these questions were in the same direction, but not as strong as those for the variable reported in this study.

122

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

Table 2 ANCOVA Model of purchasing participants' estimates of the % likelihooda they would recommend using MetalMarkets exchange by assurance scope and timing Effect

df

SSE

F

p

Scope Timing Scope × Timing Design of MetalMarket's Web Siteb Propensity to Trust c Trust in suppliers in the scenariod Complexity of technologye Years of purchasing work experience Error Model F Adjusted R2 = 0.54

1 1 1 1 1 1 1 1 65

2195.1 3699.2 51.43 3378.3 5071.2 12205.1 4305.9 3267.6

4.09 6.90 0.10 6.30 9.43 22.75 8.03 6.09

0.047 0.011 0.757 0.015 0.003 b0.001 0.006 0.017

9.37

b0.0001

a b c d e

From 0 to 100 percent. Average of questions regarding the attractiveness, ease of navigation, and response time for the web site. Average of four questions regarding individual propensity to trust. Propensity to trust suppliers in this specific purchasing scenario. Rating of the complexity of process and technology in the specific B2B exchange context.

independent accountants' report verifying MetalMarkets suppliers' delivery, volume and specifications performance was provided on the web site, and (3) the independent accountants' report(s) provided continuous assurance, current as of today's date. Participants who received the systems reliability report were significantly more likely to agree with the statement regarding the systems reliability assurance report than those who received the information reliability assurance report (mean = 5.44 versus 3.90, p b 0.01). Similarly, participants who received the information reliability assurance report were more likely to agree with the statements regarding the presence of the information reliability assurance (mean = 4.28 versus 3.34, p b 0.01). Finally, individuals receiving the continuous assurance report were more likely to agree with the statement about continuous assurance than those who received the static report (mean = 4.79 versus 3.32, p b 0.01).5 In addition to these manipulations, we sought to emphasize the importance of the decision to the company by emphasizing the importance of accurate information on vendor performance as well as systems reliability to the participants' employer. We asked two questions to confirm the degree to which participants believed that information and systems reliability were important. Means for both of these questions (measured on 7-point Likert scales) were high, suggesting that participants did indeed perceive that vendor and systems information were important (mean = 6.27 for vendor information importance and 6.08 for systems information importance). No significant differences existed between experimental conditions for these responses. Finally, two questions directly assessed participants' beliefs about the underlying need for assurance. Again, on a 7-point Likert scale, participants indicated they would strongly desire some assurance that the information (mean 6.51) and the system (mean 6.48) was reliable. No significant differences were found between experimental conditions for these responses. Thus, we conclude that the participants understood the experimental setting and the various experimental manipulations were successful. 5

As a sensitivity analysis, we analyzed the data after deleting six subjects answering the assurance scope or assurance timing manipulation question at the opposite end of the scale compared to the expected answer. The results (not shown) are substantially the same as that shown in Table 2 with the significance of all variables remaining substantially the same. In particular, Scope and Timing remained significant at p = 0.05 and p = 0.02, respectively.

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

123

Table 3 Analyses of % Likelihood a that purchasing participant would recommend using MetalMarkets exchange by assurance scope and timing comparison to control group Panel A: Assurance treatment cells—adjusted means Assurance scope

Information Reliability Systems Reliability Overall Mean Control—no assurance

Assurance timing Static mean (N)

Continuous mean (N)

Overall mean (N)

43.12 (22) 54.83 (18) 48.39 (40)

58.56 (17) 68.47 (17) 63.52 (34)

49.85 (39) 61.46 (35) 57.65 (74) 66.00 (19)

Panel B: Contrasts to Compare Cells F p-value Hypothesis 1: Systems Assurance will be more effective than Information Assurance Information to Systems 3.89 0.05 Control to Information 5.39 0.02 Control to Systems 0.44 0.51 Hypothesis 2: Continuous Assurance will be more effective than Static Assurance Static versus Continuous 7.34 0.01 Control to Static 7.04 0.01 Control to Continuous 0.14 0.71 Additional Follow-up Comparisons Control to Static Information Control to Continuous Information Control to Static System Control to Continuous System Info. Static to Info. Continuous Systems Static to System Continuous a

10.02 0.91 2.13 0.11 4.28 3.05

0.01 0.34 0.15 0.75 0.04 0.08

From 0 to 100%.

3.4. Control variables Information was also collected related to a number of attitudinal control variables. Selected descriptive statistics for these variables are provided in Panel B of Table 1, including participants' perceptions of the design of the MetalMarket's web site, general propensity to trust (Lee and Turban, 2001), the degree to which they believed they could trust the suppliers in the scenario, and the degree to which they believed the processes and technology for transacting at the B2B exchange would be complex. Perception of the web site design included question regarding attractiveness, ease of navigation and response time. Reliability of the measure was sufficient with a coefficient alpha of 0.75. The overall mean was 5.04 suggesting, in general, the participants found the web site design adequate. Propensity to trust was a 4-item measure taken from previous research (Lee and Turban, 2001). Reliability of the measure was again judged sufficient with a coefficient alpha of 0.92. The mean propensity to trust was about in the middle of the scale at 3.72. The mean propensity to trust suppliers in this specific purchasing scenario was measured with one question with a mean of 4.33. Finally, participants rated the complexity of process and technology in the specific B2B exchange context at 4.13. All four variables exhibited considerable range and are included in the analyses. Other possible control variables were collected, but do not appear in the paper because

124

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

they did not have a significant influence on the analysis. These include participants' perceptions of CPAs, perceptions of commitment to relationships with suppliers, perceptions of interdependence with suppliers, level of knowledge about suppliers, and the impact of poor decisions. 4. Results We analyzed the data using general linear model analysis of covariance (GLM), with individual cell mean comparisons using the control group as a benchmark to get at the nature of the effects observed. Results can be found in Tables 2 and 3. The GLM results in Table 2, which includes only the four treatment cells, show a significant main effect for both assurance scope (F = 4.09, p b 0.05) and timing (F = 6.90, p b 0.01), but no interactive effect, after removing the influence of the four attitudinal control variables and years of purchasing experience. Thus, potential support exists for Hypotheses 1 and 2, but not Hypothesis 3, which posited a strong effect for continuous assurance when the scope of the assurance was information reliability. To provide more detailed information regarding Hypotheses 1and 2, an additional GLM analysis was performed with the same control variables and the independent variable as the five cells in the experiment, including the control group. The results are provided in Table 3, with means (as adjusted for the control variables, i.e. lsmeans) shown in Panel A and the results of contrasts to perform hypothesis tests shown in Panel B. The cells are also plotted in Fig. 1. Examining Hypothesis 1 in more detail reveals that the overall likelihood that individuals receiving the information reliability assurance would recommend that their company use the B2B exchange was 49.85, while the likelihood that individuals receiving the systems reliability assurance would make the same recommendation was 61.46. These adjusted means were significantly different from 80

Likelihood of Recommending E-Hub Use

70

Systems (p=.08) 60

p=.002 50

Information (p=.04)

40

30

20

10

0 Control

Static

Continuous

Assurance Scope Fig. 1. Comparison of adjusted cell means.

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

125

one another (F = 3.89, p = 0.05). Thus, Hypothesis 1, that systems reliability assurance would be valued more than information reliability assurance, is supported. However, comparing to the control group reveals unexpected insights. The adjusted mean for the control group was 66.00, meaning that participants who did not see any assurance were, on average, more favorable regarding the B2B exchange than those who saw either information or systems reliability assurance. Using contrasts, the information reliability assurance groups were significantly worse than the control group (F = 5.39, p = 0.02), while there was no significant difference between the systems reliability assurance group and the control group (F = 0.44, p = 0.51). This suggests that the observed effect for assurance scope may have resulted from a negative influence for information reliability assurance and a neutral affect for systems assurance. This result will be discussed in more detail in the next section. Using Table 3 to analyze Hypothesis 2 suggests similar results to those for Hypothesis 1. The mean likelihood of recommending the B2B exchange for individuals receiving static assurance was 48.39, while the mean likelihood of recommending the B2B exchange for individuals receiving continuous assurance was 63.52. These means are significantly different from one another (F = 7.34, p = 0.01), supporting the superiority of continuous assurance in a B2B e-commerce setting. However, contrasting the static assurance groups to the control group suggests that static assurance had a significant negative influence (F = 7.04, p = 0.01), while continuous assurance had a neutral affect (F = 0.14, p = 0.71). Examining all cells in the design, significant or marginally significant differences exist between the control condition and the information static conditions (F = 10.02, p = 0.01), between the information static and information continuous conditions (F = 4.28, p = 0.04) and between the systems static and systems continuous conditions (F = 3.05, p = 0.08). These differences are labeled in Fig. 1 for clarity. 5. Discussion and opportunities for further research We conducted a 2 × 2, between-participants experiment varying the scope of assurance and the timing of assurance. Compared to a control group, the results suggest that both specific information reliability assurance and assurance provided on a static basis overall negatively influenced managers' decisions to recommend use of a B2B exchange. Further, both systems assurance and continuous assurance had negligible effects. These results are despite participants' answers to debriefing questions suggesting a strong need for assurance over both systems and information (means of 6.51and 6.48 on a 7-point scale). Hence, these results beg the question of why continuous systems assurance is not valued more by managers and when and how assurance providers might find ways to add value for companies, particularly those participating in B2B e-commerce. Other factors from our experiment may provide some guidance regarding these questions. In particular, the nature of the B2B exchange or the characteristics of the assurance service itself may suggest clues about when and how assurance may add value. Each of these factors and their implications for further research will be examined below. 5.1. The nature of the B2B exchange Kovar and Mauldin (2003) suggest that, in the face of a need for assurance, assurance providers are only one potential source of assurance. Another important source of assurance may be intermediaries. In the purchasing scenario examined in this study, the exchange provider is a vital intermediary. It is possible that data reliability controls evident in the design of the exchange itself provided a form of assurance, reducing the incremental value of third-party assurance. In particular, edit checks were conducted on each field to assure validity of data entered, participants

126

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

received confirmation of the accuracy of the data entered and the successful transmission of their purchase order. Supporting this idea, recent research in B2B (inter-organizational) exchanges empirically demonstrates that the presence of high control transparency and outcome feedback lead to positive judgments of information quality and contribute to enhanced trust to the exchange partner and reduced risk in the use of the exchange (Nicolaou and McKnight, 2005). Because they apply to every transaction, the presence of these controls may specifically influence the perceived value of continuous assurance. Another type of assurance provided by the exchange itself might involve initial screening of subjects as a requirement of exchange membership. Though subjects in our study perceived a need for assurance over the information provided by suppliers, to the extent that subject screening by the exchange is performed and results in increased trust in suppliers at the exchange, it may serve as a substitute for third-party assurance, even if the third party assurance offers more timely, specific assurance over the matching information used at the exchange. Hence, future research could directly examine whether or not controls and other features such as supplier screening offered by the exchange may substitute for third-party assurance, particularly third-party assurance provided on a continuous basis. 5.2. The characteristics of the assurance service While the assurance provided in our exchange was modeled after an actual assurance service displayed on a real B2B exchange web site, it is possible that participants simply did not view the assurance as valuable or even viewed it negatively. This may have related to the nature of the assurance provider or the nature of the report itself. Evidence related to the competence of the CPA as provider of the assurance was less than compelling. When asked to rate CPAs on a number of factors using a 7-point Likert scale where 1 represented high competence and 7 represented low competence, the average rating was 3.59. In debriefing, one participant suggested that similar, more valuable assurances would be available from other sources. Another individual, speaking on behalf of a colleague who completed the survey stated that, “She interpreted the questions as indicating that an accountant would be certifying the materials. If this is the case, her company would not use the system.” Past research involving ElderCare (Burke et al., 2003) and privacy (Greenstein and Hunton, 2002) both support the notion that CPAs may not be perceived as the best providers for all different types of assurance. Regarding the report, past research has suggested that consumers may be wary of the legalistic wording of assurances provided in typical assurance reports like the one used in this paper (Burke et al., 2003). Further, the presence of an assurance report displayed on the web site may draw attention to the risks associated with transacting at a B2B exchange–either in general, or for the specific exchange in question–thus making participants more wary of transacting. This explanation would also support the positive or neutral influence for similar, seal-based assurance found in past B2C research (Mauldin and Arunachalam, 2002; Kovar et al., 2000). These explanations of the study's results would suggest that care must be taken in displaying assurance reports on B2B exchange web sites. However, the results do not necessarily eliminate the possible benefits of assurance services themselves in B2B e-commerce. Assurance may still be used to improve actual information and systems reliability, which can increase managers' willingness to transact. Additionally, reports may be useful in some contexts. Boritz and Hunton (2002) found a positive influence for the SysTrust assurance and its components when managers were making decisions about using an ASP to implement an ERP system. One possible explanation for their conflicting findings was that their situation involved a closer relationship, with a report that was

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

127

prepared for a more specific versus generalized use. Thus, the characteristics of the report cited above may have not been as important. These explanations suggest that potential assurance providers should examine methods for providing assurance that do not involve public reports, or that use other means for disseminating their findings, at least in some situations. For example, consulting engagements that simply enhance the quality of the web site and its disclosures may be equally effective, especially in a B2B context. Future research could examine such alternative methods and those situations where they have a differential influence. 5.3. Other implications The results also may provide some insight into the relative influence of systems reliability versus information reliability assurance and static versus continuous assurance. In short, the results suggest that continuous assurance has a clear benefit above static assurance, and that systems assurance has a more favorable influence than assurance over the reliability of the specific information. Future research should examine these two types of assurance in concert. While information assurance had no significant influence in this study, future research should examine whether such assurance would have an influence when combined with systems assurance. In short, participants might value information assurance for reducing motivation costs when they are confident that the information is being generated by a reliable system, but would not value it when systems reliability is in question. These results should be interpreted in context of the effect sizes observed. Examining the GLM analysis in Table 2, we see that the most influential variable in the analysis is the subject's assessment of the probability that they could trust the supplier in the scenario. (The specific question was “I would trust metal suppliers to be honest and truthful in relations related to spot purchases of the type being conducted in the scenario.”). The next most influential variable was propensity to trust, followed by complexity of the B2B exchange technology, timing of assurance, design of the site, years of purchasing work experience and finally scope of the assurance. This suggests that other factors besides assurance have a stronger influence on the individual's decision to use a B2B exchange than the presence of either continuous or systems assurance, pointing to the multi-faceted nature of purchasing decisions. Assurance services that examine these factors, such as web site design, might provide greater value than the assurance examined in this study. Further, any research conducted should examine the underlying structure of trust relationships with the idea of exploring how users' threshold trust levels vary and how these variations influence the demand for different forms of assurance. As Bedard et al. (2005) note, it is difficult to explain demand for trust services in general absent better information on how and why trust varies across participants. 6. Conclusion Taken as a whole, the results draw into the question the value of traditional assurance structures in B2B exchange situations. We note, however, that these results may not hold in all ecommerce situations. The superiority of continuous assurance over static assurance suggests that participants may value assurance in different forms than that provided using the traditional audit model. Further research examining the influence of assurances that take different forms from the legalistic report-based assurance used in this study may provide additional insight into types of assurance that could provide greater value to B2B consumers. Additionally, research examining the interplay between context, in terms of relationship characteristics such as trust and

128

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

commitment, form of assurance and actual reliability of underlying systems might be particularly interesting. Appendix A. Systems Reliability, Continuous Report of Independent Accountants We examine management's assertion that MetalMarkets, Inc., on an ongoing basis as of July 15, 2004, maintains effective controls over the MetalMarkets E-hub and related applications to provide reasonable assurance that: • • • •

The MetalMarkets E-Hub is available for operation and use as committed or agreed; The MetalMarkets E-Hub is protected against unauthorized access (both physical and logical); The MetalMarkets E-Hub processing is complete, accurate, timely, and authorized; and Information designated as confidential is protected as committed or agreed.

Our examination is conducted on a continuous basis in accordance with the attestation standards established by the AICPA and, accordingly, includes (1) obtaining an understanding of the MetalMarkets' relevant availability, security, integrity, and confidentiality controls, (2) testing and evaluating the operating effectiveness of the controls, and (3) performing such other procedures as we consider necessary in the circumstances. We believe that our examination provides a reasonable basis for our opinion. In our opinion, MetalMarkets management's assertion referred to above related to availability, security, integrity and confidentiality of processing at their e-hub is fairly stated in all material respects. Report of Independent Accountants July 15, 2004 Appendix B. Information Reliability, Static Report of Independent Accountants We have examined management's assertion that MetalMarkets, Inc., during the period April 1, 2004 through June 30, 2004, maintained effective controls for screening suppliers to provide reasonable assurance that: • Related suppliers possessed the ability to deliver goods on time, • Related suppliers possessed the ability to deliver the quantities of goods identified, and • Related suppliers possessed the ability to deliver goods to specification. Our examination was conducted in accordance with the attestation standards established by the AICPA and, accordingly, included (1) obtaining an understanding of the MetalMarkets' relevant delivery, specifications, and volume performance screening controls, (2) testing and evaluating the operating effectiveness of the controls, (3) comparing all suppliers' actual performance with information listed on the e-hub for performance, and (4) performing such other procedures as we considered necessary in the circumstances. We believe that our examination provides a reasonable basis for our opinion. In our opinion, MetalMarkets management's assertion referred to above related to screening of suppliers for delivery, specifications, and volume requirements is fairly stated in all material respects. Report of Independent Accountants July 26, 2004

E.G. Mauldin et al. / International Journal of Accounting Information Systems 7 (2006) 115–129

129

References Alles MG, Kogan A, Vasarhelyi MA. Feasibility and economics of continuous assurance. Auditing A Journal of Practice and Theory, vol. 21.; 2002. p. 125–38. March. American Institute of Certified Public Accountants. Continuous Auditing. New York: AICPA; 1999. American Institute of Certified Public Accountants. Trust Services Principles and Criteria V. 1.0; 2003 [Online]. Available http://www.aicpa.org/. Bedard JC, Jackson CM, Graham L. Issues and risks in performing SysTrust engagements: implications for research and practice. Int J Account Inf Syst 2005;6(1):55–79. Boritz JE, Hunton JE. Investigating the impact of auditor-provided systems reliability assurance on potential service recipients. J Inf Syst 2002;16:69–87 (Supplement). Burke Kimberly Gladden, Kovar Stacy E, Prenshaw Penelope J. Understanding the Satisfaction Process for New Assurance Services: The Role of Attitudes, Expectations, Disconfirmation and Performance. Adv Account 2003;20. Garicano L, Kaplan SN. The effects of business-to-business e-commerce on transaction costs. J Ind Econ 2001;49 (4):463–85. Greenstein M, Hunton JE. Extending the accounting brand to privacy services. Arizona State University -West; 2002. Working paper. Kalyvas J. Judicious partnering. Optimizemag.com; 2002. p. 37–42. December. Kaplan Steven E, Nieschwietz Robert J. A Web Assurance Services Model of Trust for B2C E-commerce. Int J Account Inf Syst 2003;4:95-114. Kaplan S, Sawhney M. E-Hubs: The new b2b marketplaces. Harvard Bus Rev 2000:97–103 (May–June). Kovar SE, Mauldin EG. Antecedents of Demand for Assurance Services: A Model for Analysis with Applications in B2B E-Commerce. The University of Missouri - Columbia; 2003. Working paper. Kovar SE, Burke KG, Kovar BR. Selling WebTrust: An Exploratory Examination of Factors Influencing Consumers' Decisions to Use Online Distribution Channels. Rev Account Inf Syst 2000;4(2):39–52. Lala V, Arnold V, Sutton SG, Guan L. The Impact of Relative Information Quality of E-commerce Assurance Seals on Internet Purchasing Behavior. Int J Account Inf Syst 2003;3:237–53. Lee MKO, Turban E. A trust model for consumer internet shopping. Int J Electron Commer 2001;6(1):75–91. Mauldin EG, Arunachalam V. An Experimental Examination of Alternative Forms of Web Assurance for Business-toConsumer E-Commerce. J Inf Syst 2002:33–54 (Supplement). Milgrom P, Roberts J. Economics, Organization and Management. Englewood Cliffs, NJ: Prentice Hall; 1992. Murthy US, Groomer SM. A continuous auditing web services model for XML-based accounting systems. Int J Account Inf Syst 2004;5:139–63. Nicolaou AI, McKnight H. Perceived Information Quality in Data Exchanges: Effects on Risk, Trust, Performance, and Intention to Use. Bowling Green State University; 2005. Working paper. Pathak J, Chrouch B, Sriram RS. Minimizing cost of continuous audit: Counting and time dependent strategies. J Account Public Policy 2005;24:61–75. Vasarhelyi MA. Concepts in Continuous Assurance. In: Arnold Vicky, Sutton Steve G, editors. Researching Accounting as an Information Systems Discipline. Sarasota, FL: American Accounting Association; 2002. Woodroof J, Searcy D. Continuous Audit: Model Development and Implementation within a debt covenant compliance domain. Int J Account Inf Syst 2001;2:169–91.