Financial analysts' evaluation of enhanced disclosure of non-financial performance indicators

The British Accounting Review 43 (2011) 87–101

Contents lists available at ScienceDirect

The British Accounting Review journal homepage: www.elsevier.com/locate/bar

Financial analysts’ evaluation of enhanced disclosure of non-financial performance indicators Paul J. Coram a, *, Theodore J. Mock b, c, Gary S. Monroe d a

The University of Melbourne, Melbourne, VIC 3010, Australia University of California, Riverside, 900 University Ave., Riverside, CA 92521, USA c University of Maastricht, 6200 MD Maastricht, The Netherlands d The University of New South Wales, Sydney, NSW 2052, Australia b

a r t i c l e i n f o

a b s t r a c t

Article history: Received 19 June 2009 Received in revised form 19 November 2010 Accepted 19 November 2010

This study examines whether enhanced disclosure of non-financial performance indicators influences financial analysts’ decision processes and the information they pay attention to when performing stock-price valuations. These questions are addressed through a verbal protocol study that examines the information-processing behaviours and types of information used by analysts in valuing companies. The protocol analysis provides a detailed, descriptive analysis of the use of non-financial performance indicators in this task. The results demonstrate considerable attention to non-financial performance indicators. However, that attention was asymmetric depending on the trend-direction of the financial information. Financial information received greater attention when the trend was negative whereas non-financial performance indicators received greater attention when the financial information showed positive trends. Overall, these results elucidate the processes by which analysts utilise non-financial performance information in making valuation and subsequent investment decisions. Ó 2011 Elsevier Ltd. All rights reserved.

Keywords: Company valuation, enhanced disclosure Financial analysts, non-financial performance indicators Protocol analysis

1. Introduction Non-financial performance measures are becoming an important type of disclosure in the corporate environment as evidenced by calls for more of this type of disclosure by organisations such as the Enhanced Business Reporting Consortium (EBRC, 2005) and the Institute of Chartered Accountants in England and Wales (ICAEW, 2003). Non-financial performance measures are based on measures that complement financial statements such as “operational measures on customer satisfaction, internal business processes, and the organisation’s innovation and improvement activities” (Kaplan and Norton, 1992, p. 71).1 The importance of this issue is documented within the American Accounting Association’s Financial Accounting Standards Committee (AAA FASC, 2002) review of the research literature relating to the disclosure of non-financial performance measures. While this review demonstrated that there was value in non-financial disclosures, it also found that such

* Corresponding author. Tel.: þ61 3 8344 7023; fax þ61 3 9349 2397. E-mail address: [email protected] (P.J. Coram). 1 ‘Non-financial measures’ as discussed in this study are closely aligned with some of the measures as proposed by Kaplan and Norton (1992) for use in their BSC. As noted by Kaplan and Norton (1992, p. 71): “The balanced scorecard includes financial measures that tell the results of actions already taken. And it complements the financial measures with operational measures on customer satisfaction, internal processes, and the organisation’s innovation and improvement activities – operational measures that are the drivers of future financial performance.” (Italics added). 0890-8389/$ – see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.bar.2011.02.001

88

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

disclosures were relatively uncommon and that, when they occurred, they were varied and unstructured. The AAA FASC acknowledged that archival studies have found associations between certain types of non-financial disclosures and share prices but cautioned that these results show association and not necessarily evidence of usage. Three key research questions are examined in this study:  To what extent do financial analysts use non-financial performance indicators when they are performing company valuations?  Does the trend of the associated financial information affect the use of non-financial performance indicators? And  Does the trend of the financial information affect the way in which information is processed by analysts in undertaking a valuation? The major contribution to knowledge from this study and the way in which it differs from prior research into the value of non-financial disclosures (Amir & Lev, 1996; Ittner & Larcker, 1998; Banker, Potter, & Srinivasan, 2000) is that it assesses how non-financial performance indicators impact financial analysts’ decision processes by using verbal protocol analysis. Verbal protocol analysis is a research method that requires participants to ‘think aloud’ while performing a task (Ericsson & Simon, 1993). Verbalisations are taped and then transcribed to be used as evidence about the decision-making and informationevaluation processes. By using verbal protocol analysis, this study provides insights into information used in decision-making that cannot be provided by archival studies. This type of research also has advantages over those using experimental methods because it can provide more information on the way in which this information is used. The examination of how professional financial report users utilise non-financial disclosures is relevant to managers in companies thinking about enhanced disclosure policies, regulators considering mandating disclosure of this type of information, as well as to professional and non-professional financial report users in evaluating companies. This study therefore contributes to the discussion on the value of enhanced non-financial disclosure. A further contribution to knowledge of this study is the examination of whether positive or negative trends of financial information affect the level of attention participants pay to non-financial information, an issue not previously examined in the academic literature. For example, if analysts are using financial information with a negative trend, one might argue that analysts might seek more information about such negative financial information. Prior research in psychology does suggest that the trend of financial information analysts consider may affect analysts’ behaviour, particularly their use of non-financial information. This literature implies that negative information possesses greater diagnostic value (Skowronski & Carlston, 1989) and elicits more cognitive analysis (Taylor, 1991) than positive information, which suggests that individuals place greater reliability and reliance on negative information. The corollary of these findings is that positive information is perceived to be less diagnostically-valuable and receives less cognitive analysis. Therefore, it is anticipated that there would be less perceived-reliability and greater uncertainty about the disclosure of positive information. Under these circumstances, analysts would be expected to seek alternative information in evaluating companies. If non-financial performance indicators are perceived to be value-relevant, they would thus be expected to receive greater attention as an alternative information source. Such a finding would suggest context-specific reliance on nonfinancial performance indicators, which has not been previously examined in the literature. An archival study by Hutton, Miller, and Skinner (2003) found that managers were more likely to supplement good-news disclosures with verifiable forward-looking estimates – suggesting that companies believe the nature of disclosures affects users’ interest in other information. To address the general research question as to the use of non-financial performance information by users of financial reports, a verbal protocol study was performed using eight financial analysts as subjects. The research was conducted in 2003 in Australia during a fairly stable period in financial markets.2 The verbal protocol study required participants to value a medium-sized private company that was intending to list on the stock exchange. Participants were provided with the financial statements, for which there was an unqualified audit report, and non-financial information presented in a Balanced Scorecard (BSC) format. The data were collected by obtaining concurrent verbal protocols while the participants performed valuations. The verbal protocols indicate that substantial attention was paid to non-financial performance indicators by analysts when performing company valuations. However, the degree of attention paid to this information and the decision-making processes adopted differed depending on the trend of the associated financial information. Specifically, when financial information showed a positive trend, more attention was paid to non-financial performance indicators. Further, fewer ‘algebraic calculations’3 were performed, and there was greater reliance on information from memory. The remainder of the paper is structured as follows. Section 2 provides a review of the literature and the development of the research questions. Section 3 outlines the research method, which is use of the verbal protocol method. Section 4 presents the results and Section 5 concludes the paper.

2 The year 2003 was in a five year period (1999–2004) when the Australian Stock Exchange’s ‘All Ordinaries’ index stayed relatively stable. In 2004 it started to increase significantly until the collapse of 2007. 3 ‘Algebraic calculations’ or ACs are one of the operators a participant could utilise when completing the task. See Appendix 3 for definitions of the operators that were used.

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

89

2. Literature review and research question development A few studies using archival data have examined whether non-financial measures are useful in company valuations. These studies have found non-financial measures to be of value (Amir & Lev, 1996), particularly as leading indicators of financial performance (Ittner & Larcker, 1998; Banker et al., 2000). However, a recent investigation by Ittner, Larcker, and Randall (2003) shows that the use of balanced scorecards, which include both financial and non-financial performance measures, was not associated with stock return measures, although they were associated with higher measurement-system satisfaction. As noted, our study addresses a caution raised by the AAA FASC that archival results show ‘.whether the non-financial measure is associated with stock prices, not whether investors actually use the measure’ (AAA FASC, 2002, p. 355, emphasis added). By using verbal protocol analysis, we provide insights into the actual decision processes adopted and information used by analysts that are not captured by archival studies as examined by Research Questions 2 and 3 discussed later in this paper. Although not explicitly examining the use of non-financial information, Anderson (1988) in a verbal protocol study found that financial analysts showed significant interest in non-financial data, particularly in relation to product lines, competitors and customers. Bouwman, Frishkoff, and Frishkoff (1995) examined the importance of GAAP-based information to financial analysts in examining companies and found that such information primarily was used as a screening function for investments. In deciding to invest in a company, analysts looked at more qualitative, future-orientated, non-financial information. To expand this line of research, our first research question addresses the extent of use of non-financial performance indicators by analysts in performing company valuations. RQ1: To what extent do analysts use non-financial performance indicators in performing company valuations? Prospect theory suggests that investors are more sensitive to losses than gains (Kahneman & Tversky, 1979). Further, this theory is also applicable to positive and negative information. Consistent with prospect theory and framing-effect research4 is the hypothesis that the negative aspects of an object, event or choice are weighted more heavily than positive aspects in forming judgments (Kahneman & Tversky, 1984; Peeters & Czapinski, 1990). This literature suggests that individuals react more to losses or negative information than to gains or positive information. This proposition is supported by research showing that markets react more to negative information than to positive information (Brown & Harlow, 1988). Research has also documented the tendency for managers to delay communicating bad news (Chambers & Penman, 1984; Penman, 1987). These biases or tendencies are supported by a verbal protocol study by Anderson (1988), which found professional analysts attached greater weight to characteristics perceived as negative compared to those perceived as positive. Given this apparent ‘negativity bias’ of investors, it is expected that when financial information shows a negative trend analysts will focus relatively more on the financial information than other information that might be provided – such as nonfinancial disclosures. Alternatively, when financial information shows a positive trend, analysts will be less concerned about the positive historical financial information provided, and be more future-focussed by seeking alternative relevant information sources to provide support for the positive trend going forward. These are the circumstances in which we expect nonfinancial information to be of most interest and benefit to financial statement users. This proposition is consistent with the findings of Hutton et al. (2003) that managers are more likely to supplement good-news analyst forecasts with verifiable forward-looking estimates. These issues are addressed by investigating the following research question: RQ2In performing company valuations, do analysts under conditions of financial information showing a positive (negative) trend pay relatively more (less) attention to non-financial performance indicators and relatively less (more) attention to historical financial statements? As discussed, prospect theory suggests that individuals react more to losses or negative information than to gains or positive information. However, other psychological studies have also evaluated the different cognitive processes observed when individuals respond to negative compared to positive information. Skowronski and Carlston (1989) proposed a ‘category diagnosticity approach’ to explain negativity bias, that is, the negative information ‘category’ is more likely to be perceived as diagnostic, therefore subjects confronted with two equal but opposite cues will assign the negative cue more weight. Taylor (1991, p. 67) suggests that negative events ‘elicit more physiological, affective, cognitive, and behavioural activity and prompt more cognitive analysis than neutral or positive events.’ Studies have also found that negative events or information cause individuals to narrow and focus their attention on the features that elicited the negative events or information (Broadbent, 1971; Eysenck, 1976; Wegner & Vallacher, 1986), and this is more pronounced than for positive events and information (Peeters & Czapinski, 1990). These findings have been utilised in marketing (Anderson & Salisbury, 2003) to explain why market-level expectations are more sensitive to decreases compared to increases in perceived quality.

4

‘Framing’ relates to the fact that the wording of information can portray it in positive or negative terms.

90

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

In trying to draw predictions from this research, it seems likely that financial information showing a negative trend will be viewed as more important (Anderson & Salisbury, 2003) and weighted more heavily than financial information showing a positive trend (Taylor, 1991), thereby resulting in more cognitive processing of the financial information showing a negative trend. Consistent with this prediction, in the context of company valuations, Anderson (1988) suggested there is greater reliance on negative information and, therefore, less reliance and greater uncertainty in positive information. In relation to financial valuations, there is also the issue that although there may be greater reliance on the negative information, there may be more concern about the information because of potential questions about ongoing financial viability. Differences such as these should be reflected in the valuation process of analysts, which should be evident in specific observed ‘activities’ or ‘operations’ such as retrieving information from memory or searching documentation for relevant information. The coding processes used in verbal protocol analysis identify and code such ‘activities’ (Ericsson & Simon, 1993). See Appendix 3 for definitions of the activity codes used in this study. Prior research suggests that the following phenomena are expected to be observed from examination of the information-acquisition and processing operations reported in this study. When financial information showing a negative trend is provided, we expect:  Less information retrieval from memory, because the focus is more on the information presented.  Less generation of assumptions, because the focus is more on the information presented and there is less perceived ambiguity (perceived uncertainty).  Less generation of queries, because the focus is more on the information presented and there is less perceived uncertainty.  More ‘algebraic calculations’, because the focus is more on the financial information presented which is conducive to algebraic and other types of arithmetic calculations. When financial information showing a positive trend is provided, it is expected that the reverse of all the above will occur. These issues are addressed by considering the following research question: RQ3: In performing company valuations, do analysts under conditions of financial information showing a negative trend make more use of information-acquisition and processing operators that reflect more certainty, and draw more from financial information provided than under conditions of financial information showing a positive trend? 3. Research method A verbal protocol study with eight financial analysts as subjects was conducted to address the three research questions previously discussed. 3.1. Verbal protocol research methodology Payne, Braunstein, and Carroll (1978) state that protocol analysis is particularly useful in exploratory studies and for evaluating complex tasks. These conditions are relevant to the present study, as there is little research into how non-financial information affects the complex task of performing company valuations. Further, Ericsson and Simon (1993) note that, despite the small number of subjects associated with the technique, protocol analysis provides otherwise unobservable evidence of the sequence and type of information considered and informs judgements about the validity of theoretical or hypothesised models of behaviour. Under these circumstances, ‘verbal reports serve as evidence for the existence of certain cognitive structures and processes’ (Ericsson & Simon, 1993, p. 216). Experiments have the advantage of permitting the systematic examination of the impact of disclosures that are not widely available in practice. However, Biggs and Mock (1983) note that verbal protocols provide relatively more complete data on a participant’s behaviour that is rated high in ‘temporal density’ and can therefore suggest answers to some questions that would be difficult to obtain using traditional experimental or archival methods. Temporal density of decision data relates to how much of the decision-making thought process is captured. Concurrent verbal protocol data collection is high in temporal density, but still only partially captures the thought processes of decision makers (Newell & Simon, 1972). The authoritative text by Ericsson and Simon (1993) was used to determine the appropriate procedures for conducting this verbal protocol study. In particular, they note that protocol analysis involves ‘thinking aloud’ rather than describing or explaining thoughts (Ericsson & Simon, 1993). This is an important distinction because if participants are required to describe or explain their thoughts, it may cause them to consider other information not normally required to complete the tasks and may change their thought sequences and decision processes. The value of protocol analysis is derived from the articulated thought processes, not by an explanation of them. Klersey and Mock (1989) listed three main methodological criticisms of concurrent verbal protocols. First, verbalisations may change subjects’ performances and, therefore, their cognitive processes. Second, the data may be incomplete because a considerable portion of the information used by the participant in decision-making is not verbalised, which is more likely if the participant is an expert. Third, subjects may report parallel processes that are independent of the actual thought processes. However, Ericsson and Simon (1993) do not believe that instructions to ‘talk or think aloud’ alter the sequence of cognitive processes. To achieve the goal of capturing decision makers’ thought processes, Ericsson and Simon (1993) suggest that when using concurrent verbal protocols, interactions with subjects should be minimised and that they should be

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

91

Table 1 Verbal protocol participants and phrases and valuation judgements. Participant

A

B

C

D

E

F

G

H

Avg.

Years of experience as financial analyst Ageb Highest tertiary qualification Coded phrases of protocold (number) Share price ($) Confidencee

1

2

41

10

12

6

3

3

5.3a

35–44 Grad Dip

25–34 MA

55þ BEc

35–44 BCom

45–54 None

25–34 BEc

25–34 BCom

25–34 BCom

39c n/a

383

212

210

193

153

264

73

286

222

1.18 3

1.65 5

1.00 5

1.55 3

0.80 10

0.82 8

0.82 7

0.73 6

1.07 5.9

a

This figure excludes C. When C is included the mean increases to 9.8. The participants were asked to record their age by noting it in a range. c Approximate mean age. d A–D were provided with financial information showing a positive trend and E–H were provided with financial information showing a negative trend. e This was the participant’s self-assessed confidence in their share price assessment measured on an eleven point scale, with ‘0’ being ‘not confident’ and 10 being ‘very confident’. b

provided with a practice problem. To minimise these risks to internal validity, in the present study concurrent verbal protocols were used and participants were given a practice problem before commencing the actual study. Once the current study commenced, the only interaction with participants was via the following instruction, which was given only if a participant was silent for more than 15–20 s:  ‘What are you thinking about now?’ To satisfy Ericsson and Simon’s second suggestion, the following practice problem was given:  Try to multiply these two numbers in your head and think aloud as you get the answer: ‘What is the result of multiplying 24  36?’ A numeric calculation is suggested in Ericsson and Simon (1993) as a good method to familiarise participants with the required task. Following the practice exercise, the decision-task protocols were started and tape-recorded. 3.2. Participants and task Ten financial analysts from six stockbroking firms located in Sydney and Perth Australia agreed to participate in the verbal protocol study. However, the final number of participants analysed was eight due to one participant withdrawing because he could not do the task while another participant’s transcript contained many major periods of silence and thus was not coded. The number of participants is similar to other accounting and auditing protocol studies, for example, 4 in Biggs and Mock (1983), 7 in Anderson (1988), 4 in Biggs, Mock, and Watkins (1988), 3 in Pentland (1993), 4 in Anderson and Potter (1998), 12 in Bierstaker, Bedard, and Biggs (1999), and 8 in Mock, Wright, Srivastava, and Lu (2008). Klersey and Mock (1989) address the issue of small sample size and note that it is not unusual for three or four subjects to be used in this type of research. Other disciplines, including psychology and education, also confirm small sample sizes as the norm, for example, 12 in Robie, Brown, and Beaty (2007) and 6 in Crisp (2008). Details of participants are provided in Table 1. All the participants had sufficient qualifications and experience to complete the task. In relation to their highest educational qualifications, five had economics- or commerce-related undergraduate degrees, one had a master’s degree and one had a graduate diploma. One participant did not have a degree but had twelve years experience as a financial analyst. All of the participants had work experience as financial analysts and the few with low levels of financial analyst experience had previously worked in a related field, usually in chartered accountancy. The participants’ task was to value a medium-sized private retail company intending to list on the Australian Stock Exchange. To gain more insights into decision-making processes and the research questions, the trends of the financial information and the non-financial performance indicators, and assurance of the non-financial performance indicators were varied in the case materials. Financial performance was either showing a positive or negative trend over three years.5

5 The non-financial performance indicators were in the form of a BSC that was showing performance either better or worse than target and a BSC assurance report was either provided or not provided on this information. These variations in the case materials did not significantly affect the participants’ evaluation of the non-financial performance indicators as it was observed that their evaluations were by intrinsically evaluating the actual figures rather than performing a comparative process against the target figures.

92

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

The case was based on a real company in the retail industry. The information provided consisted of background company information, non-financial performance indicators and financial statements. 3.3. Case study development The company background information was not manipulated and provided general information about the company including: product range; operating history; number of stores; sourcing of products; and general market strategy. The company was described as one that “sells mid- to upper-market casual apparel and accessories for outdoor activities through its chain of retail outlets across Australia”. Industry information was also provided, including: level of competition; total sales; and different types of retail formats. This information was developed by reviewing the information provided by the real company supplemented by that for several other comparable companies in the retail industry. In addition, the average and range of price/earnings (P/E) ratios for the industry were provided. The non-financial performance indicators were disclosed in the form of a BSC,6 outlining performance measures that were related to the strategy employed by the company. These measures did not come from the real company or from comparable companies because they were not provided in their annual reports. Instead, they were based on BSC information used for similar types of companies in the prior literature (Kaplan & Norton, 1996; Lipe & Salterio, 2000).7 Although BSCs typically include some financial information, this present study focused only on the provision of non-financial information as part of the BSC framework. In the case study, the BSC was introduced and briefly explained, followed by a discussion of the company’s strategy. The strategy was worded to ensure that it linked clearly to the specific non-financial performance indicators provided and was described as “the Company’s strategy is to produce high-quality products and always put its customers first.” The BSC measures as presented to participants are shown in Appendix 1 and it can be seen from the quality and customer measures that the company’s strategy clearly links to these measures. This linkage is important because a key feature of the BSC approach to performance measurement is in ensuring that measures are clearly linked to firms’ strategic objectives. The BSC was chosen as the structure to represent non-financial information because it is widely used as an internal managementtool and because it relates strategy to non-financial performance. In explaining the value of the BSC, Kaplan and Norton (1996) argue that many traditional financial-reporting measures are generally lagging indicators of financial performance, that is, they tell the story of events already completed. However, nonfinancial performance indicators are more often leading indicators of financial performance, revealing the value drivers of long-term financial performance. The BSC was based on the case studies of RadWear and Workwear developed by Lipe and Salterio (2000). Their cases were developed following Kaplan and Norton’s (1996) Kenyon Stores example. The Lipe and Salterio (2000) study evaluated differences in perceptions between common and unique measures within the BSC framework. Most of the scorecard measures in the present study were developed from the ‘common’ measures used in the Lipe and Salterio (2000) case studies, which were based on medium-size retail operations, similar to the type of company used in the present study. An example of the BSC disclosed to analysts is provided in Appendix 1. Not providing this non-financial information in financial reports is the norm rather than the exception. As noted, part of the reason why the BSC is evaluated in this study is because it is a type of non-financial information that has been suggested for further disclosure (EBRC, 2005; ICAEW, 2003). The latest income statement, balance sheet and cash-flow statement were provided to participants together with two years’ comparative information. Two versions of the financial statements were developed, one showing a positive trend and the other showing a negative trend, with participants receiving one of the versions. The positive trend case showed operating profit after income tax increasing by 20 percent a year; in the negative trend case it decreased by 20 percent a year. 3.4. Development of the protocols for coding and coding schemes The audio-taped protocols were first professionally transcribed. After transcription, the written documents were validated by listening to the tapes, comparing and amending if necessary. This was to ensure that the protocols had been correctly transcribed and also to assist in identifying pauses that indicated different protocol phrases. When this procedure was completed the transcribed protocols were further reviewed and parsed into discrete protocol phrases for coding. A phrase is defined as a separate thought or information-processing activity that takes the participant from one knowledge state to another. It may be a sentence but can be shorter and can be a single word. However, a phrase is rarely longer than a sentence. An example of coding is provided in Appendix 2. To illustrate how this coding occurred, we will explain how the first three phrases were coded. Phrase 28 was straight out of the case materials so it was clearly evident that the participant

6 The BSC is a performance measurement and reporting approach that is based on integrating leading indicator non-financial measures with financial ones (Kaplan and Norton, 1992). 7 While this study presents one version of a BSC, we acknowledge that it is not meant to be representative of all BSCs because they are specifically designed for each company and come in a multitude of forms.

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

93

Table 2 Development of activity codes. Biggs and Mock (1983) operators Task structuring 1. Set goal Information acquisition 2. Information search 3. Algebraic calculation 4. Information retrieval Analytical 5. Generate assumption 6. Conjecture 7. Comparison 8. Evaluation 9. Generate query Action 10. Generate alternative 11. Decision rule 12. Sample size decision 13. Temporary sample size 14. Other decisions

The present study operators

Information acquisition 1. Read 2. Algebraic calculation 3.Information retrieval – memory 4. Information retrieval – case Information processing 5. Generate assumption 6. Comparison 7. Generate query 8. Evaluation Decision operators 9. Decision support 10. Decision rule 11. Temporary valuation 12. Final valuation

was ‘reading’, it also related to the ‘products’ section of the company information. Phrase 29 related to the participant drawing from internal memory in relation to the information read in phrase 28, so it was coded ‘information retrieval – memory’. Phrase 30 was the participant making a judgment about this information, so it was coded as an ‘evaluation’. Both phrases 29 and 30 still related to the information code of ‘products’. Research questions 1 and 2 required the coding of both activity and information-item codes. Activity codes are used to document the decision-making processes of the analysts and the information codes are used to document what information they used in performing the task. This approach is similar to Biggs (1984) who coded the types of financial information utilised separately from coding of processes the analysts applied to the information. Bouwman, Frishkoff, and Frishkoff (1987) also studied analysts’ search behaviour and coded each phrase in three ways: an activity code, an information-item code, and a report code. Appendix 3 provides definitions of the activity codes used in this study. While the activity codes were double-coded, the information codes were not because they were quite objective categories. The coding scheme for the activity codes is based on Biggs and Mock (1983). In turn, their scheme was based on Newell and Simon’s (1972) theory of human processing, which indicates that “participants’ behaviour will be composed of a sequence of operations (processes) applied to states of knowledge (information)” (Biggs & Mock, 1983, p. 238). Changes to the operators used by Biggs and Mock were made due to the different nature of the task and the different research questions being addressed; and were based on the first partial coding of Participant B performed by the two coders (discussed in Section 3.5). A comparison of the two coding schemes is presented in Table 2. The fact that this case study was a short, clearly-defined task compared to the Biggs and Mock (1983) study meant that the ‘information search’ operator was not required and was replaced by the ‘read’ operator which is an operator used in many protocol studies. The ‘information retrieval’ operator was split into information retrieved from the case and information retrieved from internal memory. This refinement was to more specifically address Research Question 3. As noted, independent test coding was done by two coders on part of the protocol for Participant B and was used as a basis for finalising the appropriate operators as well as the operator definitions. For example, the research questions being addressed encompass no need for the use of the ‘set goal’ operator. The test coding also resulted in a decision to use the ‘assumption’ operator only rather than both ‘assumption’ and ‘conjecture’ operators. This was agreed because combining them did not affect any of the research questions. The decision operator definitions used also differed from those used by Biggs and Mock (1983) because of task differences. Biggs and Mock examined auditor decision processes in relation to the evaluation of internal controls audit-scope decisions. The present study examines the performance of a company valuation which requires different decisions and thus different operators. The ‘decision support’ operator was used to code when there was a clear link between a protocol phrase and a valuation. The ‘decision rule’ operator was used to code when the participant verbalised a method in performing the valuation. As the participants typically made a series of provisional valuations before settling on a final valuation, the valuation decision was split into two operators, ‘temporary valuation’ and ‘final valuation’. Overall, these adjustments left 12 activity codes for the present study. The information codes were mainly taken straight from the information presented in the case study. This applied to codes 1–16. The only information codes that were added were when the participants created new information by calculating financial ratios and this relates to codes 17–20. These 20 information codes are shown in Appendix 4. Protocol phrases were coded with an information code when they directly used the information provided in the case study or involved the calculation of financial ratios.

94

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

3.5. The coding process Two coders independently coded each of the phrases in the protocols with operators from the activity code definitions listed in Appendix 3. The first part of the coding involved parsing the transcription into phrases in which each phrase contained an operation. For coding purposes, the parsed protocols were then transferred from a Word document onto an Excel spreadsheet. Next, the two coders discussed the activity coding scheme and the definitions of the operators to assist in developing an agreed understanding of the definitions. This process involved some amendments to the activity codes as discussed above. This was followed by each coder independently coding the first 77 phrases of the protocol for Participant B. The codes were then compared and the amount of non-chance agreement was measured using the kappa coefficient. Based on the 12 activity code categories the kappa coefficient was calculated as 0.78 for the first 77 phrases of Participant B. This indicates a high level of reliability compared to prior studies (e.g., Biggs and Mock, 1983 reported 0.66). The disagreements were then discussed and reconciled. This process assisted in clarifying the shared understanding of the definitions of the operators for the coding of the remaining phrases in the protocols. As part of this process, it was also noted that there were some disagreements about the parsing of the protocols into phrases. These disagreements were few and were not counted as coding disagreements for the purposes of calculating the kappa coefficient. However, based on discussions related to these disagreements, one of the coders reparsed the remainder of the protocol for Participant B as well as all of the other protocols. Once the protocols had been reparsed, the remainder of Participant B’s transcript and the rest of the protocols were independently coded by each coder. Kappa coefficients were then calculated on the remainder of the protocol for Participant B as well as the other protocols. The kappa coefficient across the 12 activity codes for each of the eight protocols ranged from 0.67 to 0.81, with an average overall kappa coefficient of 0.76.8 Any differences in coding of the phrases were discussed and reconciled between the two coders to obtain the final agreed coding for all of the phrases. The agreed coded protocols were then entered into NVivoÒ, a software package which, for our purposes, mainly assisted in compiling the largely qualitative data. After all of the data were entered into NVivoÒ, the information codes were assigned to each of the protocol phrases.9 The compilations from this package are shown in Tables 3 and 4. The number of coded phrases of protocols, as well as share price decisions and confidence in their decisions are presented in Table 1. As can be seen, the number of coded phrases demonstrates that the task required a reasonable degree of thought and effort by the analysts. The variation in recommended share price and confidence can be explained by the manipulation of the financial information. Participants A to D, who examined financial statements showing a positive trend, specified a higher share price (mean $1.35), but a lower confidence in their assessment (mean 4.0 on an eleven point scale, with 0 being ‘not confident’ and 10 being ‘very confident’). In contrast, participants E–H, who examined financial statements showing a negative trend, assessed a lower share price (mean $0.79), but a higher confidence (mean 7.8) in their assessments. The higher confidence is consistent with our expectation that information with a negative trend will be viewed as more reliable. The next section discusses how these results inform the research questions. 4. Results Research Question 1 relates to whether analysts utilise non-financial performance indicators in performing company valuations. Table 3 shows the number and percent of the information-item codes for the positive and negative conditions as well as the overall totals. The attention paid to information, as measured by the number of phrases, is considered to represent the ‘weight’ placed on the information in other protocol studies (e.g., Anderson, 1988). The financial statements including the audit report were the sources most utilised, accounting for 46.4 percent of all protocol phrases, followed by non-financial information (28.3 percent), then company information (19.1 percent). The percentage of phrases relating to non-financial performance indicators varied from 23.2 percent to 41.9 percent for the participants who received the positive condition, with an average of 33.6 percent, and from 18.4 percent to 32.8 percent for the participants who received the negative condition, with an average of 22.2 percent. As well as these measures of information utilisation, there were a number of specific verbalisations concerning nonfinancial information made in undertaking the valuation that indicate what information was used. Some of these phrases are presented below. First, were comments that highlighted the importance placed on this information while undertaking the valuation task. Participant C After reviewing general information on the BSC (the numbers indicate the line number within each protocol): 35: obviously they have looked at themselves from a number of different angles, 36: which is encouraging but viewing themselves for a public listing no doubt.

8 The kappa coefficient is a measure of the level of non-chance agreement between raters. Landis and Koch (1977) provided some guidance on the magnitude of agreement for the kappa coefficient. They said that between 0.61 and 0.80 provided substantial agreement, which would indicate that the overall average of 0.76 for this study is fine. 9 NVivoÒ is a software package for analysing qualitative data, such as from interviews. Coding verbal protocol data using this software package facilitates the investigation of the data. It allows the researcher to search for patterns of coding or particular words, the option of filtering data for further analysis, the ability to generate statistics on coding, and the ability to generate many other different types of tailored reports.

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

95

Table 3 Summary of participants’ protocol phrases coded to information-item codes across positive and negative financial conditions.a Positive No.

Negative %

No.

Total %

No.

%

Company information Company information – general Products Brand Name Retail Industry Total Price earnings ratio

47 32 36 21 136 55

6.1 4.2 4.7 2.7 17.7 7.2

67 27 14 32 140 34

9.9 4.0 2.1 4.7 20.7 5.1

114 59 50 53 276 89

7.9 4.1 3.4 3.7 19.1 6.2

Non-financial information Balanced scorecard Strategy Customer-related Internal business processes Learning and growth Total

67 58 51 35 47 258

8.7 7.6 6.6 4.6 6.1 33.6

41 26 46 19 18 150

6.1 3.9 6.8 2.8 2.6 22.2

108 84 97 54 65 408

7.5 5.8 6.7 3.8 4.5 28.3

93 34 86 5 43

12.1 4.4 11.2 0.7 5.6

126 15 70 7 44

18.7 2.2 10.4 1.0 6.5

219 49 156 12 87

15.2 3.4 10.8 0.8 6.0

Ratios Debt to equity Current ratio Gross margin Other Total Audit report

10 3 8 6 288 30

1.3 0.4 1.1 0.8 37.6 3.9

26 5 27 8 328 23

3.9 0.7 4.0 1.2 48.6 3.4

36 8 35 14 616 53

2.5 0.6 2.4 1.0 42.7 3.7

Total

767

Financial information Income statement Earnings per share Balance sheet Share capital Cash-flow statement

100

675

100

1442b

100

a

This table shows protocol phrases coded to the information items in the case study for the four participants who received financial information showing a positive trend, the four participants who received financial information showing a negative trend, and the overall information items totals. b This figure represents the phrases of protocol that were directly related to the information categories presented in the case study and reflected in the categories in this table. It therefore does not equal the total phrases of protocol reported in Table 4.

After going through BSC indicators: 77: well I understand those things, 78:pretty good to have that information. Participant D 69: Comment about a balanced scorecard approach is useful, 70: because that gives some sort of sense of what is driving the assessment of the company internally and what they think are important, 71: and linking the strategy to performance is also quite important. 72: In many industries or companies it’s very hard to get a sense of the method in the madness used in running the companies, 73: but companies that are able to show some sort of tool set to be able to do that show some sort of a robustness in their reporting systems, management systems, etc., 74: so that is quite valuable. After going through BSC indicators: 97:so that is actually very good information. 98: It’s not something that you often see. 99: Such companies that are willing to put things like non-financial performance indicators, quantify those, discuss the differences between target and actual its quite useful stuff, 100: and not many companies do it, 102: when you do see companies provide that sort of stuff it does give you a better sense and better feel, 103: and probably a higher level of confidence in your assessment of the company. These phrases illustrate that non-financial information provides a different and valuable perspective to analysts in evaluating companies. This extra information seemed to generate more confidence in the overall valuation process as well. Second, there was also evidence of specific re-evaluation of the non-financial performance indicators when making the final valuation decision by three of the analysts.

96

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

Table 4 Summary of participants’ protocol phrases coded to activity codes across positive and negative financial conditions.a Positive No.

Negative %

No.

Total %

No.

%

Information-acquisition operators Read Algebraic calculation Info. retrieval – memory Info. retrieval – case

200 55 123 35

20.0 5.5 12.3 3.5

185 73 40 42

23.9 9.4 5.2 5.4

385 128 163 77

21.7 7.2 9.2 4.3

Information-processing operators Generate an assumption Comparison Generate query Evaluation

63 59 130 256

6.3 5.9 13.0 25.8

50 80 89 150

6.4 10.3 11.5 19.3

113 139 219 406

6.4 7.8 12.3 22.9

21 34 18 4

2.1 3.4 1.8 0.4

34 12 17 4

4.4 1.5 2.2 0.5

55 46 35 8

3.1 2.6 2.0 0.5

Decision operators Decision support Decision rule Temporary valuation Final valuation Total

998

100

776

100

1774

100

a

This table shows protocol phrases coded to the activity codes in the case study for the four participants who received financial information showing a positive trend, the four participants who received the financial information showing a negative trend and the total activity codes.

Participant A Before finalising valuation. 314: I’ll just read through these again and see what this can tell me to help to get a feel for how the company is travelling. Participant A then re-evaluated all of the non-financial performance indicators before making his final valuation assessment. Participant B 182: My feeling is that this company would probably command something of a premium to the average in the casual apparel industry. 183: Reasons for that: 184: It would appear to have a high quality and stickier client base. 185: It has a long history in the business. 194: But the company would command a premium because of the quality of its client base. 195: The non-financial indicators appear to stack up well. Participant E In the final valuation process he briefly went back to the non-financial performance indicators and examined the following. 137: Non-financial performance indicators. 138: Repeat sales. 139: Returns by customer. An important point to note from these phrases is the timing of when they were made as part of the valuation task. The analysts had previously considered this information in their decision-making processes but made conscious decisions to reappraise the information when finalising their valuation. In terms of what parts of the non-financial information were used by analysts, the largest number of phrases related to the introduction of the detailed non-financial performance indicators stating that the company was using the BSC approach and that it was linked to the company’s strategy. The relative attention paid to the specific non-financial performance indicators was highest for the category of ‘customer-related’, followed by ‘learning and growth’ and then ‘internal business processes’. The income statement was the financial statement most used with 18.6 percent of all phrases of the protocols. This is consistent with previous verbal protocol studies that have examined this issue (e.g., Biggs, 1984; Bouwman et al., 1987, 1995; Anderson, 1988). The next most important was the balance sheet with 11.6 percent; followed by the cash-flow statement with 6.0 percent of the phrases. The calculation of ratios from the financial statements accounted for 6.5 percent of the phrases. The industry average P/E ratio was reported separately and accounted for 6.2 percent of all phrases of the protocols. The EPS figure, which is part of financial statements, accounted for 3.4 percent of all protocol phrases. All analysts used the EPS and the P/E ratio to perform their valuation by using the capitalisation of future maintainable profits method, which explains the high usage of these figures. In summary, while the protocols showed that analysts use the financial statements in similar ways as documented in prior studies, there is also evidence of a substantial attention to non-financial performance indicators in performing company valuations. This is evidenced by the high level of overall phrases relating to this information (28.3 percent), the specific

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

97

comments about the usefulness of this information, and the fact that several analysts re-evaluated the non-financial information before making their final valuation decisions. Research Question 2 considers whether analysts pay greater attention to non-financial performance indicators and use relatively less financial statement information when financial information shows a positive trend than analysts under conditions of financial information that shows a negative trend. As can be seen in Table 3, the proportion of non-financial performance indicators used by participants in the group with financial information showing a positive trend was 33.6 percent compared to 22.2 percent in the group with financial information showing a negative trend. Further, of the five participants who performed further processing of the non-financial information, these included all four of the participants in the positive financial information condition. This is a fairly consistent finding about the importance of non-financial information when financial information is positive. Finally, this differentiation in information used is also illustrated in the proportion of financial information used by participants in the group with financial information showing a negative trend, where it was 48.6 percent (range of 40.5–51.8 percent) compared to 37.6 percent (range of 31.5–41 percent) in the group with financial information showing a positive trend. These findings are consistent with the theory dealing with the salience of negative information. They show that positive financial information receives less attention, which could be due to greater scepticism concerning this type of disclosure. This is also illustrated by low levels of confidence in the valuations made for the positive trend cases (4.0) compared to negative trend cases (7.8) as measured on a Likert scale anchored by ‘0’ not confident and ‘10’ very confident. Research Question 3 relates to whether financial information showing a negative trend elicits differing informationacquisition and processing behaviours, specifically whether this condition results in greater utilisation of informationacquisition and processing operators that reflect more certainty and draw more from the financial information contained in the case, than under conditions of financial information showing a positive trend. In elaborating on Research Question 3, specific sub-questions were developed for the relevant activity codes as discussed previously. Table 4 shows the number and percent of the activity codes for the positive and negative conditions as well as for the overall totals. In the group with financial information showing a negative trend, 9.4 percent (range of 5.3–13.7 percent) of the phrases were algebraic calculations compared to 5.5 percent (range of 3.6–9.5 percent) for the group with financial information showing a positive trend. The differences were due to more calculations of the debt/equity and gross margin ratios in the negative trend group. This was also reflected in relatively more comparisons in this group (10.3 percent, with a range of 5.5– 11.4 percent) compared to the positive trend group (5.9 percent, with a range of 3.1–7.1 percent). The information retrieval from memory activity code was coded to 5.2 percent (range of 2.7–6.5 percent) of the phrases for the negative trend group, compared to 12.3 percent (range of 8.6–20.8 percent) of the phrases for the positive trend group. These results are consistent with two of the four sub-questions developed to address Research Question 3. Both of the subquestions where differences in protocols were observed relate to the information-acquisition operators (algebraic calculations and information retrieval from memory). The two sub-questions for which no substantial differences were observed relate to the information-processing operators (generate assumption and generate query). In summary, the protocols show important differences in the level of information acquisition given financial information with a negative trend. However, they do not show significant differences in processing the information related to the trend of the financial information. 5. Conclusions The main objective of this study is to undertake a detailed, descriptive analysis into whether non-financial performance indicators affect financial analysts’ decision processes and the information they utilise in performing company valuations. The examination of how professional financial report users utilise non-financial disclosures is relevant to managers in companies considering enhanced disclosure policies, regulators considering mandating disclosure of this type of information, as well as to professional and non-professional financial report users in evaluating companies. In addition, this study examines whether the direction of financial information as showing a positive or negative trend affects the level of attention participants pay to non-financial information. This issue has not been examined previously in the academic literature. The research questions were addressed by examining detailed process-tracing evidence obtained from verbal protocols. The first research question examined the extent to which analysts utilise non-financial performance indicators in performing company valuations. The overall average level of attention to the non-financial performance indicators was 28.3 percent of the information-processing activity. In addition, there were a number of specific verbalisations highlighting the importance of this information as well as evidence of specific re-evaluations using non-financial performance indicators in finalising valuation decisions. This finding is consistent with the associations that have been observed in the archival literature (Amir & Lev, 1996; Ittner & Larcker, 1998; Banker et al. 2000). The second research question explored whether financial information showing a positive compared to a negative trend results in differential use of non-financial performance indicators. The results show that when financial information showing a positive trend is provided, more attention is paid to non-financial performance indicators. Where financial information showing a negative trend is provided, more attention is paid to financial statements. This result implies that the perceived value to the provision of this type of enhanced disclosure is context-specific. The third research question related to differences due to information acquisition and processing when analysts were provided with financial information showing a negative compared to positive trend. It was found that more algebraic

98

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

calculations were performed and less information was retrieved from memory when the financial information was showing a negative trend, both of these were information-acquisition operators. This result implies that there was more use of the financial information when it was showing a negative trend compared to a positive trend. The findings from the second and third research questions suggest that analysts pay more attention to financial statement information when it is showing a negative trend. Whether this is due to their perception of it as more reliable or due to more concern about the nature of this negative-trending information is difficult to discern. When financial information showed a positive trend, participants evidently changed their processing to focus on other sources, such as their own experience and knowledge, as well as the non-financial information. These behaviours are consistent with prior research into how individuals react to negative information (e.g., Kahneman & Tversky, 1984; Peeters & Czapinski, 1990). They are also consistent with how managers expect markets to react to negative information (e.g., Chambers & Penman, 1984; Penman, 1987). How negative information actually affects the decision processes of professional financial-report users has not been previously examined. The different processing and information-utilisation behaviours by analysts, which depended on the trend of the financial information, provide an insight that is very difficult to obtain from archival or experimental studies, and is relevant in examining the value of enhanced disclosure policies of companies. Our study also observed consistency in the decision-making processes across each of the two financial performance conditions, providing some assurance about the reliability and generalisability of the findings. The limitations of this study are primarily related to the methods employed and have been well documented in the past (e.g., Payne et al., 1978; Biggs & Mock, 1983; Ericsson & Simon, 1993). The main limitation is generalisability due to the restricted ‘sample’ size. The high cost of obtaining, transcribing, coding and analysing verbal-protocol data makes obtaining larger samples problematic. However, as noted, the observed consistency in the decision-making processes across each of the two financial performance conditions provides some assurance regarding the generalisability issue. It is our view that the benefits of accessing decision-making processes by using the verbal protocol method, which are otherwise unobservable in experiments or other research methods, makes the external validity trade-off worthwhile. Acknowledgements We would like to thank Shannon Anderson, Geoff Burrows, George Klersey, Anne Lillis, Axel Schulz, Roger Simnett, Chad Stefaniak, Ken Trotman, and particularly David Woodliff for helpful comments on this paper. Thanks also to participants at the 2006 AAA Accounting Behavior and Organizations Research Conference and workshop participants at The Australian National University, The University of Melbourne, Monash University and The University of New South Wales for insightful comments on earlier versions of this paper. Finally, we thank the financial analysts who participated in this study.

Appendix 1. Non-financial performance indicators as presented to participants

Non-financial performance indicators

Target

Actual

Percentage difference

30% 8% 85%

34% 7% 90%

13.3 12.5 5.9

5% 16% 60%

4.5% 14% 63%

10 12.5 5

2.5 15 3.3

2.8 17 3.5

Customer-relateda

1. Repeat sales 2. Returns by customers (% of sales) 3. Customer satisfaction rating Internal business processes

1. Returns to suppliers 2. Average markdowns 3. Sales from Trailblazers’ brand Learning and growth

1. Average tenure of sales personnel (years) 2. Hours of employee training per employee 3. Employee suggestions per employee a

12 13.3 6.1

Customer-related measures are calculated as follows:

1. Repeat sales and 3. Customer satisfaction rating The customer surveys are given to customers whenever they buy a product. They are induced into filling out the survey by a prize of $500 of products drawn each month. They may fill it out in the store or return it in a reply paid envelope. These surveys measure a number of aspects of the shopping experience including repeat sales and the overall customer satisfaction rating. The customer satisfaction rating is measured on a five-point scale. The above figure of 90% represents the number of customers who indicated 4 (good) or 5 (very good) on that scale. 2. Returns by customers (% of sales) This measure is taken from the sales records.

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

99

Appendix 2. Protocol coding example. Extract from Participant B Verbatim protocol Sixty-five percent of its products are sourced overseas, predominantly from Thailand. From a commercial point of view there are a few issues there potentially with foreign exchange. Okay, one of the most valuable assets is its brand name. That is something you can’t get from purely looking at the tangible assets so we do have an intangible asset in the brand. How do you value that? Well that is something that you really need to look at in comparison with others with what you might consider as a comparable brand or a comparable power brand Phrases and coding Phrases of protocol

Activity code operator

Information code

28: Sixty-five percent of its products are sourced overseas, predominantly from Thailand. 29: From a commercial point of view there are a few issues there potentially with foreign exchange. 30: Okay 31: One of the most valuable assets is its brand name. 32: That is something you can’t get from purely looking at the tangible assets 33: So we do have an intangible asset in the brand. 34: How do you value that? 35: Well that is something that you really need to look at in comparisonwith others with what you might consider as a comparable brand or a comparable power brand.

Read

Products

Information retrieval-memory

Products

Evaluation Read Information retrieval-memory

Products Brand name Brand name

Evaluation Generate query Generate query

Brand name Brand name Brand name

Appendix 3. Activity code definitions used in verbal protocol.a

Information–acquisition operators

1. Read: assigned when participant reads a piece of information from the case materials. 2. Algebraic calculation: assigned when participant performs a mathematical calculation. 3. Information retrieval – memory: assigned when participant retrieves a previously stored piece of information from internal memory. (Use this operator when the participant clearly draws from knowledge or experience and applies it to the case.) 4. Information retrieval – case: assigned when participant retrieves a piece of information obtained earlier directly from the case study. Information-processing operators

5. 6. 7. 8.

Generate an assumption: assigned when participant generates a premise or assumption. Comparison: assigned when participant makes a judgment based on a comparative process. Generate query: assigned when participant raised a question about the task or case. Evaluation: assigned when participant makes a teleological judgment about the task based on a criterion (explicit or implicit).

Decision operators

9. Decision support: assigned when participant provides a verbalisation that is directly related to their valuation decision. 10. Decision rule: assigned when participant specifies a method in performing the company valuation. 11. Temporary valuation: assigned when participant makes an interim conclusion about the company valuation. 12. Final valuation: assigned when participant makes a final conclusion about the company valuation. a

Activity codes based on Biggs and Mock (1983).

100

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

Appendix 4. Information codes used in verbal protocol.

Coding number

CATEGORIES/information codesa

1 2 3 4 5

COMPANY INFORMATION Company information – general Products Brand name Retail industry Price earnings ratio

6 7 8 9 10

NON-FINANCIAL INFORMATION Balanced scorecard Strategy Customer-related Internal business processes Learning and growth

11 12 13 14 15 16

FINANCIAL INFORMATION Statement of financial performance Earnings per share Statement of financial position Share capital Statement of cash flows Audit report

17 18 19 20

RATIOS [CALCULATED BY PARTICIPANTS] Debt to equity Current ratio Gross margin Other

a These codes were based on the information as presented in the case study (with the exception of codes 17–20, which were calculated by participants from the information contained in the case).

References American Accounting Association Financial Accounting Standards Committee. (2002). Recommendations on disclosure of non-financial performance measures. Accounting Horizons, 16(4), 353–362. Amir, E., & Lev, B. (1996). Value-relevance of non-financial information: the wireless communications industry. Journal of Accounting and Economics, 22(1/3), 3–30. Anderson, E. W., & Salisbury, L. C. (2003). The formation of market-level expectations and its covariates. Journal of Consumer Research, 30(1), 115–124. Anderson, M. J. (1988). A comparative analysis of information search and evaluation behavior of professional and non-professional financial analysts. Accounting, Organizations and Society, 13(5), 431–446. Anderson, M. J., & Potter, G. S. (1998). On the use of regression and verbal protocol analysis in modeling analysts’ behavior in an unstructured task environment: a methodological note. Accounting, Organizations and Society, 23(5/6), 435–450. Banker, R. D., Potter, G., & Srinivasan, D. (2000). An empirical investigation of an incentive plan that includes non-financial performance measures. The Accounting Review, 75(1), 65–92. Bierstaker, J. L., Bedard, J. C., & Biggs, S. F. (1999). The role of problem representation shifts on auditor decision processes in analytical procedures. Auditing: A Journal of Practice & Theory, 18(1), 18–36. Biggs, S. F. (1984). Financial analysts’ information search in the assessment of corporate earning power. Accounting, Organizations and Society, 9(3/4), 313–323. Biggs, S. F., & Mock, T. J. (1983). An investigation of auditor decision processes in the evaluation of internal controls and audit scope decisions. Journal of Accounting Research, 21(1), 234–255. Biggs, S. F., Mock, T. J., & Watkins, P. R. (1988). Auditor’s use of analytical review in audit program design. The Accounting Review, 63(1), 148–161. Bouwman, M. J., Frishkoff, P. A., & Frishkoff, P. (1987). How do financial analysts make decisions? A process model of the investment screening decision. Accounting, Organizations and Society, 12(1), 1–29. Bouwman, M. J., Frishkoff, P. A., & Frishkoff, P. (1995). The relevance of GAAP-based information: a case study exploring some uses and limitations. Accounting Horizons, 9(4), 22–47. Broadbent, D. E. (1971). Decision and stress. San Diego: Academic Press. Brown, K. C., & Harlow, W. V. (1988). Market overreaction: magnitude and intensity. Journal of Portfolio Management, 14(2), 6–14. Chambers, A. E., & Penman, S. H. (1984). Timeliness of reporting and the stock price reaction to earnings announcements. Journal of Accounting Research, 22(1), 21–48. Crisp, V. (2008). The validity of using verbal protocol analysis to investigate the processes involved in examination marking. Research in Education, 79(1), 1–12. Enhanced Business Reporting Consortium (EBRC). (2005). The enhanced business reporting framework. Enhanced Business Reporting Consortium. Available from: http://www.ebr360.org/downloads/ebr.framework.publicexposure.2005.10.pdf. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis. Cambridge, Massachusetts: The MIT Press. Eysenck, M. W. (1976). Arousal, learning and memory. Psychological Bulletin, 83(3), 389–404. Hutton, A. P., Miller, G. S., & Skinner, D. J. (2003). The role of supplementary statements with management earnings forecasts. Journal of Accounting Research, 41(5), 867–890. Institute of Chartered Accountants in England & Wales (ICAEW). (2003). Information for better markets – new reporting models for business. London: Institute of Chartered Accountants in England & Wales. Ittner, C. D., & Larcker, D. F. (1998). Are non-financial measures leading indicators of financial performance? An analysis of customer satisfaction. Journal of Accounting Research, 36(Suppl.), 1–35.

P.J. Coram et al. / The British Accounting Review 43 (2011) 87–101

101

Ittner, C. D., Larcker, D. F., & Randall, T. (2003). Performance implications of strategic performance measurement in financial services firms. Accounting, Organizations and Society, 28(7/8), 715–741. Kahneman, D., & Tversky, A. (1979). Prospect theory: an analysis of decision under risk. Econometrica, 47(2), 263–291. Kahneman, D., & Tversky, A. (1984). Choices, values and frames. American Psychologist, 39(4), 341–350. Kaplan, R. S., & Norton, D. P. (1992). The balanced scorecard – measures that drive performance. Harvard Business Review, 70(5), 71–79. Kaplan, R. S., & Norton, D. P. (1996). The balanced scorecard: Translating strategy into action. Boston, Massachusetts: Harvard Business Press. Klersey, G. F., & Mock, T. J. (1989). Verbal protocol research in auditing. Accounting, Organizations and Society, 14(1/2), 133–151. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. Lipe, M. G., & Salterio, S. E. (2000). The balanced scorecard: judgmental effects of common and unique performance measures. The Accounting Review, 75(3), 283–298. Mock, T. J., Wright, A., Srivastava, R. P., & Lu, H. (2008). The framing and evaluation of multiple hypotheses. Asia-Pacific Journal of Accounting & Economics, 15, 123–140. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, New Jersey: Prentice-Hall. Payne, J. W., Braunstein, M. L., & Carroll, J. S. (1978). Exploring pre-decisional behavior: an alternative approach to decision research. Organizational Behavior and Human Performance, 22, 17–44. Peeters, G., & Czapinski, J. (1990). Positive-negative asymmetry in evaluations: the distinction between affective and informational negativity effects. European Review of Social Psychology, 1(1), 33–60. Penman, S. H. (1987). The distribution of earnings news over time and seasonalities in aggregate stock returns. Journal of Financial Economics, 18(2), 199–229. Pentland, B. T. (1993). Getting comfortable with the numbers: auditing and the micro-production of macro-order. Accounting, Organizations and Society, 18(7/8), 605–620. Robie, C., Brown, D. J., & Beaty, J. C. (2007). Do people fake on personality inventories? A verbal protocol analysis. Journal of Business and Psychology, 21(4), 489–509. Skowronski, J. J., & Carlston, D. E. (1989). Negativity and extremity biases in Impression formation: a review of explanations. Psychological Bulletin, 105(1), 131–142. Taylor, S. E. (1991). Asymmetrical effects of positive and negative events: the mobilization-minimization hypothesis. Psychological Bulletin, 110(1), 67–85. Wegner, D. M., & Vallacher, R. R. (1986). Action identification. In R. M. Sorrentino, & E. T. Higgins (Eds.), Handbook of motivation and cognition: Foundations of social behavior. New York: Guilford Press.