Chemico: Evaluating performance based on the Balanced Scorecard

J. of Acc. Ed. 28 (2010) 221–236

Contents lists available at ScienceDirect

J. of Acc. Ed. journal homepage: www.elsevier.com/locate/jaccedu

Educational case

Chemico: Evaluating performance based on the Balanced Scorecard Monte Wynder ⇑ University of the Sunshine Coast Maroochydore DC, Qld 4556, Australia

a r t i c l e

i n f o

Article history: Available online 23 April 2011 Keywords: Balanced Scorecard Strategy map Performance evaluation Sustainability Environmental performance measurement

a b s t r a c t For the past decade Chemico Inc. has been pursuing a successful strategy producing an innovative type of plastic that is replacing steel in the production of cars. Chemico has plants around the world, and demand continues to grow. Operations have recently commenced at three new plants in China. You have obtained performance data for the three plants in the form of a Balanced Scorecard (BSC) in which each manager’s performance is compared against the company’s targets. As the Financial Controller for Chemico it is time for you to evaluate the three plant managers and distribute the bonus pool between them. Ó 2011 Elsevier Ltd. All rights reserved.

1. Introduction Chemico Inc. produces an innovative type of plastic that is replacing steel in the production of cars. The industry is very competitive and prices continue to fall as the production process is improved. Chemico has a very successful strategy focused on continually reducing production costs and the effective utilization of its expensive plant and equipment. Chemico seeks to capture market share by providing the cheapest product on the market; however, quality is also important. The production process involves thousands of variables that must be monitored and adjusted to determine the energy requirements and yield (output compared with input). Furthermore, the highly toxic waste associated with the production process must be managed carefully and must undergo expensive treatment before being released. Highly trained and experienced engineers are the key to ensuring that the process is efficient and that improvements are continually identified.

⇑ Tel.: +61 7 5430 1263; fax: +61 7 5430 1210. E-mail address: [email protected] 0748-5751/$ - see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.jaccedu.2011.03.006

222

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

In summary, Chemico’s strategy is to invest in training and workplace conditions to improve the engineers’ innovation and process management skills. This then results in more efficient operations and lower production costs. Market share then increases through low prices while maintaining a high gross margin. The production process requires huge amounts of energy, which has led Chemico to set up three new plants in China where there is an abundance of low-cost coal. The plants were established 6 months ago and it is now time to evaluate the performance of each plant. In this regard, please complete Exhibits 1 and 2.

2. Implementation guidance and teaching note 2.1. Case synopsis This case provides an engaging tool for teaching the principles of the Balanced Scorecard (BSC) with an emphasis on the relationships between lead and lag performance measures. In addition to the causal relationships between Learning and Growth, Internal Processes, Customer, and Financial measures, the impact of environmental performance can also be recognized by integrating environmental measures into the organization’s strategy map (SM). In evaluating the managers, students make judgments about the relative importance of the different performance measures thereby exposing their implicit weightings and biases. An important feature of the case is that students are evaluating the managers’ first 6 months of operations. This allows the relationships between lead and lag indicators to be clearly tracked across time. Furthermore, the subsequent debriefing of the evaluations demonstrates the power of the SM and clearly highlights the time lag between implementing the company’s strategy and achieving financial targets.

3. Overview of the task Students assume the role of Financial Controller and evaluate the performance of three managers based on measures presented in the BSC format (Exhibit 1). The task is based on a research instrument developed by Lipe and Salterio (2000). Of particular interest in their research was the finding that individuals may not be using the BSC effectively for performance evaluation. For example, subsequent research found that evaluators focus on common measures of performance rather than the measures that are specific to the organization’s strategy (for example, Dilla & Steinbart, 2005).1 Determining the implicit importance that students place on various measures within the BSC exposes the typical biases and cognitive limitations identified in this previous research. Reflecting on their experience in evaluating the three managers provides a rich basis for a discussion of the behavioral issues involved in implementing the BSC. Performance of the three managers is contrived such that the sum of all the percentages above and below target is approximately the same. Therefore, an equal weighting placed on each measure would lead to equal evaluations and rewards. Manager A’s performance, however, is consistent with an investment in the lead indicators of performance identified in the organization’s strategy, but which has not yet flowed through into satisfactory financial performance. In contrast, Manager B achieves well above target in the financial (i.e., lag) indicators of performance. Upon further analysis, however, it can be seen that this is achieved at the expense of performance that, according to the organization’s successful strategy, will determine financial performance in the future. Specifically, results indicate that Manager B has not invested in employee training and the result is beginning to be seen in employee dissatisfaction and turnover. In summary, financial performance is significantly above target, but the lead indicators of performance suggest that the organization’s strategy is not being pursued, and the superior financial performance is unlikely to continue. 1

Further discussion of the research findings is presented in Table 2.

223

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

Measure

Target for the six months

Plant A Actual

Plant B

% better (or worse) than target

Actual

Plant C

% better (or worse) than target

Actual % better (or worse) than target

Financial Perspective 1. Return on Investment

12%

10

-16.67%

15.2

2. Revenue Growth

15%

12.5

-16.67%

18

3. Net Profit Margin

13

8.33%

20.00% 15.5

26.67%

3.33%

8%

6.5

-18.75%

10

25.00%

9.5

18.75%

10%

11.5

15.00%

8

-20.00%

11

10.00%

85%

86

1.18%

79

-7.06% 88

3.53%

2. Price per kg

79

79

0.00%

72

8.86% 78

1.27%

3. Customer satisfaction rating

90

94.5

5.00%

83

-7.78% 89

-1.11%

35%

30

-14.29%

43

22.86% 37

5.71%

75%

73

-2.67%

69

-8.00%

76

1.33%

10

12

20.00%

8

-20.00%

11

10.00%

50kg per hr

53

6.00%

65

30.00%

63

26.00%

80%

77

-3.75%

90

12.50%

83

3.75%

1.Employee satisfaction survey

80

95.5

19.38%

66

-17.50%

82

2.50%

2. Employee suggestions

25

24

-4.00%

26

4.00%

26

4.00%

3. Employee turnover (engineers)

10%

7.5

25.00%

12

-20.00%

9

10.00%

4. Hours of employee training

70 hr

79.5

13.57%

55

-21.43%

66

-5.71%

1010

-1.00%

995

0.50% 1200

-20.00%

500ppm

490

2.00%

495

1.00% 550

-10.00%

50 tons

51

-2.00%

52.5

-5.00%

55

-10.00%

20 liters

18.5

7.50%

18

10.00%

26

-30.00%

4. Reduction in Production Costs Customer perspective 1. Customer retention

4. Market Share Internal process perspective 1. Quality measure 2. Number of improvements implemented 3. Employee productivity (output per employee hour) 4. Capacity utilization percentage Learning and growth perspective

Environmental perspective 1. Toxicity of air emissions 2. Toxicity of water emissions 3. Energy efficiency (tons of coal per 10,000kg of output) 4. Accidental release of untreated waste

1000ppm

Exhibit 1. Balanced scorecard for Chemico’s three plants for the first 6 months.

224

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

Evaluation of the manager of Plant A: (place an X somewhere on the scale below) Reassign: sufficient improvement unlikely

0

Very Poor: Considerably below expectation

Poor:

Average:

Good:

Somewhat below expectations

meets expectations

somewhat above expectations

2

4

6

8

Very Good: considerably above expectations

10

Excellent: far beyond expectations, manager excels

12

------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------Evaluation of the manager of Plant B: (place an X somewhere on the scale below) Reassign: sufficient improvement unlikely

0

Very Poor: Considerably below expectation

Poor:

Average:

Good:

Somewhat below expectations

meets expectations

somewhat above expectations

2

4

6

8

Very Good: considerably above expectations

10

Excellent: far beyond expectations, manager excels

12

------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------Evaluation of the manager of Plant C: (place an X somewhere on the scale below) Reassign: sufficient improvement unlikely

0

Very Poor: Considerably below expectation

Poor:

Average:

Good:

Somewhat below expectations

meets expectations

somewhat above expectations

2

4

6

8

Very Good: considerably above expectations

10

Excellent: far beyond expectations, manager excels

12

------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------The total year-end bonus is $300,000. Determine how much you would allocate to each manager: Manager A: $______________ Manager B: $______________ Manager C: $______________ (note that the total should equal, but not exceed $300,000) Explain how you arrived at your decision: Exhibit 2. Balanced Scorecard Response Sheet. As the Financial Controller of Chemico, Inc. it is your responsibility to evaluate performance and determine the distribution of the bonus pool of $300,000 across the three managers. Your decision will need to be presented at the upcoming remuneration committee meeting so it is important to explain and justify in writing how you arrived at your decision.

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

225

3.1. Incorporating sustainability The strategic importance of sustainability is now widely recognized (Molina-Azorín, Claver-Cortés, López-Gamero, & Tarí, 2009). A number of authors have advocated a separate sustainability scorecard (Perrini & Tencati, 2006; Graham, 2009; Kittiya & James, 2006). Others have argued for incorporating sustainability in the organization’s existing BSC (Figge, Hahn, Schaltegger, & Wagner, 2002), either as a separate perspective or integrated throughout the existing perspectives. There are pros and cons of these various approaches that can be explored based on the students’ experience with the format provided in this case. Manager C’s performance is positive on all four of the BSC perspectives commonly used, but significantly below target on the environmental measures. This provides the basis for a discussion of the strategic importance of environmental performance. Note that, although the environmental perspective is included in the BSC, students tend to ignore these measures when making their evaluations. This tendency can be explored and the merits of alternative approaches, such as integrating environmental measures into the other perspectives, can be considered. Furthermore, other dimensions of sustainability, such as social performance, could be incorporated either in place of, or in addition to environmental performance. In this case the organization has a purported commitment to environmental performance, but the connections to financial performance are not explicated in the strategy. This is typical of many organizations and provides a point for discussion. The extent to which students ignore environmental performance in their evaluations indicates the importance of explicitly linking environmental performance to financial performance. After these relationships are illustrated in the SM even the most economic-oriented students can see that investing in environmental performance can have important long-term financial consequences. 4. Implementing the case 4.1. Preparation (30–40 min) The case encourages students to reflect on the difficulty and resultant biases that often emerge in the implementation and use of the BSC (de Waal, 2003). Therefore, I recommend its use at the beginning of a discussion of the BSC, after students have been introduced to the mechanics of the BSC (most management accounting texts describe the BSC’s structure). After identifying the four typical perspectives, and the potential for additional perspectives, students can then be given the evaluation exercise as an illustration of a particular organization’s BSC. A copy of the BSC and the evaluation form (Exhibits 1 and 2) are provided to each student, or group of students. After a brief discussion of the organization’s strategy (it is not necessary to go into detail) the performance measures chosen to implement the strategy can be defined. Simple definitions of each measure used at Chemico are provided in Table 1. The case can also be used to illustrate the use of a BSC in service organizations and the right hand column of Table 1 provides measures for an alternative setting that is described in Section 5.2. These measures for BrightWorld were used to make the case applicable to students studying public policy. They demonstrate how the case can be adapted for service providers, or a specific industry. Although Table 1 could be provided to students, a feature of this case is parsimony (the case can be printed on two pages). Therefore I recommend verbal clarifications where necessary. A brief discussion of specific performance measures provides an opportunity to emphasize the importance of choosing measures that are specific to the organization’s strategy. This can be achieved by building the organization’s SM with students, one measure at a time (available as a Microsoft PowerPoint file from the author). Students are asked at each step to evaluate the plausibility of each arrow as a series of if-then statements. For example, the SM in Exhibit 3 (disregard plant specific performance at this stage) can be read as follows:  Chemico has found that employee training increases employee satisfaction. When employees are more satisfied they are more likely to stay and hence employee turnover is reduced. Employees

226

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

Table 1 Definitions of performance measures and alternative measures for an alternative strategy. Measures for Chemico

Definition for Chemico

Alternative measures for BrightWorld (Healthcare)

Financial perspective 1. Return on investment 2. Revenue growth 3. Net profit margin 4. Reduction in production costs

Net profit after tax divided by total assets. Percentage increase in sales revenue. Net income/net sales Percentage reduction in production costs.

Return on investment (%) Revenue growth (%) Net profit margin (%) Reduction in Operating Costs (%)

Percentage of customers making repeat purchases. Sales price Based on a survey of existing customers, score is out of 100. Sales revenue as a percentage of total industry sales in the country.

Customer retention (%) Average daily rate (in $) Customer acquisition

The product’s score on an aggregate measure of quality. Number of employee suggestions chosen to be implemented. Total output of completed product divided by total operating employee hours. Actual production level as a percentage of ideal operating capacity.

Service quality (independent rating) Employee skills (independent evaluations) Employee productivity ratio

Score (out of 100) on an employee satisfaction survey. Number of employee suggestions submitted through the formal employee suggestion scheme. Percentage of engineers that cease employment.

Employee satisfaction (survey) Relative pay (% industry average) Employee turnover (%)

Hours of training per employee (cumulative measure for the year to date).

Employee training (Hrs)

Parts per million of toxic substances in the air emissions. Parts per million of toxic substances in the wastewater. Quantity of coal used for each 10,000 kg of completed product. Quantity of waste released into the environment (cumulative measure for the year to date).

Laundry (kg/patient day)

Customer Perspective 1. Customer retention 2. Price per kg 3. Customer satisfaction rating 4. Market share Internal process perspective 1. Quality measure 2. Number of improvements implemented 3. Employee productivity (output per employee hour) 4. Capacity utilization percentage Learning and growth perspective 1.Employee satisfaction survey 2. Employee suggestions 3. Employee turnover (engineers) 4. Hours of employee training Environmental perspective 1. Toxicity of air emissions 2. Toxicity of water emissions 3. Energy efficiency (tons of coal per 10,000 kg of output) 4. Accidental release of untreated waste

Market share (%)

Occupancy rate (%)

Energy efficiency (Kw) Waste (kg) Environmental ranking (independent rating)

who have worked longer at their jobs are more likely to understand the processes and be able to make suggestions for improvements. How many of those suggestions are implemented, however, will depend on the employees’ experience (as determined by turnover) and their level of training (both determinants of job-specific knowledge). Successful implementations lead to increased employee productivity and/or capacity utilization, both of which can reduce the production costs and thereby increase the net profit margin and ROI.  Improvements may also relate to the quality of the product which will increase customer satisfaction and hence the extent to which customers continue to use Chemico as their supplier. Retaining customers is an important determinant of Chemico’s share of the total market. Market share is also determined by the price. In addition to a direct effect on market share, price also affects market share by influencing customer satisfaction, given the level of quality. Also note, however, that lower prices will have a negative effect on net profit.

227

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

Customer Satisfaction

Water emission toxicity

Customer Retention

8

1

7

Quality

1

5

23

RISK

9

20

20

Energy Efficiency

Market Share

9

Employee Suggestions

4

Employee Turnover

Release untreated waste

Air emission toxicity

8

10

Strategy Map Chemico Plant B

Revenue Growth

Price per Kg

20 18

20

Number of Improvements 20

ROI 27% 9

Employee Satisfaction

13

Hours of Employee Training

25

20

Capacity Utilisation

21

21

Employee Productivity

30

Reduction in Prod Costs

20

Net Profit Margin

TIME

Exhibit 3. Strategy map for Plant B (Different shapes are used here to represent the different perspectives of the BSC. The size of the arrow and the shading indicate the extent above or below target (with the percentage given in the arrow). Shades of color (from light to dark red or green), however, are the most effective way to illustrate the deviation from target. Microsoft Word and PowerPoint files using color are available from the author). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Note that other stories might have been told, and other links drawn between the performance measures. Class discussion may explore the possibilities but, ultimately, it is important to note that this is the organization’s preferred strategy, that it has been successful in the past, and that managers are expected to implement it. In other words, no matter what the relationships might be, these have been identified as the measures of strategy implementation and success. Environmental performance has not been integrated in the preceding story, but is seen as a determinant of risk that should be considered when evaluating ROI (i.e., the risk–return relationship). Alternatively, these environmental performance measures could be integrated through their impact on customer satisfaction and/or productivity. Further explanation of the BSC document (Exhibit 1) should include a discussion of the targets and percentages. Consistent with management by exception, targets that have not been achieved are bold. It is also worth noting that the targets for each manager are the same, that each plant has only been in operation for 6 months, and that the strategy has been effective in the other plants operated by Chemico. Attention can then be drawn to the evaluation form. After resolving any queries, students can be given time to evaluate each manager on the 11-point scale, allocate the bonus pool between the three managers, and justify their evaluations. Allocating the bonus emphasizes the consequences of the performance evaluation and also makes students explicitly rank the performance of the three managers. 4.2. Completing the evaluations (20–40 min) Students can generally complete the evaluations in 20–30 min if working alone, and 30–40 min if working in groups. Although some students would like to have more time, this is long enough to distinguish between the performance of the three managers in preparation for the subsequent discussion. The emphasis is not on making a definitive evaluation, but to recognize the difficulty in doing so. It is also important to encourage students to reflect on the processes by which they are making their evaluations, hence the importance of the justification section (Exhibit 2).

228

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

The ability to complete the task quickly makes it an ideal activity for inclusion in a 2 h lecture format (I have used it in lectures of 100 + students and seminars of 20 students). Completion of the task provides students with a break from the lecture, and an opportunity to apply what they have learned so far. 4.3. Debriefing – multi-dimension performance measurement (15–40 min depending on learning objectives) Importantly, the task exposes the difficulty in actually applying the BSC and provides the basis for a discussion of alternative approaches (i.e., subjective versus objective weightings) to making evaluations. A show of hands demonstrates the different approaches taken by students and these can be discussed in terms of the benefits and weaknesses identified in the literature. Various strategies are typical: equal weighting, a focus on financial measures, or an emphasis on particular measures (such as environmental) that the student personally values. Importantly, some students will not be able to express the basis for their evaluations. Making a correct evaluation is not the objective. Rather, learning occurs as the student’ recognizes the difficulty in dealing with the multitude of measures and the biases that they have exhibited. Ultimately, however, students should see the importance of evaluating the measures within the context of the organization’s strategy, and the importance of recognizing the drivers of future financial performance. This is illustrated by recasting the SM for each manager to show their performance on the various measures (Exhibit 3). 4.4. The strategy map Comparing separate SMs for each manager, with performance now highlighted with shades of red or green, or arrow size as shown in Exhibit 3, emphasizes the lead and lag nature of the measures. Since the plants have only been operating for 6 months, students can clearly see two very different ‘stories’ in the performance of Plants A and B. At this point it is also useful to highlight that the cause-effect relationships unfold across time. The story that is unfolding from Manager A’s SM is that an investment in the lead indicators of performance has not yet flowed through into the lagged financial measures. In contrast, Manager B is performing well on financial measures but the lead indicators suggest that this performance is not sustainable (Exhibit 3). Students can be encouraged in the debriefing to ‘tell the rest of the story’, i.e., to predict the likely impact on the lag performance measures from Manager B’s failure to invest in employee training. Some possible points in the interpretation are as follows: Manager B has very high levels of ROI, but let us consider how that has been achieved. Employee productivity is very high, but employee satisfaction is very low. Employee suggestions are above target, but the number of those suggestions actually implemented is very low indicating that the quality of those recommendations is low. Price per kg is very competitive, but quality is low – it seems that Manager B is achieving revenue growth through lowering the price per kg, but with low customer retention we can predict that market share will decrease with a diminishing pool of potential new customers. As noted, the relationship between lead and lag indicators can be illustrated by color-coding the strategy map and including an arrow along the bottom to highlight the temporal dimension of those relationships. Future performance can be predicted by looking to the left of the SM. Increasingly darker shades of green toward the left can be expected to ‘flow through’ to the right as their positive impact is felt in the lagging indicators. The converse is also true, with dark red indicators to the left being indicative of poor performance in subsequent periods. When the strategy map is presented in this way it clearly shows the differences between Managers A and B.2

2

Color SMs for each of the three managers in Microsoft Word and PowerPoint formats can be obtained from the author.

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

229

In summary, the ‘story’ is told by discussing the relationship between each performance measure that is connected by an arrow. Since managers have only been operating for 6 months this flow through from lead to lag indicators is not complete, but can be predicted. Using greenfield operations is an important feature of the case. For operations that have been going for a longer period of time the lag indicators will be affected by previous managerial decisions. At this point it is appropriate to revisit the discussion of how individual measures might be weighted. Whereas lead and lag indicators may be equally weighted for an established business, a new operation or the implementation of a new strategy may be better served by a heavier reliance on lead indicators. Manager C is performing reasonably well on all measures of performance except environmental. Therefore, the student’s evaluation of Manager C indicates the extent to which they have been influenced by the poor environmental performance. Interesting discussions arise as students who have previously asserted that environmental performance is important find themselves evaluating performance solely on the financial measures. Students can be encouraged to speculate on the potential consequences of Manager C’s poor environmental performance. This provides the basis for a discussion of the strategic benefits and risks associated with environmental performance, and how environmental measures can be integrated into the SM.

4.5. Links to theory An understanding of the theoretical background and some relevant research findings is useful to enrich the discussion of the case, particularly when used for advanced undergraduate or postgraduate courses. As students experience difficulty in evaluating the three managers they can then identify with the research findings in the area. An important feature of the BSC is that it focuses attention on a limited number of measures, i.e., 20–25, designed to encapsulate the organization’s strategy. Individuals must still, however, find ways to deal with the cognitive complexity of what remains a relatively large number of performance cues (Rich, 2007). Kaplan and Norton argue that the BSC provides a causal map of the organization’s strategy. This provides a schema by which the performance measures can be understood. Empirical research indicates, however, that most organizations do not understand the causal linkages in their own BSCs. In 2002/01 Speckbacher, Bischof, and Pfeiffer (2003) conducted a survey of the 200 most publicly traded firms in Germany, Austria and Switzerland. Their respondents were board members and heads of departments. Of the firms that used the BSC only half employed cause-effect chains. Without an understanding of the relationships between lead and lag indicators, that is, how investment in lead indicators will ultimately increase lag indicators, such as financial performance, it is difficult to see how the BSC can guide the implementation of strategy. Furthermore, despite the lack of a SM, more than two-thirds of users linked their reward system to the BSC. Linking rewards to performance measures is important in driving performance and so is an important step in transforming the BSC from a measurement system to a management system (Kaplan & Norton 1996). However, if the causal linkages are not clearly understood, rewards may magnify unsustainable and short-sighted behavior, as illustrated in the performance of Manager B. Students can be asked to speculate on how Manager B’s performance is consistent with evaluations and rewards that focus on financial performance. In this case students are told that the strategy is effective in creating financial performance. It can be noted that poor performance on financial measures may be the result of an ineffective strategy (Wong-On-Wing, Guo, Li, & Yang, 2007). Students can be reminded, however, that a manager has a responsibility to implement the organization’s strategy. Furthermore, a certain degree of faith may be required given the lagged nature of the financial measures of performance. In their quasi-experimental field study of actual BSC implementation in a banking institution, Davis and Albright (2004) found that financial measures do increase with improving lead indicators, but not initially. Indeed, in their study the financial performance of the control group (i.e., non-BSC adopters) initially rose relative to the BSC adopters for the first 6 months. Similarly, Crabtree and DeBusk (2008) find an association between lead indicators and shareholder returns. But again, in their longitudinal study shareholder returns were not significantly better until the second year after adoption.

230

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

Despite these findings that demonstrate the importance of lead indicators, Ittner, Larcker, and Randall (2003) find that superiors evaluate BSCs subjectively with a focus on financial measures. Although this subjectivity can introduce biases toward lagged indicators, (Gibbs, Merchant, Van der Stede, and Vargus, 2004; Gibbs, Merchant, Van der Stede, and Vargus, 2005) note that subjectivity may be important to deal with limitations in quantitative measures (2004; 2005). There is certainly room for debate on the advantages and problems of subjectivity in performance evaluation compared with a rigid weighting system. Commencing with a study by Lipe and Salterio (2000), a number of recent studies have considered the cognitive limitations and biases that might explain the failure to effectively use the BSC. Attention has focused on the emphasis placed on common measures of performance, rather than unique measures that are more pertinent to the organization’s specific strategy. Consistent with previous judgment and decision-making research (Slovic & MacPhillamy, 1974), the finding is that individuals rely more heavily on common measures (Lipe & Salterio, 2000). In a later paper they explain why: The Balanced Scorecard with its large number of performance measures presents a complex task to a manager asked to use the scorecard to evaluate a division’s performance. The manager could, theoretically, weight and combine the many measures into an overall evaluation of the business unit but this is, cognitively, a very difficult thing to do. Research in cognitive psychology has repeatedly shown that humans are able to retain and use only a small number of items in working memory (Baddeley, 1994; Miller, 1956). With this limit on working memory, holding 20 or more individual measures in one’s head and mentally manipulating them simultaneously is extremely difficult, if not impossible. Thus, the volume of data in a Balanced Scorecard suggests that it may overload human decision makers with information. (Lipe & Salterio, 2002, 532). Subsequent research has sought to identify ways to debias the application of the BSC (Banker, Chang, & Pizzini, 2004; Libby, Salterio, & Webb, 2004; Roberts, Albright, & Hibbets, 2004). Because this case uses a similar instrument to prior research, the student’s experience can be compared with the results of previous research (including the responses of accounting controllers provided in Section 5.3). The discussion can focus on ways to overcome the cognitive load (such as the SM), thereby reducing the biases that have been demonstrated. Connections between learning objectives, discussion questions, and research are provided in Table 2.

5. Evidence of the benefit of case usage 5.1. Learning objectives This short case provides a rich discussion for the role of the BSC in implementing an organization’s strategy. An important feature of the BSC that distinguishes it from other performance measurement systems is its emphasis on causal links (Kaplan & Norton, 1992; Kaplan & Norton, 1993; Kaplan & Norton, 2000a; Hoque & James, 2000; Speckbacher et al., 2003). It has been argued that such an approach drives performance (Iselin, Mia, & Sands, 2008) and is important in overcoming an over-reliance on financial measures of performance (DeBusk, Killough, & Brown, 2005; Kaplan & Norton, 1992). Recognizing the lagged nature of financial measures, the BSC identifies the lead indicators (Learning and Growth, Internal Processes, and the Customer Perspective) which can be linked in a set of cause-effect relationships that Kaplan and Norton (1992) call a strategy map. This case demonstrates the power of the SM as a way to understand the relationships between performance measures. By building the SM with students they see how the SM helps to overcome the cognitive limitations faced when evaluating performance based on the BSC alone. They are encouraged to see the ‘story’ told by the performance measures that is unfolding across time. Furthermore, by incorporating environmental performance measures into the SM the ultimate impact on financial performance can be seen. Color-coding the SM based on actual performance then illustrates the implementation of the organization’s strategy and allows students to make predictions about future performance. Students also see firsthand the difficulties associated with using the BSC as a performance measurement tool. Their implicit weightings on the various performance measures are exposed and the

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

231

implications considered. For example, espoused commitment to environmental performance can be compared with the performance evaluation and rewards assigned to a manager with poor environmental performance. This case can be used during the introduction to the BSC as a means of demonstrating its use in practice, highlighting that the performance measures used should be consistent with the organization’s strategy (DeBusk, Brown, & Killough, 2003). As an experiential task, the case also provides a rich basis for discussion of performance evaluation and reward, the implementation of the BSC, the development of a SM, the cognitive limitations that can undermine the use of multi-dimensional performance evaluation schemes, and the strategic importance of environmental performance. Furthermore, because the case is similar to an experimental task used in research, there is a wealth of interesting research findings that students can be referred to. A list of possible learning objectives, depending on the emphasis chosen by the instructor, is provided in Table 2. 5.2. Reflections on usage- different levels and majors The case has been used effectively for undergraduate and postgraduate students and only requires a minimal explanation of the basic principles and structure of the BSC. Various textbooks provide such an introduction. Journal articles, such as Kaplan and Norton’s Harvard Business Review papers (e.g., 1992, 1993, 2000b, 2008), also provide an excellent introduction. I have also used the case in lectures without any preassigned readings. After approximately 40 min of introduction to the BSC students are ready to see how it is applied. I have even used the case effectively with Grade 10 high school students. The depth of discussion, of course, differs as the students’ experience in completing the case can be evaluated at different levels. The first iteration of the case was used in 2008 with third-year management accounting students at a small regional university in Australia. Since then, variations have been used in introductory and capstone accounting courses in Australia, and Accounting Controlling courses for first, second, and third year students at three Universities of Applied Science in Germany. Previous management accounting studies are not necessary. First-year, non-accounting business majors, social economics majors, and accounting majors all indicated that they found the case reasonably easy to understand (mean 5.93/10 for first year students). The following is an example of an alternative strategy that is appropriate for another setting chosen to be of interest to social economics majors (based on a nursing home). Specific performance measures can then be chosen to be consistent with the strategy (see Table 1). If the performance percentages are kept the same as for Chemico, the evaluation of the managers and debriefing will be fundamentally the same:  BrightWorld Ltd. is an aged care provider that has been growing rapidly over the past 10 years based on a successful strategy that focuses on increasing market share, reducing operating costs, and carefully managing its environmental impact. This is achieved by providing patients with great service from friendly and efficient staff in an environmentally friendly way. BrightWorld’s vision statement is to be the community’s provider of choice and their mission statement is: ‘‘BrightWorld nursing homes are committed to providing the highest quality of health care through service excellence, compassionate care and careful environmental management’’ In summary, BrightWorld’s strategy is to hire and train local people to empower them to provide outstanding service. Empowered staff are more satisfied and this also contributes to a better experience for patients. Market share and revenues then increase as reputation and customer retention improve. Highly trained and experienced staff are also important in reducing operating costs as they learn to be more productive. The company is also committed to improving their environmental performance by reducing waste and energy consumption. BrightWorld recently established three new nursing homes. The nursing homes were opened 6 months ago and it is now time to evaluate their performance. The nursing homes are equivalent in all other respects and so your evaluation should be based entirely on the reported results.

232

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

Table 2 Learning objectives and associated questions for discussion. Possible learning objectives Upon completion of this case, students will Understand the importance of linking performance measures to the organization’s strategy

 Evaluate managerial performance based on a range of lead and lagged measures

 Understand the cognitive processes and potential problems involved in evaluating performance based on the BSC.

Additional questions for discussion

Relevant research finding

1. Explain how performance measures drive performance

 There is a link between measuring a dimension and performance on that dimension. Aligning strategic goals and the reporting system improves performance (Iselin et al., 2008)  Implementing the BSC leads to improved financial performance (Davis & Albright, 2004)

2. Predict the effect of including measures in the BSC that are inconsistent with the organization’s strategy 1. Should performance evaluations be based entirely on past performance, or should consideration be given to future performance? What are the issues associated with taking either approach?

2. On what basis did you evaluate the manager’s performance? How were your evaluations influenced by the organization’s strategy? 3. The BSC may be used objectively or subjectively to evaluate a manager’s performance. Explain the difference between these two approaches and the implications for the evaluator and the individual being evaluated

4. Describe the benefits of using a SM when evaluating the managers performance 5. How does color-coding performance help you to understand the relative performance of Managers A and B?

Explain the significance of lead and lag indicators of performance. Predict the effect of investments in learning and growth upon the

1. Consider Manager’s A and B at Chemico. Assuming that these two

 Performance measurement systems present a cognitively complex task and individuals often respond by emphasizing historical financial measures compared with nonfinancial measures. Evaluators also suffer from a halo bias in that perceptions of nonfinancial performance are influenced by financial performance (DeBusk et al., 2005)  There are a range of behavioral factors which are important in the successful implementation and use of the Balanced Scorecard (de Waal, 2003)  Managers use simplifying strategies, are biased, and do not realize which measures they rely on when evaluating performance (Rich, 2007)

 Top management ignore underlying strategic links thereby creating conflict in performance evaluations (Wong-OnWing et al., 2007)  Evaluators focus on common measures and ignore measures that are unique to the managers strategy (Lipe & Salterio, 2000)  The common measure bias can be overcome by providing detailed information about the business unit’s strategy (Banker et al., 2004)  Disaggregating the BSC allows superiors to utilize unique as well as common measures. The BSC is used extensively in compensation decisions (Roberts et al., 2004)  Subjective bonuses complement quantitative measures and insure employees against downside risk. As trust increases, subjective measures become more effective and lead to greater employee satisfaction (Gibbs et al., 2004)

 The BSC can be an effective means of improving sustainability

233

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236 Table 2 (continued) Possible learning objectives subsequent measures of performance.

Construct a strategy map based on the cause-effect relationships implicit in an organization’s strategy.

Additional questions for discussion

Relevant research finding

managers continue to make the same relative level of investment in employee training, predict who will have the highest Return on Investment in 2-years time 2. Consider Manager C at Chemico. What risks to financial performance arise from the poor environmental performance?

management. Various approaches can be taken to create a Sustainability Balanced Scorecard (Figge et al., 2002)

1. Consider the measures in the BSC. Develop a SM in which all of the Learning and Growth, Internal Processes, Customer, and Financial measures are linked in series of causeeffect relationships 2. Consider the environmental performance measures. How might these be incorporated into the BSC?

 From a review of 32 studies, most find a positive relationship between environmental performance and financial performance (Molina-Azorín et al., 2009)  The BSC is used to various levels of effectiveness: from a combination of financial and non financial measures to an integrated strategic management system which focuses on cause-effect relationships and is linked to rewards (Speckbacher et al., 2003)  The number of performance measures, and the performance measures used in a BSC, depend heavily on the strategy of the organization (DeBusk et al., 2003)

5.3. Analysis of practitioner feedback In 2009 the Chemico case was completed by 31 financial controllers at a conference in Germany. These controllers had at least 1 year of experience at a managerial level (range 1–36 years, mean 7.17 years, s.d. = 7.785 years). On separate 11-point scales with 0 being lowest and 10 being highest, they indicated the extent to which they believed the case was: easy to understand, difficult to make an evaluation, and realistic. Recognizing the complexity of the performance evaluation, practicing controllers found it somewhat difficult to evaluate the manager’s performance. Their mean response was 6.42 (s.d. = 2.233) on a scale with 10 labeled as ‘‘Extremely difficult.’’ This is an important point to make when using the case with students, i.e., that performance evaluation is not straight forward and that even practicing accountants find it difficult. The financial controllers did, however, find the case easy to understand (mean = 7.42, s.d. = 1.336) and realistic (mean = 6.97, s.d. = 2.041) (with 10 being extremely easy and extremely realistic, respectively). The evaluations made by these practicing controllers also provide interesting comparisons that can be made during the debriefing stage of the case (see Table 3). For example, practitioners with extensive experience (greater than 10 years at a managerial level) were more likely to reward Manager A and penalize Manager B based on lead indicators of performance, even though the financial perfor-

Table 3 Practitioner evaluations and bonus allocation.

Evaluation Manager A (negative lag indicators) Bonus Manager A Evaluation Manager B (negative lead indicators) Bonus Manager B Evaluation Manager C (negative environmental measures) Bonus Manager C

1–4 years exp (n = 16)

5–10 years exp (n = 8)

10 + years exp (n = 7)

Average (n = 31)

6.69

5.75

8.57

6.87

$94,714 6.32

$83,812 5.88

$125,960 4.86

$98,956 5.87

$83,938 8.63

$96,052 7.00

$59,437 8.71

$81,531 8.23

$121,347

$120,135

$114,601

$119,511

234

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

Table 4 Student feedback of understandability, difficulty, and realism. Mean and (s.d.).

How easy did you find the case to understand? How difficult did you find it to make an evaluation? How realistic did you find the case?

1st year students (n = 46)

2nd year students (n = 27)

3rd year students (n = 80)

Average (n = 153)

5.93 (2.187)

6.46 (1.896)

6.59 (1.527)

6.37 (1.824)

5.02 (1.849)

5.59 (2.094)

5.67 (1.904)

5.46 (1.932)

6.59 (1.861)

5.67 (0.827)

6.93 (1.440)

6.83 (1.532)

Scale: 1 = ‘‘Not at all (easy to understand, difficult, realistic)’’,. . ., 10 = ‘‘Extremely (easy to understand, difficult, realistic)’’.

mance of Manager B was superior. This is consistent with more experienced managers recognizing the strategic consequences of poor lead indicators and therefore a long-term perspective on performance evaluation. 5.4. Analysis of student feedback So far, 153 students have provided feedback on the understandability, difficulty, and realism of the case (see Table 4). On 11-point scales, students found it somewhat easy to understand with a mean of 6.37 (s.d. = 1.824) compared to 10 which was labeled ‘‘Extremely easy to understand’’. The mean response to difficulty in evaluating the manager’s performance is 5.46 (s.d. = 1.932), compared with 0 for ‘‘Not difficult at all’’ and 10 for ‘‘Extremely difficult.’’ Students also indicated that they found the case to be realistic (mean = 6.83, s.d. = 1.532), with 10 being ‘Extremely realistic’’. Three groups of students (44 social economics students, 28 tourism students, and 16 general business students) at two different Universities of Applied Science in Germany provided feedback on the impact of the case on their understanding of the BSC and SMs. On 13-point scales students indicated

Table 5 Pre and post questions. Question To what extent do you believe that the Balanced Scorecard is useful for strategy development and implementation in the Tourism (Health) Industry? Rate your understanding of the Balanced Scorecard Rate your understanding of the Strategy Map

Mean (Min 0 Max 12)

N

Standard deviation (s.d.)

Pre

8.01

69

1.819

Pre Pre

4.47 2.59

72 71

2.454 2.845

Scale: 1 = ‘‘Not at All’’ or ‘‘No understanding’’ ,. . ., 12 = ‘‘Extremely useful’’ or ‘‘Complete Understanding’’.

Table 6 Student evaluations and bonus allocations. Means and (s.d.).

Evaluation Manager A (negative lag indicators) Bonus Manager A Evaluation Manager B (negative lead indicators) Bonus Manager B Evaluation Manager C (negative environmental measures) Bonus Manager C

1st year students (n = 80)

2nd year students (n = 99)

3rd year students (n = 133)

Average (n = 312)

5.66 (1.904)

5.55 (2.213)

6.04 (1.961)

5.79 (2.036)

$84,615 (33,510) 6.22 (2.081)

$83,730 (43,059) 5.59 (2.092)

$90,879 (38,988) 6.45 (2.139)

$86,999 (39,062) 6.11 (2.136)

$94,059 (34,325) 7.29 (2.044)

$87,109 (46,631) 7.09 (2.263)

$98,645 (39,183) 7.19 (1.996)

$93,803 (40,747) 7.18 (2.091)

$121,326 (37,163)

$124,070 (47,339)

$109,617 (41,922)

$117,216 (42,974)

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

235

the extent to which they believed the BSC was useful in strategy development and implementation within their field and rated their understanding of the BSC and SM. Students provided this feedback before and after completing the case. Significant improvements were achieved on all three measures, as indicated in Table 5. All differences between pre and post case responses are significant at p < 0.001. Particularly interesting is the increase in rated understanding of the SM. Many of the students had heard of the BSC in previous courses, but had not heard of the SM or seen its use. This is particularly concerning since BSC implementation failure is often attributed to a failure to tie the performance measures to the organization’s unique strategy (Speckbacher et al., 2003). The evaluations and bonuses from 312 students in first, second, and third year can be seen in Table 6. 6. Conclusion The major contribution of this case is that students experience the importance of viewing the individual BSC measures within the context of the organization’s strategy, and hence the importance of the SM. Furthermore, by using color-coding and emphasizing the passage of time in the cause-effect relationships, students are able to see the importance of anticipating the effects of the manager’s investment (or lack thereof) in the lead indicators of future financial performance. Environmental performance can be recognized as a driver of financial performance, or a risk factor to be considered when evaluating financial returns, and incorporated into the SM. Acknowledgements I offer my sincere appreciation to the Editor-in-Chief, David E. Stout, the Associate Editor and the two anonymous reviewers who took the time to provide extensive and very helpful comments and suggestions. Furthermore, the research instrument developed by Lipe and Salterio (2000) provided the inspiration for this case study. References Baddeley, A. (1994). The magical number seven: Still magic after all these years? Psychological Review, 101(2), 353–356. Banker, R. D., Chang, H., & Pizzini, M. J. (2004). The balanced scorecard: Judgmental effects of performance measures linked to strategy. The Accounting Review, 79(1), 1–23. Crabtree, A. D., & DeBusk, G. K. (2008). The effects of adopting the balanced scorecard on shareholder returns. Advances in Accounting, 24(1), 8–15. Davis, S., & Albright, T. (2004). An investigation of the effect of balanced scorecard implementation on financial performance. Management Accounting Research, 15(2), 135–153. de Waal, A. A. (2003). Behavioral factors important for the successful implementation and use of performance management systems. Management Decision, 41(8), 688–697. DeBusk, G. K., Brown, R. M., & Killough, L. N. (2003). Components and relative weights in utilization of dashboard measurement systems like the balanced scorecard. The British Accounting Review, 35(3), 215–231. DeBusk, G. K., Killough, L. N., & Brown, R. M. (2005). Financial measures bias in the use of performance measurement systems. Advances in Management Accounting, 14, 61–89. Dilla, W. N., & Steinbart, P. J. (2005). Relative weighting of common and unique balanced scorecard measures by knowledgeable decision makers. Behavioral Research in Accounting, 17, 43–53. Figge, F., Hahn, T., Schaltegger, S., & Wagner, M. (2002). The sustainability balanced scorecard - linking sustainability management to business strategy. Business Strategy and the Environment, 11(5), 269–284. Gibbs, M., Merchant, K. A., Van der Stede, W. A., & Vargus, M. E. (2004). Determinants and effects of subjectivity in incentives. The Accounting Review, 79(2), 409–436. Gibbs, M. J., Merchant, K. A., Van der Stede, W. A., & Vargus, M. E. (2005). The benefits of evaluating performance subjectively. Performance Improvement, 44(5), 26–32. Graham, H. (2009). Measuring organizational performance: Beyond the triple bottom. Business Strategy and the Environment, 18(3), 177–191. Hoque, Z., & James, W. (2000). Linking balanced scorecard measures to size and market factors: Impact on organizational performance. Journal of Management Accounting Research, 12(1), 1–17. Iselin, E. R., Mia, L., & Sands, J. (2008). The effects of the balanced scorecard on performance. The impact of the alignment of the strategic goals and performance reporting. Journal of General Management, 33(4), 71–85. Ittner, C. D., Larcker, D. F., & Randall, T. (2003). Performance implications of strategic performance measurement in financial services firms. Accounting, Organizations and Society, 28(7,8), 715–741. Kaplan, R., & Norton, D. (1992). The balanced scorecard – measures that drive performance. Harvard Business Review, 70(1), 71–79.

236

M. Wynder / J. of Acc. Ed. 28 (2010) 221–236

Kaplan, R., & Norton, D. (1993). Putting the balanced scorecard to work. Harvard Business Review, 71(5), 134–142. Kaplan, R., & Norton, D. (1996). The Balanced Scorecard. Boston MA: Harvard Business School Press. Kaplan, R., & Norton, D. (2000a). The Strategy-Focused Organization. Boston MA: Harvard Business School Press. Kaplan, R., & Norton, D. (2000b). Having trouble with your strategy? Then map it. Harvard Business Review, 78(5), 167–176. Kaplan, R., & Norton, D. (2008). Mastering the management system. Harvard Business Review, 86(1), 62–77. Kittiya, Y., & James, G. (2006). An extended performance reporting framework for social and environmental accounting. Business Strategy and the Environment, 15(5), 309–321. Libby, T., Salterio, S. E., & Webb, A. (2004). The balanced scorecard: The effects of assurance and process accountability on managerial judgment. The Accounting Review, 79(4), 1075–1094. Lipe, M. G., & Salterio, S. E. (2000). The balanced scorecard: Judgmental effects of common and unique performance measures. The Accounting Review, 75(3), 283–298. Lipe, M. G., & Salterio, S. E. (2002). A note on the judgmental effects of the balanced scorecard’s information organization. Accounting, Organizations and Society, 27(6), 531–540. Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63(2), 81–97. Molina-Azorín, J. F., Claver-Cortés, E., López-Gamero, M. D., & Tarí, J. J. (2009). Green management and financial performance. A literature review. Management Decision, 47(7), 1080–1100. Perrini, F., & Tencati, A. (2006). Sustainability and stakeholder management: The need for new corporate performance evaluation and reporting systems. Business Strategy and the Environment, 15(5), 296–308. Rich, V. (2007). Interpreting the balanced scorecard: An investigation into performance analysis and bias. Measuring Business Excellence, 11(1), 4–11. Roberts, M. L., Albright, T. L., & Hibbets, A. R. (2004). Debiasing balanced scorecard evaluations. Behavioral Research in Accounting, 16, 75–88. Slovic, P., & MacPhillamy, D. (1974). Dimensional commensurability and cue utilization in comparative judgment. Organizational Behavior and Human Performance, 11, 172–194. Speckbacher, G., Bischof, J., & Pfeiffer, T. (2003). A descriptive analysis on the implementation of balanced scorecards in Germanspeaking countries. Management Accounting Research, 14(4), 361–388. Wong-On-Wing, B., Guo, L., Li, W., & Yang, D. (2007). Reducing conflict in balanced scorecard evaluations. Accounting, Organizations and Society, 32(4/5), 363–377.