The contribution of measurement and information infrastructure to TQM success

Omega 34 (2006) 372 – 384 www.elsevier.com/locate/omega

The contribution of measurement and information infrastructure to TQM success W.A. Taylora,∗ , G.H. Wrightb a School of Management, University of Bradford, Emm Lane, Bradford, BD9 4JL, England, UK b Manchester Metropolitan Business School, Aytoun Building, Aytoun Road, Manchester, M1 3GH, England, UK

Received 22 June 2004; accepted 6 December 2004 Available online 11 February 2005

Abstract There is currently some debate about which TQM practices contribute most to superior performance outcomes. Several proponents argue that softer TQM practices such as leadership, human resource management, and customer focus have more impact than benchmarking, process analysis or performance measurement. The evidence for which TQM factors contribute most to improved performance is not yet conclusive, and sometimes contradictory. Using data from a longitudinal study of 67 TQM firms we contribute to this debate. Our central hypothesis is that measurement of key TQM practices and performance outcomes is essential for TQM success. We examine the measurement practices of this cohort of firms, and report on the changes in their measurement behavior over time. Specifically, we analyze seven dimensions of measurement relating to customer satisfaction, employee satisfaction, process performance, impact of TQM on costs, impact of TQM on sales, selfassessment, and benchmarking. We calculate a measurement-intensity score for each firm, based on how many of these seven parameters were being measured, and we show that increased measurement intensity is strongly associated with perceived TQM success. Finally, using multivariate discriminant analysis, we identify eight variables that explain the level of TQM success with a classification accuracy of almost 90%. We conclude that to attain the highest levels of TQM success, firms need to engage in the measurement practices of self-assessment and benchmarking, but our data suggest that an appropriate measurement framework needs to be in place beforehand. 䉷 2005 Elsevier Ltd. All rights reserved. Keywords: Total quality management; Measurement; Success; Benchmarking; Self assessment; Longitudinal study

1. Introduction This paper provides further insight into a cohort of 67 firms practicing TQM. The research design is longitudinal, and has tracked their TQM implementations since 1992. The specific focus of this paper is on the measurement practices of the firms and the relationship between measurement behavior and perceived TQM success. Our core hypothesis is that, since measurement and management by fact is a key ∗ Corresponding author. Tel.: +44 1274 234325; fax: +44 1274 234355. E-mail address: [email protected] (W.A. Taylor).

0305-0483/$ - see front matter 䉷 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.omega.2004.12.003

tenet of TQM, firms that gather data on parameters such as customer satisfaction, employee satisfaction, and process performance will be likely to experience higher levels of success with TQM. In other words, measurement can be as much a driver of improvement as a mere confirmation. In the first study of these firms in 1992 [1], we found that many were gathering very little data on which to base their judgments of the business impact of TQM, although for some, TQM was a relatively recent phenomenon. At that time the cohort comprised 113 firms. Based on our analysis of this initial data set, we questioned the sustainability of TQM in the absence of measurement and customer focus, and highlighted the need for greater organizational

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384

awareness of performance through the practices of benchmarking and self-assessment [2]. In the second phase of this study, we found that 37% of the cohort had discontinued TQM, largely in favor of ISO9000, leaving 67 firms where TQM was still operational. These 67 firms were achieving varying levels of success with TQM, with only 17 classifying it as very successful compared to 25 reporting less success than anticipated [3]. Subsequent analysis showed that the level of success was significantly related to five factors: (i) The time of adoption, with early TQM adopters achieving more success. (ii) Understanding of the meaning and purpose of TQM, with those who recognized that TQM was about making the customer the focus of all business processes gaining more success than those who saw it solely as an internally-focused problem solving activity. (iii) Understanding of the relationship between TQM and ISO9000, wherein those who understood ISO9000 to have a small but significant part to play within TQM experienced more success than the rest of the cohort. (iv) Treating TQM as a strategic rather than an operational business issue, with those that had written quality plans and objectives within their strategic business plans gaining more TQM success. (v) Senior management involvement in leading TQM, where more success was achieved than if responsibility was devolved to a quality manager or TQM coordinator. Interestingly, our earlier investigations showed that the level of success derived from TQM was not associated with size of organization or holding ISO9000 certification. These findings have broad support in the literature [4,5]. The focus of this current paper is on the measurement practices of this cohort of 67 firms, and in particular whether measuring the effects of TQM is associated with higher levels of success; there is, as yet, little empirical evidence to support the viewpoint. We therefore revisited the cohort of firms to establish the extent of their measurement practices and to identify whether there was any noticeable change in measurement intensity over time. The three objectives of this paper are: (i) To identify the changes in measurement practice in each firm to see whether or not more firms were now measuring key dimensions of TQM. (ii) To explore whether or not there was a relationship between the intensity of measurement practice and TQM success. (iii) To identify which measurement practices had the most significant impact on TQM success, and what was their influence relative to the five significant influences on TQM success identified in our earlier work, and listed above.

373

In the next section we review the evidence supporting the need for measurement as part of TQM implementation, and its relationship with improved business performance. We then outline our research method and the analysis techniques employed. Finally we discuss the implications of our findings and conclude with some observations about opportunities for further research. 2. Literature review Ten years ago, Fuchs highlighted a lack of measurement as one of the four obstacles to progress with TQM implementation [6]. The need for measurement was underlined by Voss, whose study of 202 manufacturing firms revealed low levels of measurement, such that many firms were regarded as “optimists who have little real understanding of where they stand” [7]. Indeed, one of the most widely cited TQM assessments of that period, interpreted by many as signaling the widespread failure of TQM, revealed that 50% of firms had no information with which to measure its effects on their performance [8]. Measurement is a core principle of TQM [9], emphasizing the use of data-driven approaches to evaluate customers’ needs and expectations, to energize continuous improvement and to empower employee groups and teams [10,11]. As an evaluation mechanism, measurement directs time and attention to results, facilitates the early diagnosis and correction of problems, and indicates what works and what does not. Measurement also supports the recognition of success and provides further impetus for targeted improvement [12]. Further, it enables communication between managers and employees, and contributes to an empowering environment for involving all organizational members in managing by fact [13]. The pervasive importance of information and analysis within TQM programs has also been underscored by more recent studies [14–16]. Therefore we posit that, to excel with TQM, firms need to develop a framework of measurement, data and analysis [17,18], enabling “informed management” and decision-making [2,19]. The practice of measuring is a tangible testament to a firm’s true commitment to the tenets of TQM. While many TQM programs “generate more enthusiasm than tangible improvement”, this is often because of a failure to link programs with results [20]. To espouse a customer focus without regularly and systematically measuring customer satisfaction, or to profess to be employee centered without similarly tracking employee satisfaction would undermine such claims. It is reminiscent of Vince Lombardi’s quotation that “if you are not keeping score you are only practicing”. As a football coach, Lombardi understood the need to measure: “In any sport, it is difficult to determine how well your team is doing unless you have complete, accurate and upto-date information on the team’s performance. If you want to determine the team’s standing and see how far you are

374

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384

from first place, you also need measures that allow you to compare your team to the very best in the league. Most important, if you expect to win, then all players must work cooperatively to achieve common goals” [21].

these ‘hard’ analytical practices relative to softer issues such as senior management understanding of TQM, and leadership, from our earlier work.

Lombardi’s philosophy has influenced many leading US practitioners of TQM, such as Milliken, Motorola and AT&T, all former Baldrige winners, whose approaches to measurement was one of the prime reasons for their awards [22]. Fundamentally, the emphasis on measurement is predicated on the belief that measuring results can lead to better results [23] and can increase the chances of a TQM program’s success [24]. By contrast, some recent studies of TQM indicate that the key practices that lead to superior outcomes are the softer, more intangible ones such as executive commitment, open organization and employee empowerment [25–27] rather than benchmarking, process analysis or performance measurement. Indeed, while Powell observed that these latter activities are nevertheless “indispensable to a fully-integrated TQM initiative, they apparently do not produce performance advantages in the absence of the intangibles” [25]. Powell’s study was one of the first to question the “interdependence assumption”, i.e. that organizations must implement the full set of TQM practices, which in combination, will produce superior performance. This integrated approach to TQM pervades much of the literature [28–31], positing that it is the combination of practices and their synergistic effects that lead to superior outcomes. Some recent studies have also questioned the interdependence assumption [32,33], and have suggested, for example, that only workforce commitment, shared vision and customer focus contribute to superior performance from TQM, and that the ‘harder’ practices such as benchmarking, advanced manufacturing technologies and cellular work teams are not essential [34]. Similarly, Samson and Terziovski found that three softer TQM practices, leadership, human resources management and customer focus were stronger predictors of performance than the more analytically-based dimensions of information and analysis, strategic planning and process analysis [27]. The evidence for which TQM factors contribute most to improved performance is not yet conclusive, and sometimes contradictory [35]. For example, Powell found only a weak relationship between customer focus and performance, whereas in Dow et al.’s (1999) study, customer focus had the strongest correlation with quality outcomes. This latter study also concluded that cellular work teams and personnel training were ‘hard’ rather than ‘soft’ TQM practices, which is debatable. Moreover, the few studies that have explored the relative contribution of these so-called hard and soft practices have differed in their sample sizes, constructs and methods of analysis. Our longitudinal data provides an opportunity to explore the measurement practices of our cohort of firms, and to investigate the association between the extent of measurement and TQM success, and the effect of

3. Research method 3.1. Sample When this study commenced in 1992, there were 113 TQM firms in the sample. This TQM cohort was a subsample of a survey of 2000 organizations in the textiles, clothing, food and drink, engineering, retailing, general manufacturing and service sectors, from which 682 responses were obtained. At that time therefore, the 113 firms practicing TQM represented 17% of the sample. By 1997, only 67 were continuing with TQM, with the remainder showing considerable preference for ISO9000. Of these 67 firms, their size classifications were: 22 small (up to 99 employees), 30 medium (100–499 employees) and 15 large (500 or more employees). We had taken several steps to sustain the involvement of this cohort in our study, including distribution of management reports of our earlier findings, and maintenance of contact by letter, telephone and by personal visits. For this current phase, we contacted respondents by telephone to establish if they were still continuing with TQM, and to make them aware of the forthcoming study, and then followed up with a letter outlining our objectives, and giving a further assurance of confidentiality. We had already engendered some trust in this regard by demonstrating integrity in the treatment of their previous responses. All 67 firms responded to this third survey, which is perhaps indicative of their level of interest in the findings. It is also notable that all 67 were still continuing with TQM. 3.2. Objectives We wished to establish three aspects of measurement behavior in this cohort of 67 firms. First, to identify the changes in measurement practice in each case to see whether or not more firms were now measuring key dimensions of TQM. Second, to explore whether or not there was a relationship between the intensity or extent of measurement practice and perceived TQM success. Finally, to identify which measurement practices had the most impact on TQM success, and what was their influence relative to the five significant factors identified in our earlier work, as reported in the introduction to this paper. 3.3. Instrument development and procedures We used a modified version of our existing survey instrument, since it had already been extensively pilot-tested and reviewed in the light of the previous two distributions in 1992 and 1997. Two questions were added to explore the

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384

extent to which benchmarking and self-assessment were being practiced. The dependent variable, TQM success, was a categorical, self-reported measure of the extent to which TQM had contributed to the improved performance of the organization relative to its specific business strategy. The dependent variable comprised three categories of success: unsuccessful or less than anticipated, quite successful and very successful. The questionnaire was mailed to the most senior manager with responsibility for TQM in each firm. A follow-up letter with another copy of the questionnaire was sent six weeks later, and telephone calls were used to chase any outstanding responses. Since respondents had already indicated their willingness to participate, the main reasons for delayed replies were pressure of workload and lack of time.

375

discriminant analysis does not affect the interpretation of results [41,42]. We tested for normality, examining the zcoefficients for skewness and kurtosis, and found that all variables were within acceptable limits. Similarly, tolerance and variance inflation factor values indicated no serious problems with multicollinearity. We used the direct discriminant procedure, where all independent variables are submitted concurrently, since we had formed no initial hypotheses about their relative importance [43]. Further, although the sub-groups were of unequal size, we adopted the conservative approach of using equal classification probabilities, since we could not be certain that the observed proportions in the sample were representative of their population [39]. The sample size did not permit the use of a hold-out sample for model validation [44].

3.4. Data analysis To address the first research objective, we asked each firm whether or not they were still measuring the five variables examined in our earlier studies, i.e. the impact of their TQM implementation on costs, sales, customer satisfaction, process performance and employee satisfaction. This enabled us to identify the changes in measurement behavior for each firm. For the second objective we calculated a measurement intensity function, i.e. the degree to which all five of these measures were being monitored, using the mean score for all five measurement variables for each firm. This approach has some precedent in the literature [36,37]. Since in this current study we also asked about two additional measurement practices, relating to internal self-assessment, and benchmarking of TQM practices and performance against other firms, we therefore calculated a second measurement intensity function comprising these two new variables in combination with the original five. For both intensity scores we used the chi-square statistic to test for a significant association with reported TQM success. For the final objective, given the categorical nature of the dependent variable, we utilized canonical discriminant analysis to identify those variables that significantly predict TQM success. Discriminant analysis provides a statistical procedure to identify the research variables that will discriminate best between a priori defined groups [38]. Discriminant analysis is sensitive to the ratio of sample size to the number of independent variables, with the recommended minimum ratio being 5:1. Our sample met this criterion with a ratio of 5.6:1. It is also recommended that all sub-groups represented in the dependent variable should have more cases than the number of independent variables used for analysis [39]. Our smallest sub-group, where TQM was classified as very successful, had 17 cases, relative to the 12 independent variables used, which again is acceptable. The accuracy of classifications from discriminant analysis are sensitive also to assumptions about the independent variables, particularly regarding normality and multicollinearity [40], although some have argued that multicollinearity in

4. Results Table 1 presents the bi-variate correlations of the variables. The definition of variables 1–5 is detailed in full in our previous papers [1–3]. Variables 6–12 are measures of the extent to which each measurement practice is used regularly in each firm. There is clearly some multicollinearity in Table 1, reflecting what many others have found, that firms tend to implement TQM practices in combination. However, as Dow et al. (1999) point out, this does not mean that all practices must be implemented together in order to be effective, and that the degree of inter-correlation may simply indicate that “the various proponents of TQM have been successful in convincing managers to adopt their entire package” [34].

4.1. Changes in measurement behavior Of the 67 firms continuing with TQM, there was an overall improvement in measurement behavior across all five of the original metrics, see Fig. 1. The biggest increases were in the measurement of sales and cost data to assess the impact of TQM implementation on firm performance, although it should be noted that over half of the cohort are still not using the latter sales measure. By contrast, customer satisfaction, which is a more direct indicator of the effect of TQM, represents the lowest overall increase in measurement. Measures of two key TQM enablers, employee satisfaction and process performance also displayed healthy improvements of 50% and 32.4%, respectively. However, analysis on a case-by-case basis reveals some interesting changes in these firms’ proclivity to measure. For example, as Fig. 2 shows, while measurement of customer satisfaction had a net increase of one firm, there were nine firms that had begun to measure customer satisfaction since 1997, while another eight had ceased this practice.

376

Table 1 Bi-variate correlation matrix v2

v3

v4

v5

v6

v7

v8

v9

v10

v11

0.202 0.298* 0.349** 0.128 0.341** 0.340** 0.337** 0.267* 0.328** 0.357** 0.298*

0.135 0.187 0.273* 0.391** 0.315** 0.361** 0.351** 0.389** 0.379** 0.272*

0.211 0.382** 0.313* 0.336** 0.303** 0.298* 0.433** 0.312** 0.328**

0.198 0.317** 0.387** 0.287* 0.317* 0.315** 0.301* 0.217

0.298* 0.317** 0.399** 0.329** 0.345** 0.315** 0.495**

0.316** 0.368** 0.364** 0.340** 0.277* 0.428**

0.304** 0.466** 0.304** 0.377** 0.305**

0.377** 0.498* 0.328** 0.490*

0.378** 0.308** 0.367*

0.366** 0.413**

0.306**

v1 When TQM began. v2 Understanding of the TQM-ISO900 relationship. v3 Senior management leading TQM. v4 Having written quality plans and objectives. v5 Understanding of TQM. v6 Sales used to assess TQM impact. v7 Costs used to assess TQM impact. v8 Practicing self-assessment. v9 Customer satisfaction measured v10 Employee satisfaction measured. v11 Process performance measured. v12 Practicing benchmarking. ∗ p < 0.05. ∗∗ p < 0.01.

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384

v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12

v1

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384 Employee satisfaction

20

TQM impact on Sales

37

10

15

Process performance

34

11

Customer satisfaction TQM impact on Costs

1

10

20

30

16

40

50

1

Self assessment only Both

18

26

25

0

Benchmarking only 22

48

47

Neither

32

20

377

4 15 0

60

10

20

30

40

50

60

70

Number of firms

Number of firms Measuring in 1997

Current increase

Still not measuring

Fig. 1. Improvements in measurement activity.

Still not measuring

10

Stopped measuring

8

Begun to measure

9

Still measuring

40 0

10

20

30

40

50

60

70

Number of firms Fig. 2. Measuring customer satisfaction?

Of the eight firms that had stopped measuring customer satisfaction, six were small and two medium-sized. When asked for their reasons for discontinuing this measurement, the most prominent explanation was the heavy administrative burden of data collection and analysis, which was perceived to be a cost that had a disproportionately small benefit. Similarly, there were also 4 firms that had discontinued the measurement of employee satisfaction and one that had stopped using costs as an indicator of the impact of TQM on the firm’s performance. These instances of measurement retrenchment were located within the same subgroup, i.e. the four firms no longer measuring employee satisfaction were also no longer measuring customer satisfaction. The one firm no longer measuring the cost-impact of TQM had also stopped measuring customer satisfaction and employee satisfaction. 4.2. Intensity of measurement and TQM success Turning to the second research question, we now consider the association between measurement intensity and TQM success. First, by calculating the mean value of the composite measurement behavior of each firm, using the five measurement components in Fig. 1, we can locate firms across the measurement-intensity spectrum. At one end are 22 firms that are using all five measures, and at the other, there are 11 firms using none. By grouping these mean scores into three categories, we then explored the relationship between measurement behavior and success, Table 2. Clearly, there is a positive and significant association between measurement and success (p < 0.001). Perhaps most

Fig. 3. Using benchmarking and self-assessment?

strikingly, all 15 firms with little or no measurement experienced least TQM success. In the current study we also established whether firms were using self-assessment or benchmarking as part of their TQM measurement practices. Some 19 firms were using self-assessment and 16 were employing benchmarking (Fig. 3). Only 15 were using both approaches. By including these two additional measures for selfassessment and benchmarking, we extended the analysis in Table 2, to calculate a measurement intensity score for each firm, for all seven measurement variables. Table 3 shows the link between TQM success and the level of adoption of all seven measurement practices. There remains a significant and positive relationship, again with all 15 firms with poor measures reporting least success (p < 0.001). So, collectively, the intensity of measurement behavior is significantly associated with TQM success. The final question is therefore, which of these seven measurement practices have most impact, and do they have more or less influence relative to the five significant variables from our earlier work? 4.3. Relative influence of measurement on TQM success To investigate this, we conducted multiple discriminant analysis to identify which independent variables explain the classification of firms according to the reported TQM success categories. Given three classes of outcome for the dependent variable (Table 3), two discriminant functions were produced, Table 4. Both discriminant functions were significant (p < 0.0001) as measured by the Wilk’s lambda and chi-square statistics. The two functions account for 92.2% of the variance, which is generally considered to be very acceptable [40]. The discriminant functions are statistically significant, with good classification accuracy. This permits further consideration of the functions and the relative importance of each independent variable. The standardized discriminant function coefficients and discriminant loadings are given in Table 4. Discriminant loadings, also known as structure correlations, measure the linear correlations between each independent variable and the extracted discriminant function [40,45]. They are considered to be equivalent to factor loadings, and represent the relative contribution of each variable to the discriminant function [46]. These loadings are used to

378

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384

Table 2 Association of number of measures used with TQM success TQM success

Unsuccessful or less than anticipated Quite successful Very successful Total
2 = 46.34, 4 df, p < 0.001

Extensive measures (4 or 5)

Some measures (2 or 3)

Little or no measurement (1 or no measures)

Total

3

7

15

25

16 17 36

9 0 16

0 0 15

25 17 67

Table 3 Overall association between measurement practices and TQM success TQM success

Unsuccessful or less than anticipated Quite successful Very successful Total
2 = 49.908, 6 df, p < 0.001

Extensive measures (6–7)

Some measures (4–5)

Weak measurement (2–3 measures)

Poor measurement (1 or no measures)

1

2

7

15

10 14 25

7 3 12

8 0 15

0 0 15

Table 4 Results of discriminant analysis Variable

Using benchmarking Practicing self-assessment Costs used to assess TQM impact Customer satisfaction measured Employee satisfaction measured Process performance measured Who leads TQM Written plans and objectives for quality When TQM began Sales used to assess TQM impact Understanding of TQM-ISO9000 relationship Understanding of TQM Eigenvalue Canonical correlation Wilk’s Lambda


2

P

Function 1

Function 2

Structure correlations

Discriminant function coefficients

Structure correlations

Discriminant function coefficients

0.842a 0.397a 0.134 0.141 0.182 0.135 0.095 0.067 0.054 0.125 0.085

0.936 0.390 0.171 0.048 0.111 0.067 0.058 0.011 −0.104 −0.193 0.083

−0.230 −0.047 0.761a 0.718a 0.472a 0.430a 0.357a 0.342a 0.269a 0.237a 0.211a

−0.248 −0.009 0.426 0.492 0.289 0.015 0.171 0.001 0.182 0.002 0.048

0.093

0.209a

0.007

0.153

16.717 0.971 0.023 219.556 (24 df) 0.000

a Largest absolute correlation between each variable and any discriminant function.

1.407 0.765 0.15 51.397 (11 df) 0.000

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384 Table 5 Group centroids for the two discriminant functions Category of TQM success I. Unsuccessful or less success than anticipated II. Quite successful III. Very successful

Function 1

Function 2

2.688

1.285

1.957 −6.832

−1.391 0.157

independent variables used, and secondly by evaluating the Malhanobis D 2 distance of each observation from the group centroid. No obvious pattern could be detected. 5. Discussion and conclusions

determine the significance of the variables. While there are no rigid rules about the goodness of these values, the generally accepted guideline is that values above 0.3 are considered acceptable [39,47,48] The discriminant functions provide good separation between the three levels of TQM success, as indicated by the Wilk’s Lambda value, and comparison of the group centroids or discriminant Z-scores, Table 5. Function 1 is characterized by two measurement variables: (i) benchmarking of TQM practices and performance against other firms, and (ii) internal self-assessment. Function 2 is characterized by four measurement practices, impact of TQM on costs, customer satisfaction, employee satisfaction and process performance, together with the incorporation of written quality objectives and goals in strategic plans, and senior management leadership of TQM, as discussed in our earlier papers. The variables that do not significantly discriminate between the three classes of TQM success are: • • • •

379

The time since adoption of TQM. Measuring the impact of TQM on sales. Understanding of the meaning and purpose of TQM. Understanding of the relationship between TQM and ISO9000.

Closer inspection of the group centroids suggests that function 1 discriminates between the firms where TQM is very successful, and the rest, i.e. between group III versus groups I and II in Table 5, whereas function 2 discriminates between the quite successful and unsuccessful firms i.e. between group II versus group I also in Table 5. However, because function 1 accounts for a much larger proportion of the variance, caution must be used to infer the impact of the variable loadings on function 2. The discriminant functions classify correctly 89.6% of the cases (Table 6), which is considerably better than that expected by chance (maximum chance criterion = 37.3% (see [39, p. 268]); proportional chance criterion = 34.3%), see [39, p. 269]). Only seven cases were classified incorrectly, with six of the seven cases experiencing less success than predicted. It is recommended that misclassified observations should be examined to identify any characteristics they possess in common that could be incorporated into the discriminant analysis to improve classification accuracy [39]. These seven cases were examined, firstly by profiling them on the twelve

5.1. Implications for research This paper has produced five contributions to the literature. In relation to our first research question, this work remains one of a very few longitudinal studies of TQM implementation, and shows how measurement behavior has changed over time. For this cohort of 67 firms, their propensity to measure key dimensions of TQM implementation has increased significantly. Nevertheless, many of the firms continue to avoid measuring many of these parameters, with, in extremis 16 (24%) still not measuring the impact of TQM on cost efficiency and 37 (55%) not measuring employee satisfaction. In a small number of cases, measurement had been discontinued, mainly due to perceptions about the disproportionately high cost of data collection relative to the benefits. Such continuing absence of measurement leads, in our opinion, to the real possibility that these TQM programs will be discontinued due to a lack of tangible results. Our second research question was predicated on the notion that there would be a positive relationship between the extent of measurement and TQM success. By computing a measurement intensity factor, we have shown that a broader portfolio of measures is associated with higher levels of TQM success. This does not mean that the mere act of measuring necessarily leads to more success from TQM. Of at least equal importance are the methods of quantification of these outcome measures and the ways that they are communicated and used within each firm. Nevertheless, measuring seems to be an important first step in the process of establishing a cybernetic feedback loop between activities and achievements, such that employees may correct their behavior without additional management intervention [9]. The practices of self-assessment and benchmarking did not permit longitudinal comparison, since these two variables were not included in the earlier surveys. Nonetheless, these two activities are currently not widely practiced within the cohort, casting further doubt upon the continuance of many of these TQM programs. Our third research question concerned the TQM practices that had most impact on TQM success. By using discriminant analysis, we have established the relative contribution of each measurement practice to TQM program success. Further, by incorporating the five variables discussed in our earlier work, we have provided additional insight into the relative effects of some of the purported ‘hard’ and ‘soft’ TQM practices. The discriminant functions have classified almost 90% of the cases accurately, which is much higher than a chance-based classification strategy. More specifically, six factors differentiate quite successful from unsuccessful or disappointing TQM initiatives, these being in order of

380

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384

Table 6 Within-group prediction accuracy Actual group

Predicted group I

I. Unsuccessful or less than anticipated II. Quite successful III. Very successful Total cases correctly classified

Discriminant function II

III

predictive accuracy (%)

19

6

0

76

0 0

25 1

0 16

100 94.1 89.6

significance: • Measurement of the impact of TQM implementation on cost efficiency. • Measurement of customer satisfaction. • Measurement of employee satisfaction. • Measurement of process performance. • Senior management leadership of the TQM program. • Incorporation of quality plans and objectives in the strategic planning process. Moreover, the two measurement practices that distinguish the very successful TQM programs from the others are self-assessment and benchmarking. Given the emerging body of empirical findings suggesting that most benefit can be derived from softer TQM practices [27,29], our results are somewhat surprising at first glance, in that both self-assessment and benchmarking could be regarded as so-called ‘hard’ measurement practices. However, the artificial dichotomy between soft and hard practices is not clear-cut. For example, both Powell (1995) and Dow et al. (1999) classify customer focus as a softer practice, yet if customer focus includes measurement of customer satisfaction it could easily be regarded as the opposite. Similarly, self-assessment is essentially a subjective, judgmental and imprecise process, and benchmarking need not be based upon results alone, e.g. process benchmarking. Perhaps a more helpful view is provided by Rahman and Bullock [49], who found that soft and hard practices should co-exist and that for hard TQM to impact performance, it must be underpinned by the softer elements of TQM, such as, in our analysis, senior management leadership. Our fourth contribution, and of no less significance, concerns our findings that understanding of the purpose of TQM and its relationship with ISO9000, the time since TQM adoption and the measurement of TQM’s impact on sales have no explanatory capacity for this cohort of firms. This of course does not mean that they are unimportant. Indeed, it our earlier studies of this cohort, the first three of these variables were found to be significantly associated with TQM success, as outlined in the introduction section of this paper. It may be that as these TQM programs have developed and matured, the inter-firm differences in these variables have diminished.

Moreover, the lack of influence of the sales measure could be explained by the relatively larger contribution of measuring customer satisfaction, which is a more direct measure of meeting customer’s need and expectations. In fact sales could be regarded as an outcome of customer satisfaction levels. Finally, taken together, these results support the view that firms may be able to capture some of the benefit from TQM without subscribing to the full ideology [25,34]. However, the converse may also hold, i.e. that firms may only be able to capture the full benefits by subscribing to the complete set of TQM principles. We do not present this as more than speculation, given the size of our sample, in the hope that it may stimulate further empirical work. Nevertheless we acknowledge that the originator of the viewpoint had an even smaller data set, and that this was not longitudinal in nature [25]. 5.2. Implications for management practice These findings have potential implications for practicing managers. One of the most important of these is the need to measure key components of TQM activity, especially its on-going progress and results through systematic self-assessment. This should be combined with comparisons of practices and concomitant results against high-performing firms through benchmarking. The corollary is that, without comprehensive measurement, TQM is likely to lead to disappointing results that may well result in its continuance being questioned. Moreover, self-assessment and benchmarking should not be regarded as optional extras, but rather they should be understood to lead to higher levels of TQM success. Since benchmarking and self-assessment appear to make most difference to the level of success achieved from TQM, should managers wanting to enhance TQM success implement these early on in the process? The short answer, in our view, is ‘no’. This is because both self-assessment and benchmarking rely upon firms having comparative trends of their own performance to hand, and this in turn suggests the need to establish measurement processes and an information infrastructure to use these data before embarking on the former practices. In a similar vein, our results

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384

Areas for improvement Areas for improvement unknown by others outside the known by others outside the organization organization

Areas for improvement known to organization

381

Areas for improvement unknown to organization

II.Corporate blindness needing better connection to customers and competitors I. Current TQM improvement focus

• • • •

Self assessment Employee measures Process measures Cost measures

• •

Benchmarking Customer satisfaction measures

IV. Knowledge about areas for improvement still undiscovered

III. Self-deception - needing better knowledge of internal performance

Learning zones

Fig. 4. Contributions of measurement to learning in TQM.

seem to resonate with Ritchie and Dale’s view that organizations should possess a certain level of TQM experience and development before adopting approaches such as selfassessment and benchmarking [50], i.e. there is a necessary lifecycle of progression in practices toward higher levels of TQM success. However, our data do not provide unequivocal support for this stance; consequently further investigation is warranted, through in-depth case study, especially among the firms in the cohort using most measures, to understand their approaches to the development of a measurement infrastructure. Nevertheless, as argued earlier, organizations cannot be regarded as legitimate exponents of TQM without measurement of fundamental parameters such as customer satisfaction, employee satisfaction and process performance. Thus, we propose that managers should develop their organizations’ information and knowledge infrastructures along two vital dimensions, see Fig. 4 [2]. These two dimensions are: • Quadrants I–II. To provide information and knowledge about the organization’s connections with its environment, to provide better feedback on performance gaps and to remove any blindness about customers’ needs and expectations, customer satisfaction levels and business performance relative to competitors. • Quadrants I–III. To give a more accurate and complete view of actual performance levels within the organization, i.e. to increase self-knowledge about employee satisfaction and process performance. Without this complementary knowledge of the firm’s resources and capabilities, the additional connectedness with the external environment may only lead to dysfunctional tensions.

5.3. Measurement and organizational learning The relationship between TQM and learning has long been acknowledged, initially in terms of the connection between continuous improvement and learning curves [51–54], but laterly by consideration of the learning processes in TQM [55–58] and their capacity to generate new knowledge [59–61]. Fig. 4 represents the central roles of measurement and information in cultivating a learning orientation in TQM programs. It helps to explain why those firms with less measurement activity have experienced less success with TQM, because they are weakly connected to their customers and competitors, and less aware of the performance of their people and processes. Fig. 4 is not intended to typify all aspects of a learning orientation within TQM, since learning is essentially a human activity, but it does emphasize the important contribution to learning that derives from measurement. Developing this notion of information and knowledge infrastructure, we can position the measurement practices from our study within Fig. 4 to represent two components of learning. The first learning component (quadrant II) is that which minimizes knowledge gaps in process and product performance relative to the delivery of products and/or services to external constituencies. The second learning component (quadrant III) enhances internal understanding of the “corporate self”. In theory, the increase in the quadrants of Fig. 4 represented by these two learning components should ultimately minimize the size of the fourth quadrant that signifies knowledge about areas for improvement as yet undiscovered by the firm, and as yet unknown to external constituencies. In practice it should at the very least reduce this quadrant and its concomitant risks to the firm.

382

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384

We believe that the role of learning within TQM has much potential for further investigation, particularly when linked to the development of an organization’s knowledge generation and innovation capabilities [62]. These connections will be explored in the next phase of our longitudinal study.

Moreover, it seems necessary to investigate the currently less successful TQM programs to identify the obstacles to enhanced measurement and adoption of self-assessment and benchmarking.

References 5.4. Limitations and future research All research has limitations and it is important to recognize these explicitly. It would be desirable to have larger samples to increase the case-to-variable ratio for statistical analysis and to increase the generalizability of results. In the same vein, these findings should be validated by using other samples. However, one of the consequences of a longitudinal research design is that the original sample is likely to diminish over time through non-responses, firms that cease to trade, and as in this study, firms that discontinue to practice TQM. We have also placed reliance upon a single respondent in each firm, and on one measure of success. In future work, it is desirable to use multiple respondents in each firm to improve respondent reliability and to have more than one indicator of the dependent variable [19,29]. We acknowledge that many perceptual, self-reported measures were used in this study. However, there are many precedents in the literature for obtaining performance information on a primary or perceptual basis [63–67]. By the same token there are acknowledged difficulties associated with self-report measures particularly when they are used for variables that are closely correlated [68]. One of these limitations is the possibility of common method variance and the contamination of potential relationships by a subject’s desire for consistency or social desirability [69]. Respondent bias is also a concern where, as in this study, some questions explore senior management behavior and have potentially uncomplimentary implications [34]. It is also desirable to disaggregate the dimensions of many of the variables such as process performance measurement and benchmarking, by inclusion of multiple-item scales. Further research is also needed to explore the methods used by responding firms to measure many of the variables such as customer and employee satisfaction, and to examine the ways in which these data are communicated, disseminated and used internally. However, there is always a practical trade-off between the coverage of a survey instrument and the burden it will place on the respondents and the consequent impact on response rates, especially when longitudinal data is required for a cohort study. In our previous studies we have used a multi-method approach, combining surveys with in-depth interviews of key informants. It now seems timely to conduct further interviews and in-depth case studies of sub-groups of this cohort, especially in those firms measuring all seven dimensions, to identify the practices they employ and the extent of their learning orientation.

[1] Taylor WA. Leadership challenges for smaller organisations: self-perceptions of TQM implementation. Omega, International Journal of Management Science 1997;25(5): 567–79. [2] Taylor WA. TQM implementation: organisation practices and information infrastructures. Omega, International Journal of Management Science 1998;26(6):715–27. [3] Taylor WA, Wright GH. A longitudinal study of TQM implementation: factors influencing success and failure. Omega, International Journal of Management Science 2003;31(2):97–111. [4] Terziovski M, Sohal A, Moss S. Longitudinal analysis of quality management practices in Australian organizations. Total Quality Management 1999;10(6):915–26. [5] Sohal A, Terziovski M. TQM in Australian manufacturing: factors critical to success. International Journal of Quality and Reliability Management 2000;17(2):158–67. [6] Fuchs E. Total quality management from the future: practices and paradigms. Quality Management Journal 1993;10(1): 26–34. [7] Voss CA. Made in Britain: the true state of Britain’s manufacturing industry. London: 1993, IBM (UK) and London Business School; p. 16. [8] Kearney AT. Total quality: time to take off the rose tinted spectacles. London: 1991, A T Kearney; p. 12. [9] Lau RSM, Anderson CA. A three-dimensional perspective of total quality management. International Journal of Quality and Reliability Management 1998;15(1):85–98. [10] Dean JW, Bowen DE. Management theory and total quality: improving research and practice through theory development. Academy of Management Review 1994;19(3):392–418. [11] Waldman DA. The contributions of total quality management to a theory of work performance. Academy of Management Review 1994;19(3):510–36. [12] Plenert G. Total quality management (TQM)—putting structure behind the philosophy. International Business Review 1996;5(1):67–78. [13] Hackman JR, Wageman R. Total quality management: empirical, conceptual and practical issues. Administrative Science Quarterly 1995;40:309–42. [14] Wilson DD, Collier DA. An empirical investigation of the Malcolm Baldrige National Quality Award causal model. Decision Sciences 2000;31(2):361–90. [15] Lee SM, Rho B-H, Lee S-G. Impact of Malcolm Baldrige National Quality Award criteria on organizational quality performance. International Journal of Production Research 2003;41(9):2003–20. [16] Merino-Diaz de Cerio J. Quality management practices and operational performance: empirical evidence for Spanish industry. International Journal of Production Research 2003;41(12):2763–86. [17] Evans JR, Lindsay WM. The management and control of quality. Minneapolis, MN: West Publishing;1996.

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384 [18] Lascelles DM, Dale BG. Difficulties and barriers to quality improvement. In: Dale BG, editor. Managing quality. Hertfordshire: Prentice-Hall; 1994. [19] Saraph JV, Benson PG, Shroeder RG. An instrument for measuring the critical factors of quality management. Decision Sciences 1989;20(4):810–29. [20] Krishnan R. et al. In search of quality improvement: problems of design and implementation. Academy of Management Executive 1993;7(4):7–19. [21] Briggs J. Strive to excel: the will and wisdom of Vince Lombardi. Nashville, TN: Rutledge Hill Press; 1997. p. 160. [22] George S, Weimerskirch A. Total quality management: strategies and techniques proven at today’s most successful companies. New York, NY: Wiley; 1994. [23] Hakes JE. Can measuring results produce results: one manager’s view. Evaluation and program planning 2001;24:319–27. [24] Capon N, Kaye MM, Wood M. Measuring the success of a TQM programme. International Journal of Quality and Reliability Management 1995;12(8):8–22. [25] Powell TC. Total quality management as competitive advantage: a review and empirical study. Strategic Management Journal 1995;16:15–37. [26] Ahire SL, Golhar DY. Quality management in large vs small firms. Journal of Small Business Management 1996;34:1–13. [27] Samson D, Terziovski M. The relationship between total quality management practices and operational performance. Journal of Operations Management 1999;17:393–409. [28] Flynn BB, Schroeder RG, Sakakibara S. A framework for quality management research and an associated measurement instrument. Journal of Operations Management 1994;11(4): 339–66. [29] Ahire SL, Golhar DY, Waller MA. Development and validation of TQM implementation constructs. Decision Sciences 1996;27(1):23–56. [30] Kaynak H. The relationship between total quality management practices and their effects on firm performance. Journal of Operations Management 2003;21:405–35. [31] Reed R, Lemak DJ, Mero NP. Total quality management and sustainable competitive advantage. Journal of Quality Management 2000;5(1):5–26. [32] Curkovic S, Vickery S, Droge C. Quality-related action programs: their impact on quality performance and firm performance. Decision Sciences 2000;31(4):885–905. [33] Chin K-S. et al. An AHP based study of critical factors for TQM implementation in Shanghai manufacturing industries. Technovation 2002;22:707–15. [34] Dow D, Samson D, Ford S. Exploding the myth: do all quality management practices contribute to superior quality performance?. Production and Operations Management 1999;8(1):1–27. [35] Martinez-Lorente AR, Dewhurst FW, Gallego-Rodriguez A. Relating TQM, marketing and business performance: an exploratory study. International Journal of Production Research 2000;38(14):3227–46. [36] Ravichandran T. Swiftness and intensity of administrative innovation adoption: an empirical study of TQM in information systems. Decision Sciences 2000;31(3):691–724. [37] Rai A, Patnayakuni R. A structural model for CASE adoption behavior. Journal of Management Information Systems 1996;13(2):205–35.

383

[38] Huberty CJ, Wisenbaker JM. Discriminant analysis: potential improvements in typical practice. In: Wilf HS, editor. Statistical methods for digital computers. New York, NY: Wiley; 1994. p. 79–96. [39] Hair JF. et al. Multivariate data analysis. New Jersey: PrenticeHall; 1998. [40] Morrison DG. On the interpretation of discriminant analysis. Journal of Marketing Research 1969;6:156–63. [41] Ramanujam V, Venkatraman N, Camillus J. Multi-objective assessment of effectiveness of strategic planning: a discriminant analysis approach. Academy of Management Journal 1988;29(2):347–72. [42] Eisenbeis RA. Pitfalls in the application of discriminant analysis in business, finance and economics. Journal of Finance 1977;32:875–900. [43] Latham DR, Stewart DW. Organizational objectives and winning: an examination of the NFL. Academy of Management Journal 1981;24(2):403–8. [44] Crask MR, Perreault WD. Validation of discriminant analysis in Marketing research. Journal of Marketing Research 1977;14:60–8. [45] Watson CJ. Approaches for the interpretation of multiple discriminant analysis in organizational research. Academy of Management Review 1982;7(1):124–32. [46] Montemayor EF. Congruence between pay policy and competitive strategy in high-performing firms. Journal of Management 1996;22(6):889–908. [47] Premkumar G, Roberts M. Adoption of new information technologies in rural small businesses. Omega, International Journal of Management Science 1999;27:467–84. [48] Magenau JM, Martin JE, Peterson ME. Dual and unilateral commitment among stewards and rank-and-file union members. Academy of Management Journal 1988;31(2): 359–76. [49] Rahman S, Bullock P. Soft TQM, hard TQM, and organisational performance relationships: an empirical investigation, Omega, International Journal of Management Science, 2005;33:73–83. [50] Ritchie L, Dale BG. Self-assessment using the business excellence model: a study of practice and process. International Journal of Production Economics 2000;66(3): 241–54. [51] Zangwill WI, Kantor PB. Toward a theory of continuous improvement and the learning curve. Management Science 1998;44(7):910–20. [52] Fine CH. Quality improvement and learning in productive systems. Management Science 1986;32:1301–15. [53] Tapiero CS. Production learning and quality control. IEEE Transactions 1987;19(4):362–70. [54] Dada M, Marcellus R. Process control with learning. Operations Research 1994;42:323–36. [55] Mukherjee AS, Wassenhove LNV. The impact of knowledge on quality. In: Karmarkar US, editor. The practice of quality management. Dordrecht, NL: Kluwer Academic Publishers; 1997. [56] Garvin DA. Building a learning organization. Harvard Business Review 1993;71(4):78–92. [57] Sitkin SB, Sutcliffe KM, Schroeder RG. Distinguishing control from learning in total quality management: a contingency perspective. Academy of Management Review 1994;19(3):537–64.

384

W.A. Taylor, G.H. Wright / Omega 34 (2006) 372 – 384

[58] Flynn B. et al. World-class manufacturing project: overview and selected results. International Journal of Operations and Production Management 1997;17(7):671–85. [59] Mukherjee AS, Lapre MA, Wassenhove LNV. Knowledge driven quality improvement. Management Science 1998; 44(11): S36–49. [60] Rungtusanatham M, Ogden JA, Wu B. Advancing theory development in total quality management. International Journal of Operations and Production Management 2003;23(8):918–36. [61] Drew SAW. From knowledge to action: the impact of benchmarking on organizational performance. Long Range Planning 1997;30(3):325–6. [62] Prajogo DI, Sohal AS. The relationship between TQM practices quality performance and innovation performance. International Journal of Quality and Reliability Management 2003;20(8):901–18. [63] Covin J, Slevin D, Schultz R. Implementing strategic missions: effective strategic, structural and tactical chaos. Journal of Management Studies 1994;31(4):481–505.

[64] Dess G. Consensus on strategy formulation and organizational performance: competitors in a fragmented industry. Strategic Management Journal 1987;8(3):259–77. [65] Choi T, Eboch K. The TQM paradox: relations among TQM practices, plant performance, and customer satisfaction. Journal of Operations Management 1998;17:59–75. [66] Ahire SL, O’Shaughnessy KC. The role of top management commitment in quality management: an empirical analysis of the auto parts industry. International Journal of Quality Science 1998;3(1):5–37. [67] Douglas TJ, Judge WQ. Total quality management implementation and competitive advantage: the role of structural control and exploration. Academy of Management Journal 2001;44:158–69. [68] Podsakoff PM, Organ DW. Self-reports in organizational research: problems and prospects. Journal of Management 1986;12(4):531–44. [69] Szajna B. Software evaluation and choice: predictive validation of the technology acceptance instrument. MIS Quarterly 1994;18(3):319–26.