CMM uncertainty analysis with factorial design

Precision Engineering 27 (2003) 283–288

CMM uncertainty analysis with factorial design Antonio Piratelli-Filho a,∗ , Benedito Di Giacomo b a

b

Department of Mechanical Engineering, Faculty of Technology, University of Bras´ılia, 70910-900 Bras´ılia, DF, Brazil Department of Mechanical Engineering, School of Engineering of São Carlos, University of São Paulo, 13560-970 São Carlos, SP, Brazil Received 1 February 2002; received in revised form 9 July 2002; accepted 13 January 2003

Abstract The purpose of this research is to present a method to estimate the Coordinate Measuring Machine (CMM) measurement uncertainty. The approach is based on a performance test using a ball bar gauge and a factorial design technique. A factorial design was applied to carry out a performance test and to investigate CMM errors associated to orientation and length in the work volume. The CMM measurement uncertainty was estimated with components of variance obtained after statistical analysis of variance applied to volumetric measurement errors. An application was performed with a Moving Bridge CMM and the results were compared to the volumetric performance test proposed by ANSI/ASME B89.4.1 standard. The results showed that the proposed method is suitable to investigate CMM hardware performance and determine the contribution of machine variables to measurement uncertainty. © 2003 Elsevier Science Inc. All rights reserved. Keywords: CMM performance test; Uncertainty; Ball bar gauge

1. Introduction The measurement uncertainty of the Coordinate Measuring Machine (CMM) has been determined by mathematical modeling, comparison, and performance test methods. The mathematical modeling method involves the calibration of CMM geometric errors to build a theoretical model and predict the uncertainty for any measurement done according to the model’s assumptions. A comparison method involves the inspection of a calibrated artifact identical to the piece under measurement and the measurement uncertainty statement has a specific application. Performance tests designate artifacts and a criterion to determine the measurement errors and a two-point length measurement uncertainty may be established as a performance parameter [1]. Performance tests are largely applied for inspection of parts and in the purchasing selection of a suitable CMM. According to known standards, gauges like a ball bar, a step gauge, and a ball plate have been used to determine the volumetric errors, inspect hardware and estimate the CMM measurement uncertainty. The best known standards are the American ANSI/ASME B89.4.1 [2], the German VDI/VDE 2617 [3], the British BS 6808 [4], the Japanese JIS B7440 [5], the ISO 10360 [6], and the manufacturers association CMMA [7]. ∗ Corresponding

author. E-mail address: [email protected] (A. Piratelli-Filho).

These performance test methods recommend different test methodologies and distinct ways of determining a performance parameter. Some standards indicate the maximum range of measured lengths as the performance parameter, and others establish the standard deviation of results. The measurement uncertainty may be established using information acquired by performance tests and may be carried out by A + B L-type statement, describing uncertainty of a two-point measurement. Other methodologies were developed to estimate task-specific uncertainties according to the ISO Guide to the Expression of Uncertainty in Measurement [8] and to an established inspection plan [9,10,11]. Due to the differences among recommended methodologies, CMMs comparison is possible when adopting the same standard. A standard largely accepted by users is the ANSI/ASME B89.4.1, as it describes tests in full details to investigate different CMM error categories [9]. These are a repeatability test, a linear displacement accuracy test, a volumetric performance test to evaluate CMM hardware, a bi-directional length test and a point-to-point probing performance test. Volumetric performance test is carried out using a low-cost artifact like an uncalibrated ball bar and it has the advantage of small time consumption. The ball bar length is measured in at least 20 different locations in the CMM work zone, and the results are presented in a graphic where the range of measured lengths is considered as a performance parameter [2].

0141-6359/03/$ – see front matter © 2003 Elsevier Science Inc. All rights reserved. doi:10.1016/S0141-6359(03)00035-7

284

A. Piratelli-Filho, B. Di Giacomo / Precision Engineering 27 (2003) 283–288

This work presents an approach to determine the CMM measurement uncertainty through a performance test. The approach consists of measuring calibrated ball bar gauges with different lengths, in distinct work volume orientations defined according to a selected factorial design. A statistical analysis of variance was implemented to improve the evaluation of the results, to estimate the measurement uncertainty as the performance parameter and to determine the contribution of variables to measurement uncertainty. A case study was performed using a Moving Bridge CMM at the Metrology Laboratory, University of São Paulo, Brazil.

Table 1 32 factorial design and experimental runs

2. Factorial design and CMM performance test

total number of runs. Thus, the position recommended to measure the gauge is in the center of the work volume and the ball bar gauge must be rigidly clamped by its center and placed over CMM table using a steel support. The probe configuration is a touch trigger with one short vertical stylus to minimize probe lobbing error effect. The sampling strategy is carried out touching seven equally spaced points on the ball surface to minimize the effect of probing repeatability. The center of the balls and the length between these points are determined by CMM software and the volumetric measurement errors are calculated by comparing the measured values to calibrated ones. After the study of the CMM variables that affect measurement errors, an experimental design is selected. A 32 factorial design is proposed to investigate two variables in three levels or values and their interactions. The chosen variables were ball bar length and ball bar orientation in work volume. Table 1 shows a codified array representing a 32 factorial design. In this array, A and B are the variables ball bar orientation and length, respectively. Numbers 1, 2, and 3 represent the variable levels or values adopted. The AB interaction is investigated to verify the influence of changing the levels of A and B simultaneously. Each line of the 32 design represents an individual run that may be replicated to estimate an experimental standard deviation. The levels adopted for orientation (A) were orientation along X-axis (level (1), XY diagonal (level 2) and XYZ diagonal (level 3) in work volume to check CMM geometric error influence. Fig. 1 shows these orientations in a nearly cubic CMM work volume. The levels of ball bar length (B) variable must be established according to the length of CMM axes and may be generically considered as small, medium, and large (levels 1, 2, and 3). A random sequence is recommended to carry out the experimental tests. Each run must be executed according to the combination of variables A and B as indicated in the design array. As shown in Table 1, the first run must be accomplished measuring the small ball bar length (variable B, level 1) oriented along X-axis (variable A, level 1). Each run must be replicated five times and thus 45 lengths are determined. The measurement errors are calculated by the difference between the measured and the calibrated ball bar length.

According to the performance test approach, the investigation of CMM hardware must be accomplished by controlling the position and orientation of an artifact in work volume. The ANSI/ASME B89.4.1 volumetric performance test recommends positions and orientations where geometric errors have the greatest effect on measurement errors [2]. Other performance tests suggest the control of orientation and length in work volume to determine performance. The investigation of the effect of length, position, and orientation in work volume on CMM measurement errors may be done using statistical techniques. The Design of experiments may be considered as a powerful tool to plan and execute experimental tests and analyze the results [12]. These techniques allow planned control of machine variables to induce the variability of the measurement errors, since calibrated gauges are measured and a statistical analysis of variance is recommended to handle the results. There are many different techniques to design an experiment. Some references in literature discuss nested [13] and Taguchi [14,15] experimental designs applied to determine uncertainty in measurement situations. The ISO Guide to the Expression of Uncertainty in Measurement [8] mentions the usefulness of analysis of variance techniques to determine type A measurement uncertainty. Montgomery [12] introduces a group of design of experiment techniques named factorial design in which experimental variables are investigated in all possible combinations of their levels or values and the results are analyzed by analysis of variance. Before selecting the design of the experiment and applying a performance test, it is necessary to consider the amount of time spent to perform experimental runs. Time consumption is closely related to the number of experimental variables, the number of levels of each variable investigated, and the amount of data required. In factorial design, the total number of runs (N) is determined using the expression N = (L)V , where L is the number of levels of each variable and V is the number of experimental variables investigated. As an example, when studying four variables with three levels each, the total number of runs is 34 and it is equal to 81 [12]. In the proposed method, some machine variables are considered as fixed parameters to simplify and minimize the

Runs

A

B

1 2 3 4 5 6 7 8 9

1 1 1 2 2 2 3 3 3

1 2 3 1 2 3 1 2 3

A. Piratelli-Filho, B. Di Giacomo / Precision Engineering 27 (2003) 283–288

   i = 1, . . . , a yijl = µ + τi + βj + (τβ)ij + εijl j = 1, . . . , b   l = 1, . . . , r

285

(2)

Admitting that the effect τ i has variance στ2 and is NID(0, the effect βj has variance σβ2 and is NID(0, σβ2 ), both independent of the residual error εijl , it can be showed that variance σy2 of response values (yijl ) is given by Eq. (3). The 2 , and σ 2 are variance components of the variances στ2 , σβ2 , στβ total variance σy2 , and this equation corresponds to the random effect model of the components of variance. στ2 ),

2 σy2 = στ2 + σβ2 + στβ + σ2

Fig. 1. Orientations adopted to carry out performance test.

An analysis of variance must be performed to study the variability of measurement errors produced by experimental design. The effect of the significant variables and its contribution to measurement uncertainty are investigated.

3. Measurement uncertainty analysis The analysis of variance works by separating the total variance of the results into partial variances associated to each source of variation. The variance partition is based on the numerator of the total variance expression, mentioned as the total sum of squares (SST), that is equal to a sum of two or more different sum of squares (SS) associated to experimental variables. As 32 factorial design investigates two variables, A and B, plus the interaction AB, the SST can be partitioned according to Eq. (1). This SST may be written as the sum of variable A, variable B, interaction AB, and the residual sum of squares (SSA, SSB, SSAB, and SSR) [12]. SST = SSA + SSB + SSAB + SSR

(1)

Since experimental variables may admit many levels, we can accept the hypothesis that the effects on experimental mean error have random nature and may be considered as random variables. This implies that changing the levels of A or B yields deviations into response values (measurement errors) in relation to the total mean µ whose significance can be verified applying the F test. Nevertheless, hypothesis tests concerning these effects may be applied since the random error component εijl presents a normal distribution with mean zero and variance σ 2 , for example, NID(0, σ 2 ). An addictive linear statistical model applied to 32 factorial design is shown in Eq. (2). In this model, yijl represents the response variable, for example, the measurement error. The τ i is ith level effect of the A variable, βj is the jth level effect of the B variable and (τβ)ij is the effect of the interaction between τ i and βj . The letters a and b are the number of levels of variables A and B, respectively, and r is the number of replicates.

(3)

The analysis of variance can be summarized in Table 2. The mean square (MS) may be considered as an estimate of the variance and calculated by SS divided by the respective degrees of freedom (DF). The F tests are performed to check the statistic significance of the variable and interaction effects and it is carried out comparing calculated and expected F values obtained in standard F tables at a given probability. It is necessary to point out that, as MS values were determined by using all measurement errors, each component of variance requires the compensation of the residual variation present in the determined MS. It may be shown that expected mean squares E(MS) may be explained as a function of variables, interaction, and residual components of variance, as given in Eqs. (4)–(7) [12]. 2 E(MSA) = σ 2 + rστβ + brστ2

(4)

2 E(MSB) = σ 2 + rστβ + arσβ2

(5)

2 E(MSAB) = σ 2 + rστβ

(6)

E(MSR) = σ 2

(7)

As measurement uncertainty is determined by a standard deviation [8], the square-root of the components of variance σ τ , σ β , and σ τβ may be considered as standard uncertainties uA , uB , and uAB and residual σ as residual uncertainty uR . As given in Eq. (7), the residual variance σ 2 is equivalent to the expected residual mean square (MSR) and thus, the squared-root of the MSR value determined in the analysis of variance is the residual uncertainty. Therefore, we can rewrite Eqs. (4)–(6) as showed in Eqs. (8)–(10) and determine the length measurement uncertainty associated to variables A(uA ) and B(uB ), and interaction AB (uAB ).   MSA − MSAB 1/2 (8) uA = br   MSB − MSAB 1/2 (9) uB = ar   MSAB − MSR 1/2 (10) uAB = r

286

A. Piratelli-Filho, B. Di Giacomo / Precision Engineering 27 (2003) 283–288

Table 2 Analysis of variance—32 factorial design Source of variation (SV)

Sum of squares (SS)

Degrees of freedom (DF)

Mean squares (MS)

Calculated F

SSA MSA = DFA SSB MSB = DFB SSAB MSAB = DFAB SSR MSR = DFR

MSA MSAB MSB FB = MSAB MSAB FAB = MSR

A

SSA

a−1

B

SSB

b−1

AB

SSAB

(a − 1)(b − 1)

Residual

SSR

ab(r − 1)

Total

SST

abr − 1

FA =

Following ISO recommendations to determine uncertainty [8], we can calculate combined measurement uncertainty uc and expanded measurement uncertainty U using these results. The combined standard uncertainty may be estimated by root-sum-of-squares of components of variance or standard uncertainties, uA , uB , uAB , and uR . Expanded uncertainty U may be estimated by multiplying standard uncertainty uc by a coverage factor k, adopting 95% confidence and υ effective degrees of freedom. Effective degrees of freedom may be determined by using the Welch–Satterthwaite formula [8]. These values of two-point measurement uncertainties are recommended as the CMM performance parameters. Fig. 3. Measurement errors.

4. Results An experimental application was performed at the Metrology Laboratory, University of São Paulo, Brazil, using a Moving Bridge CMM. This machine has three independent axes with operational range of 356 mm (X), 406 mm (Y), and 305 mm (Z). Three ball bars were calibrated and Fig. 2 shows the determined parameters. The length L0 and sphere diameters D1 and D2 were measured by using a SIP Universal Measuring Machine and the standard length L was calculated by using CMM software. The lengths between ball centers are 131.3499, 197.4931, and 269.1468 mm and the combined standard uncertainties are 0.0008, 0.0013, and 0.0017 mm, respectively.

Fig. 2. Ball bar gauge parameters.

The ball bar lengths were measured according to experimental design and measurement errors were calculated and plotted in a scattering graph. As shown in Fig. 3, the smallest measurement errors were observed at run 6, when measuring 269.1468 mm ball bar length in the XY direction. The larger measurement error was observed at run 8, when measuring 197.4931 mm ball bar length in the XYZ direction. Figs. 4 and 5 show measurement errors as a function of ball bar orientation and length, respectively. Small differences in mean error were observed in each graph, as orientation or length was modified and a large dispersion in results was perceived. Sample statistics were determined and showed that the range is 40 ␮m, the mean is 19 ␮m and the standard deviation

Fig. 4. Measurement errors at different ball bar orientations in work volume.

A. Piratelli-Filho, B. Di Giacomo / Precision Engineering 27 (2003) 283–288

Fig. 6. Normal probability plot of measurement errors.

Fig. 5. Measurement errors of different ball bar lengths.

Table 3 Analysis of variance results SV

DF

Orientation (A) Length (B) AB Residual

2 2 4 36

Total

44

MS

784.62 398.49 253.82 13.40

F calculated

F expected

3.09 1.57 18.94

287

95%

99%

6.94 6.94 2.68

18.00 18.00 4.01

is 9.4 ␮m. Before performing the analysis of variance, the results were plotted into a normal probability paper to verify the normality assumption. As shown in Fig. 6, this hypothesis can not be rejected since all data relies near a straight line. An analysis of variance was applied on measurement errors using Statistica software of StatSoft Inc. Determined MS and F values can be observed in Table 3. A sharp contribution of interaction between orientation (A) and length (B) in mean measurement error was observed and this effect was statistically significant at 99% confidence level according to the F test. This means that the simultaneous modification of orientation and length leads to a large variation in mean measurement error and this was associated to CMM squareness errors. The ANSI/ASME B89.4.1 volumetric performance test was performed using a 197.4931 mm ball bar length, posi-

tioned in 18 positions in work zone [2] and using the same probe configuration as done in the proposed test. It was observed that the machine performance parameter is 65 ␮m and that it corresponds to the data range. As pointed out by literature [12], a standard deviation is a better statistical parameter when dealing with small samples, and it was determined with test results as 19 ␮m. These performance values were larger than the sample statistics determined in the proposed test and it may be a consequence of positions adopted near borders of CMM work zone. However, a preliminary test using Taguchi L9 array showed that changing gauge positions along X, Y, and Z axis did not result in significant modification in mean error [15]. Instead of factorial 32 , another design of experiment may be used if the position must be considered, such as Taguchi L9 portraying the same number of runs. Table 4 shows the results of uncertainty analysis determined by the proposed method. The standard uncertainties of A and B variables were determined according to Eqs. (8) and (9) and the standard uncertainty of AB interaction was determined according to Eq. (10). The standard uncertainties attributed to orientation (A) is 6 ␮m, length (B) is 3 ␮m, and interaction (AB) is 7 ␮m. The residual standard uncertainty is 4 ␮m and it may be allocated to effects of other variables as gauge length uncertainty, sampling uncertainty, position in work volume, and temperature effects.

Table 4 CMM measurement uncertainty analysis

Source of uncertainty Orientation (A) Length (B) Interaction (AB) Residual Measurement uncertainty Combined standard uncertainty uc = 10 ␮m Effective degrees of freedom υeff = 24 Coverage factor (υeff , 95%) k = 2.064 Expanded uncertainity (95%) U = kuc = 20 ␮m

Uncertainty type

Symbol

Probability distribution

Sensitivity coefficient

Degrees of freedom

Standard uncertainty (␮m)

A A A A

uA uB uAB uR

Normal Normal Normal Normal

1 1 1 1

2 2 4 36

6 3 7 4

288

A. Piratelli-Filho, B. Di Giacomo / Precision Engineering 27 (2003) 283–288

The two-point length measurement uncertainty was established as performance parameter. Thus, a combined standard uncertainty was determined as given in Eq. (3) and it is 10 ␮m. CMM expanded uncertainty is 20 ␮m (95%), as shown in Table 4, and it was determined by using a coverage factor obtained from t Student table, with 24 degrees of freedom and 95% confidence. Boundaries can be determined calculating mean measurement error (19 ␮m) plus expanded uncertainty and mean measurement error minus expanded uncertainty, which holds almost all measurement errors observed, as shown in Fig. 3. Only two measurement errors located out of these boundaries were observed, within 45 measurement errors determined, resulting in a probability of 4.4 %.

5. Conclusions A method was proposed to estimate CMM measurement uncertainty using statistical methods like the design of experiments and the analysis of variance. An application performed on a Moving Bridge CMM showed an increment of information on CMM performance when compared to another method. The additional information was obtained by determining the effects of variables on mean measurement error and estimating a two-point length measurement uncertainty. The analysis of variance results showed a strong interaction between the orientation and measured length. It means that a significant variation in mean measurement error was produced when changing the levels of these variables simultaneously and it was associated to machine squareness errors. The smallest CMM measurement errors are expected when measuring lengths about 270 mm in the XY direction and the largest measurement errors are expected when measuring lengths about 200 mm in the XYZ direction. A measurement uncertainty analysis using components of variance turn out to be suitable estimators of the CMM two-point length measurement uncertainty and it is appropriate when type A sources of uncertainty are predominant. The machine standard uncertainties associated to orientation, length, and orientation–length interaction were calculated as 6, 3, and 7 ␮m, respectively. Following the ISO statements, the CMM combined standard uncertainty was calculated 20 ␮m and the expanded uncertainty was calculated 10 ␮m. The automation of the experimental procedure can be implemented to reduce the time spent in measuring a ball bar length and software may be developed to control and compute the test results. Future work dealing with other de-

sign of experiment, inserting additional variables with more than three levels, can be accomplished on another CMM configuration. Acknowledgments The authors wish to gratefully acknowledge the support of Brazilian Financing Agencies Coordenação para Aperfeiçoamento de Pessoal (CAPES) and Conselho Nacional de Desenvolvimento Cient´ıfico e Tecnológico (CNPq). References [1] Bosch JA. Coordinate measuring machines and systems. New York: Marcel Dekker; 1995. p. 444. [2] ANSI/ASME B89.4.1. Methods for performance evaluation of coordinate measuring machines. New York: American Society of Mechanical Engineers; 1997. [3] VDI/VDE 2617. Accuracy of coordinate measuring machines, parts 1–4. Düsseldorf: Verein Deutscher Ingenieure (VDI/VDE); 1989. [4] BS 6808. British standard—coordinate measuring machines, parts 1–3. London: British Standards Institute; 1987. [5] JIS B7440. Japanese industrial standard. Test code for accuracy of coordinate measuring machines. Tokyo: Japanese Standards Association; 1987. [6] ISO 10360. Coordinate metrology, parts 1–2. Geneva: International Organization for Standardization (ISO); 1994. [7] CMMA. Accuracy specification for coordinate measuring machines. London: Coordinate Measuring Machine Manufacturers Association. [8] ISO. Guide to the Expression of Uncertainty in Measurement (GUM). Geneva: International Organization for Standardization (ISO); 1995. [9] Wilhelm RG, Hocken R, Schwenke H. Task specific uncertainty in coordinate measurement. Ann CIRP 2001;50(1). [10] Schwenke H, Siebert BRL, Waldele F, Kunzmann H. Assessment of uncertainties in dimensional metrology by Monte Carlo simulation: proposal of a modular and visual software. Ann CIRP 2000;49(1): 395–8. [11] Balsamo A, Di Ciommo M, Mugno R, Rebaglia BI, Ricci E, Grella R. Evaluation of CMM uncertainty through Monte Carlo simulations. Ann CIRP 1999;48/1:425–8. [12] Montgomery DC. Design and analysis of experiments. New York: Wiley; 1991. [13] Henrion A. Quantification of variance components in amount-ofsubstance measurements by the use of nested experimental design. In: Proceedings of 9th Metrologie Conference, Bordeaux, France; 1999. [14] Koike M, Ishida H. Estimation of measurement uncertainty of Rockwell Hardness test using orthogonal array. In: Proceedings of XV IMEKO World Congress, Osaka, Japan; 1999 June 21–27. [15] Piratelli-Filho A, Giacomo BD. Application of design of experiment techniques to estimate CMM measurement uncertainty. In: Proceedings of American Society for Precision Engineering—ASPE Annual Meeting, Scottsdale, AZ, USA; 2000.