- Email: [email protected]
Validation and quality control schemes based on the expression of results with uncertainty
Analytica Chimica Acta 393 (1999) 167±175
Validation and quality control schemes based on the expression of results with uncertainty Ricardo J.N. Bettencourt da Silvaa, M. Filomena G.F.C. CamoÄesa,*, JoaÄo Seabra e Barrosb a
CECUL, Faculdade de CieÃncias da Universidade de Lisboa, 1700 Lisbon, Portugal Instituto Nacional de Engenharia e Tecnologia Industrial, Estrada do Pac,o do Lumiar, 1699 Lisbon Codex, Portugal
b
Received 2 October 1998; received in revised form 23 March 1999; accepted 25 March 1999
Abstract The bottom-up approach for expression of results with uncertainty requires validation and quality control in agreement with the principles established in the Eurachem Guide, the chemical version of the ISO Guide. These aim at the normalisation of the application of well-known metrological principles such as random error propagation laws. This work presents a validation scheme which evaluates the uncertainty estimation process and uses that tool to test accuracy. The routine work of quality control is based on the comparison of obtained con®dence intervals for control standards with expected values, instead of using control charts that enhance the dispersion of experimental results. The present approach is also suitable for non-routine work because of its independence in relation to previous data and is applied to the determination of manganese by electrothermal atomic absorption spectrometry, of zinc by ¯ame atomic absorption spectrometry and of sodium by ¯ame atomic emission spectrometry, in lettuce leaves digested with nitric acid in a microwave irradiated closed system. Although the presented methods had different precisions they were considered valid after the application of the presented scheme. # 1999 Elsevier Science B.V. All rights reserved. Keywords: Validation; Quality control; Uncertainty; Atomic spectrometry
1. Introduction Comparability of results is achieved when all of them are traceable back to some reference. The SI units are, by nature, the targets. Analytical measurements can be performed by very sophisticated instrumental techniques or by classical gravimetric and volumetric procedures, all producing results in¯uenced by random errors. Therefore, results *Corresponding author. Tel.: +351-1-3906138; fax: +351-13909352; e-mail: [email protected]
should be presented as con®dence intervals in order to describe that natural dispersion. The correct estimation of con®dence interval parameters (mean value and interval width) is a challenge only answered after knowledge of the method's accuracy and precision. Precision can be studied from the experimental dispersion of replicate analysis or from the combination of information about the precision of unit operations. The last approach, the bottom-up approach, was encouraged by the ISO Guide [1]. The speci®cities of the chemical problems promoted the development of a guide dedicated to analytical measurements [2]. Its
0003-2670/99/$ ± see front matter # 1999 Elsevier Science B.V. All rights reserved. PII: S 0 0 0 3 - 2 6 7 0 ( 9 9 ) 0 0 3 3 7 - 2
168
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
application has the advantage of estimating analytical step precisions, avoiding excessive dependence on expensive and time-consuming replicate analysis. The knowledge of the dispersion of results has no use if they do not fall near the assigned `true value'. The dispersion of the method should ideally explain the differences between obtained and expected or known sample content. Deviation can be solved through systematic error correction or elimination. In this work, we present a validation and quality control scheme involving the ISO/Eurachem Guide principles. The estimation of result uncertainty is validated and used in the evaluation of method accuracy and in routine analysis control. The presented scheme was applied to the determination of metal in lettuce leaves by atomic spectrometry. The digested samples were characterised in terms of manganese, zinc and sodium contents through electrothermic atomic absorption spectrometry (ETAAS), ¯ame atomic absorption spectrometry (FAAS) and ¯ame atomic emission spectrometry (FAES), respectively.
The proposed validation scheme is to be applied after the optimisation of both the instrumental method and other steps, such as digestion, dilution and sample weighing and will be divided in two parts, the validation of the mathematical model for calibration, and precision and accuracy tests. Optimisation can be performed with either sophisticated experimental design techniques or by using simpler models. The ®rst part aims at supporting the choice of the mathematical model used to describe the instrument calibration plot. This step is performed on the instrument calibration independently of the other method components. Therefore, those conclusions may be useful for different methods that use the same quanti®cation system. The precision and accuracy tests are developed for the method under study (Fig. 1). The aim of the precision test is to validate the quanti®cation of the uncertainty of the analytical method [3]. The application of the Eurachem Guide is not straightforward and the scheme or approximations used must be
Fig. 1. The validation of the mathematical model of the calibration graph is performed on the instrumental method and the precision and accuracy tests are performed on the analytical method.
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
supported by comparison of theory with experimental data [3]. The accuracy test is ®nally performed after the validation of uncertainty calculations on reference materials or through sample spiking. The known samples or spiked samples are analysed and the results are presented with estimated uncertainties. Results are then compared with expected results, for accuracy evaluation. 2. Validation of the calibration model Calibration graphs corresponding to a large number of degrees of freedom [4] were studied in terms of outliers (Grubbs single [5,6] and paired [7] outliers tests) and homoscedasticity (F-test [8]). The selected models were evaluated through lack of ®t tests (ANOVA [9] for linear or quadratic unweighted models). The inadequacy of such models was overcome by the use of narrow concentration ranges. All tests were performed at a 95% signi®cance level. 3. Precision and accuracy tests Precision and accuracy tests were performed with the validated standard concentrations after the equipment calibration. The number of replicate readings was reduced to be the same as for routine analyses in order to make its daily use more friendly. Fig. 2 shows the integration of precision and accuracy tests in the proposed validation scheme. 3.1. Precision test The precision test is performed before the accuracy test and is based on the study of the measurement uncertainties [1], designated by the half width of the con®dence interval associated with the best estimate of the result. If sampling is not a target of the study, the sample analysed for the repeatability test must be homogeneous. The ISO Guide for expression of results with uncertainty [1] represents a major step in the harmonisation of criteria for the application of well-known error propagation laws, but there are some analytical steps that are dif®cult to evaluate in terms of their
169
relevance to the ®nal result dispersion [3]. Some of them can be estimated based on the analyst's experience. Nevertheless, it is easily understood that some disagreement between different approaches is inevitable. The solution for overcoming some bad estimation or approximation is the confrontation of the estimation with what is simulated, i.e. repeatability tests (Fig. 2). The comparison of estimated and experimental variances through an F-test can implement this idea. For the calculation of the expanded uncertainty [2] the number of degrees of freedom should be estimated by the application of Welch± Satterwaite formula [2,6,10]. If the variances are signi®cantly different (for at least a 95% signi®cance level) estimation should be reassessed. An experimental variance higher than that estimated indicates the presence of outliers easily detectable by an adequate statistical test or an important source of uncertainty which was badly or even not taken into account. For the opposite case, overestimation is the problem. When there is agreement between the two approaches there are no reasons to think that the estimation may be wrong; therefore, it must be trusted until there are other indications, such as those that will be approached in the quality control section. The presented validation of the uncertainty uses repeatability tests not only as a tool for estimating the precision of a single quanti®cation, but also to extend the estimation process to the evaluation of daily results dispersion. This approach for the presentation of results with a measure of its quality simpli®es routine work by avoiding extensive use of replicate analysis, presenting results with a measure of its quality. The detection of sample contamination, sample projections or other gross errors may continue to ask for replicate analyses although in smaller numbers. 3.2. Accuracy test The precision test does not give information on the accuracy of the method, which must be tested by an adequate procedure. The accuracy test (Fig. 2) is performed through the use of reference materials, when they are available, or by sample spiking. Ideally, the reference material's certi®ed results should be represented by con®dence intervals for known con®dence levels. When that
170
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
Fig. 2. Validation scheme. s2, variance; , degrees of freedom; best estimation of the result; U, uncertainty; s.l., significance level; c, certified value; obs., observed value.
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
information is incomplete, the Eurachem Guide approach should be used to transform the information in a way comparable to that obtained in the laboratory. If there is overlapping of the certi®ed and obtained con®dence intervals there is no reason to suspect that the method is not as accurate as the one used for the reference material certi®cation and it can be said that the method is accurate and valid for the determination under concern. In spiking assays, sample addition should produce a con®dence interval containing the expected value (see following sections). Such conclusion is only characteristic of the speci®ed equipment, operator and laboratory. Extrapolation to other conditions should be tested. 4. Quality control The validation of an analytical method does not necessarily imply that all analyses performed by that method and tested instrumentation produce valid results. A routine analysis must be performed with a quality control scheme in order to detect deviations produced by equipment, analyst or experimental procedure and guarantee the extrapolation of validation conclusions to the daily work. The daily performance of the analytical method can be tested through the analysis of a reference material. Secondary, home-made reference materials can also be certi®ed by the simultaneous validated analysis of a primary one. This approach allows for a detailed and frequent quality control without the need for large amounts of expensive reference materials. When an analytical method includes an instrumental procedure, control standards can be used in order to verify instrumental behaviour. Ideally, standards and samples should be presented randomly [3,11±13] to instruments. However, the instrument software normally separates calibration from sample analysis allowing immediate obtaining of sample contents. Sensitivity drifting should be controlled to avoid lack of accuracy due to the use of outdated calibrations. The periodic analysis of control standards certi®es the quality of sample signal interpolation. In this work, the comparison of expected and obtained values was based on the expression of results with uncertainty instead of the use of control charts.
171
Interpolation uncertainties behaved as a quality control criterion adjustable to the daily instrumental response stability, avoiding the disadvantage of using information from several days. Control charts can estimate, by excess or defect, daily instrumental response precision preventing the detection of signal drifting or the validation of low precision results, respectively. The use of results with a measure of their uncertainty based on ISO/Eurachem Guide principles, as presented, represents an approach of the analytical work different from that of control charts. These principles are also valid for global method quality control, ideally supported by the analysis of reference materials. The analyst needs to conjugate the various control tools that care for the instrumental method performance in order to avoid the extensive use of expensive certi®ed reference materials (CRM) [14]. 5. Application of the validation scheme to atomic spectrometry The above proposed validation scheme was applied to the determination of metals in lettuce leaves by atomic spectrometry. The results refer to a dry base, the dry content determination being parallel to the metal content quanti®cation. The samples were digested in a microwave irradiated closed system in the presence of nitric acid. The instrumental methods used for the quanti®cation step were electrothermal atomic absorption spectrometry (ETAAS) for manganese, ¯ame atomic absorption spectrometry (FAAS) for zinc and ¯ame atomic emission spectrometry for sodium (FAES). Description of the validation process is as follows. 5.1. Validation of the calibration model 5.1.1. Determination of Mn by ETAAS The instrumental quanti®cation was performed on a GBC 906 AA spectrometer with D2 lamp background correction at 279.5 nm. The graphite furnace was equipped with a L'vov platform. The drying stage was performed at two stages, respectively, 1308 and 2008C, ashing was done at 10008C, atomisation took place at 20008C and, ®nally, the cleaning stage was
172
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
effective at 26008C [15,16]. A mixture of palladium nitrate/magnesium nitrate (1.0 g lÿl/1.5 g lÿ1) was used as a matrix modi®er. The instrumental system behaved linearly and with constant variance over the calibration range 0±2±3±4± 5±6 mg lÿ1. Therefore, the linear unweighted model [17] was applied to the characterisation of the calibration graph. 5.1.2. Determination of Zn by FAAS The spectrometric determinations were done with the same instrument as above in the ¯ame mode (airacetylene) using the same background correction, and studying the absorption line at 213.9 nm. The instrument produced a response over the range 0±0.05±0.1± 0.15±0.2±0.25 mg lÿ1 satisfactorily described by the second degree unweighted model [8]. 5.1.3. Determination of Na by FAES The determination of sodium was implemented with the same instrumentation as previously but in the emission mode, at 589.0 nm. The ¯ame was airacetylene. The response ®tted the unweighted second degree model for the concentration range 2±4±5±6±8± 10 mg lÿ1, zero having to be excluded in order to obtain the adjustment. That choice changed a normal calibration graph to more like second degree bracketing. This approach loses the advantage of normal linear bracketing but takes the advantage of a broader concentration range. The blank solution was spiked at 2 mg Na lÿ1 in order to ®nd its in¯uence on the sample result. 5.2. Precision test A detailed description of the precision test uncertainty estimation is available in the literature [3]. 5.2.1. Determination of Mn by ETAAS The manganese repeatability test produced experimental results with two consecutive Grubbs test single outliers. The rejection of those values proved that the experimental standard deviation (0.82 mg kgÿ1 for 98 of freedom, df) was statistically equivalent to that estimated (0.73 mg kgÿ1 for 57 500 df). This conclusion is supported by an F-test using the same signi®cance level (95%) as for the other statistical tests performed.
The average relative expanded uncertainty, AREU, i.e. average expanded uncertainty divided by the average best estimation (middle value) obtained in the precision test was 3.2%. 5.2.2. Determination of Zn by FAAS The determination of zinc in lettuce leaves produced unsatisfactory results. The experimental dispersion (standard deviation, SD: 1.86 mg kgÿ1 for 7 df) was statistically different from that estimated (SD: 0.67 mg kgÿ1 for 5900 df). The success of the precision test in other systems, the detail of the uncertainty estimations and the results of the accuracy test supported the hypothesis of heterogeneity of the test sample with respect to zinc. The in¯uence of eventual contamination, frequent for low levels of zinc, was not observed in the accuracy test, as will be shown, suggesting the trueness of the hypothesis. The AREU for this determination was 3.2%. 5.2.3. Determination of Na by FAES Fig. 3 represents the results obtained for the sodium repeatability test. Results are presented as con®dence intervals with the best estimate (middle point) being associated with expanded uncertainty estimations [2]. Overlapping of all the con®dence intervals is observed, supporting the suggestion that they represent the same sample. The standard deviation associated with the middle points of that con®dence interval (experimental SD) was 0.032% (w/w) with 11 df. The estimated SD following the Eurachem Guide (represented in Fig. 3 in the expanded formÐcoverage factor of 1.96) was 0.049% for 1 1010 df. These two SD values are statistically equivalent at the 95% con®dence level. The test sample precision results obtained had an AREU of 3.2%. 5.3. Accuracy test After the validation of the uncertainty estimation, the accuracy of the method was evaluated through a speci®c test based on the analysis of a certi®ed reference material, CRM, (NIST 1972a). This is a metal content certi®ed spinach leaves CRM which has a proven similarity with lettuce leaves in terms of the proportion of proteins, ®bre, minerals and carbohydrates [18].
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
173
Fig. 3. Precision test for the determination of the sodium content of lettuce leaves.
Alternative sample spiking tests were performed. 5.3.1. Determination of Mn by ETAAS The analysis, in three replicates, of NIST 1972a produced results (best estimations and estimated expanded uncertainty con®dence intervals) equivalent to the 95% con®dence level certi®ed con®dence intervals. The overlapping of those four intervals indicates that they could represent measurements of the same entity. It can also be seen that the ETAAS based method used in this work was less precise than the one used by NIST. This observation is easily understood taking into account the bad reputation of the precision of the applied technique as well as the well-known precision of laser-excited atomic ¯uorescence spectrometry and
instrumental neutron activation analysis used by NIST [19]. 5.3.2. Determination of Zn by FAAS The accuracy test proved to be decisive for the evaluation of the method. The overlapping of the expected value (certi®ed con®dence interval) with three replicate results proved that there were no contamination problems from the procedure (Fig. 4). The homogeneity of the CRM appeared satisfactory for the ful®lment of that criterion. The precision of the FAAS method was very similar to that shown by the certi®ed con®dence interval width. Such observation is in agreement with the known high precision of this quanti®cation step.
Fig. 4. Accuracy test for the determination of the zinc content of lettuce leaves.
174
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
Fig. 5. Accuracy test for the determination of the sodium content of lettuce leaves.
5.3.3. Determination of Na by FAES The accuracy test for sodium was also successful. In order to suggest the recommended procedure for the cases where adequate CRM is not available, spiking assays were performed. The interpolation uncertainty estimation and regression were performed on the assumption that the uncertainty associated with the preparation of standards was negligible when compared with the instrumental reading component [17]. Therefore, the spiking of samples, performed in the same way as the preparation of standards, also has negligible uncertainty. Hence, the observed con®dence intervals (middle point and expanded uncertainty) for a spiked sample should overlap the observed interval for the non-spiked sample plus the added content. This addition is performed at the interval middle point maintaining the interval width. Fig. 5 represents the comparison of the observed and expected values (sample result plus addition) for 4 mg lÿ1 spiking of a sample. There is a clear overlap of the two intervals, supporting the success of the addition. 5.4. Quality control Routine analysis quality control was performed through the analysis of the CRM, after the calibration, validating the sample preparation and through readings of control standards equivalent to calibration standards for the purpose of testing samples signal
interpolation. Readings were taken with control standards after each group of ®ve samples excluding blanks and duplicates validating previous determinations. The routine quanti®cation of manganese (ETAAS), zinc (FAAS) and sodium (FAES) samples content complied with the quality control criterion for 30± 40, 70±85, 55±65 samples, respectively, without exception. Further determinations were performed after re-calibration of the instrument. 6. Conclusions The presented analytical methods, performing at different precisions, were considered valid by the validation scheme. The estimated uncertainties describe adequately the experimental dispersion of results supporting the accuracy of the methods. The estimation of obtained results uncertainty for the reference materials and interpolation uncertainty for the control standards proved to be an adequate tool to detect routine analysis deviations. The proposed quality control scheme was applied without the need for the elaboration of control charts. The independence of this approach from previous data makes it adequate for non-routine work, for the use of various concentration ranges in heteroscedastic instrumental responses and when daily method precision varies due to lack of instrumental stability.
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
Acknowledgements Thanks are due to JNICT for ®nancial support, to CONTROLAB LDA for the instrumental facilities as well as to the teams of INETI and CONTROLAB LDA for their co-operative attitude. References [1] International Organisation for Standardisation, ISO, Guide to the Expression of Uncertainty in Measurement, Switzerland, 1993. [2] Eurachem, Quantifying Uncertainty in Analytical Measurement, Version 6, 1995. [3] R.J.N.B. Silva, M.F.G.F.C. CamoÄes, J.S. Barros, Accreditation and Quality Assurance 3 (1988) 155±160. [4] W. Penninckx, C. Hartmann, D.L. Massart, J. SmeyersVerbeke, J. Anal. At. Spectrom. 11 (1996) 237±246. [5] F.E. Grubbs, G. Beck, Technometrics 14 (1972) 847±854. [6] J.K. Taylor, Quality Assurance of Chemical Measurement, Version 5, 1994. [7] P.C. Kelly, J. Assoc. Off. Anal. Chem. 73 (1990) 58±64. [8] International Organisation for Standardisation, ISO International Standard 8466-2 Water Quality, Calibration and
[9] [10] [11] [12] [13] [14] [15] [16] [17]
[18] [19]
175
Evaluation of Analytical Methods and Estimation of Performance Characteristics. Part 2: Calibration Strategy for Nonlinear Second Order Function Geneva, 1993. D.L. Massart, B.G.M. Vandeginste, S.N. Deming, Y. Michotte, L. Kaufman, Chemometrics: a textbook, Elsevier, Amsterdam, 1988. J. Kargten, Analyst 119 (1994) 2161±2165. G. Staas, Fresenius J. Anal. Chem. 352 (1995) 413±419. J.K. Taylor, Anal. Chem. 55 (1983) 600A±608A. Analytical Methods Committee, Analyst, 119 (1994) 2363± 2366. R.J.N.B. Silva, M.F.G.F.C. CamoÄes, submitted to publication. R. Blust, A. Van der Linden, W. Decleir, At. Spectrosc. 6 (1985) 163±165. R. Chakraborty, A.K. Das, M.L. Cervera, M. de la Guardia, J. Anal. At. Spectrom. 10 (1995) 353±358. International Organisation for Standardisation, ISO International Standard 8466-2 Water Quality, Calibration and Evaluation of Analytical Methods and Estimation of Performance Characteristics. Part 1: Calibration Strategy for linear Function Geneva, 1993. W. Penninckx, J. Smeyers-Verbeke, P. Vankeerberghen, D.L. Massart, Anal. Chem. 68 (1996) 481±489. National Institute of Standards and Technology NIST, Certificate of Analysis, Standard Reference Material 1570a, Trace Elements In Spinach Leaves, USA, 1994.
Validation and quality control schemes based on the expression of results with uncertainty Ricardo J.N. Bettencourt da Silvaa, M. Filomena G.F.C. CamoÄesa,*, JoaÄo Seabra e Barrosb a
CECUL, Faculdade de CieÃncias da Universidade de Lisboa, 1700 Lisbon, Portugal Instituto Nacional de Engenharia e Tecnologia Industrial, Estrada do Pac,o do Lumiar, 1699 Lisbon Codex, Portugal
b
Received 2 October 1998; received in revised form 23 March 1999; accepted 25 March 1999
Abstract The bottom-up approach for expression of results with uncertainty requires validation and quality control in agreement with the principles established in the Eurachem Guide, the chemical version of the ISO Guide. These aim at the normalisation of the application of well-known metrological principles such as random error propagation laws. This work presents a validation scheme which evaluates the uncertainty estimation process and uses that tool to test accuracy. The routine work of quality control is based on the comparison of obtained con®dence intervals for control standards with expected values, instead of using control charts that enhance the dispersion of experimental results. The present approach is also suitable for non-routine work because of its independence in relation to previous data and is applied to the determination of manganese by electrothermal atomic absorption spectrometry, of zinc by ¯ame atomic absorption spectrometry and of sodium by ¯ame atomic emission spectrometry, in lettuce leaves digested with nitric acid in a microwave irradiated closed system. Although the presented methods had different precisions they were considered valid after the application of the presented scheme. # 1999 Elsevier Science B.V. All rights reserved. Keywords: Validation; Quality control; Uncertainty; Atomic spectrometry
1. Introduction Comparability of results is achieved when all of them are traceable back to some reference. The SI units are, by nature, the targets. Analytical measurements can be performed by very sophisticated instrumental techniques or by classical gravimetric and volumetric procedures, all producing results in¯uenced by random errors. Therefore, results *Corresponding author. Tel.: +351-1-3906138; fax: +351-13909352; e-mail: [email protected]
should be presented as con®dence intervals in order to describe that natural dispersion. The correct estimation of con®dence interval parameters (mean value and interval width) is a challenge only answered after knowledge of the method's accuracy and precision. Precision can be studied from the experimental dispersion of replicate analysis or from the combination of information about the precision of unit operations. The last approach, the bottom-up approach, was encouraged by the ISO Guide [1]. The speci®cities of the chemical problems promoted the development of a guide dedicated to analytical measurements [2]. Its
0003-2670/99/$ ± see front matter # 1999 Elsevier Science B.V. All rights reserved. PII: S 0 0 0 3 - 2 6 7 0 ( 9 9 ) 0 0 3 3 7 - 2
168
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
application has the advantage of estimating analytical step precisions, avoiding excessive dependence on expensive and time-consuming replicate analysis. The knowledge of the dispersion of results has no use if they do not fall near the assigned `true value'. The dispersion of the method should ideally explain the differences between obtained and expected or known sample content. Deviation can be solved through systematic error correction or elimination. In this work, we present a validation and quality control scheme involving the ISO/Eurachem Guide principles. The estimation of result uncertainty is validated and used in the evaluation of method accuracy and in routine analysis control. The presented scheme was applied to the determination of metal in lettuce leaves by atomic spectrometry. The digested samples were characterised in terms of manganese, zinc and sodium contents through electrothermic atomic absorption spectrometry (ETAAS), ¯ame atomic absorption spectrometry (FAAS) and ¯ame atomic emission spectrometry (FAES), respectively.
The proposed validation scheme is to be applied after the optimisation of both the instrumental method and other steps, such as digestion, dilution and sample weighing and will be divided in two parts, the validation of the mathematical model for calibration, and precision and accuracy tests. Optimisation can be performed with either sophisticated experimental design techniques or by using simpler models. The ®rst part aims at supporting the choice of the mathematical model used to describe the instrument calibration plot. This step is performed on the instrument calibration independently of the other method components. Therefore, those conclusions may be useful for different methods that use the same quanti®cation system. The precision and accuracy tests are developed for the method under study (Fig. 1). The aim of the precision test is to validate the quanti®cation of the uncertainty of the analytical method [3]. The application of the Eurachem Guide is not straightforward and the scheme or approximations used must be
Fig. 1. The validation of the mathematical model of the calibration graph is performed on the instrumental method and the precision and accuracy tests are performed on the analytical method.
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
supported by comparison of theory with experimental data [3]. The accuracy test is ®nally performed after the validation of uncertainty calculations on reference materials or through sample spiking. The known samples or spiked samples are analysed and the results are presented with estimated uncertainties. Results are then compared with expected results, for accuracy evaluation. 2. Validation of the calibration model Calibration graphs corresponding to a large number of degrees of freedom [4] were studied in terms of outliers (Grubbs single [5,6] and paired [7] outliers tests) and homoscedasticity (F-test [8]). The selected models were evaluated through lack of ®t tests (ANOVA [9] for linear or quadratic unweighted models). The inadequacy of such models was overcome by the use of narrow concentration ranges. All tests were performed at a 95% signi®cance level. 3. Precision and accuracy tests Precision and accuracy tests were performed with the validated standard concentrations after the equipment calibration. The number of replicate readings was reduced to be the same as for routine analyses in order to make its daily use more friendly. Fig. 2 shows the integration of precision and accuracy tests in the proposed validation scheme. 3.1. Precision test The precision test is performed before the accuracy test and is based on the study of the measurement uncertainties [1], designated by the half width of the con®dence interval associated with the best estimate of the result. If sampling is not a target of the study, the sample analysed for the repeatability test must be homogeneous. The ISO Guide for expression of results with uncertainty [1] represents a major step in the harmonisation of criteria for the application of well-known error propagation laws, but there are some analytical steps that are dif®cult to evaluate in terms of their
169
relevance to the ®nal result dispersion [3]. Some of them can be estimated based on the analyst's experience. Nevertheless, it is easily understood that some disagreement between different approaches is inevitable. The solution for overcoming some bad estimation or approximation is the confrontation of the estimation with what is simulated, i.e. repeatability tests (Fig. 2). The comparison of estimated and experimental variances through an F-test can implement this idea. For the calculation of the expanded uncertainty [2] the number of degrees of freedom should be estimated by the application of Welch± Satterwaite formula [2,6,10]. If the variances are signi®cantly different (for at least a 95% signi®cance level) estimation should be reassessed. An experimental variance higher than that estimated indicates the presence of outliers easily detectable by an adequate statistical test or an important source of uncertainty which was badly or even not taken into account. For the opposite case, overestimation is the problem. When there is agreement between the two approaches there are no reasons to think that the estimation may be wrong; therefore, it must be trusted until there are other indications, such as those that will be approached in the quality control section. The presented validation of the uncertainty uses repeatability tests not only as a tool for estimating the precision of a single quanti®cation, but also to extend the estimation process to the evaluation of daily results dispersion. This approach for the presentation of results with a measure of its quality simpli®es routine work by avoiding extensive use of replicate analysis, presenting results with a measure of its quality. The detection of sample contamination, sample projections or other gross errors may continue to ask for replicate analyses although in smaller numbers. 3.2. Accuracy test The precision test does not give information on the accuracy of the method, which must be tested by an adequate procedure. The accuracy test (Fig. 2) is performed through the use of reference materials, when they are available, or by sample spiking. Ideally, the reference material's certi®ed results should be represented by con®dence intervals for known con®dence levels. When that
170
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
Fig. 2. Validation scheme. s2, variance; , degrees of freedom; best estimation of the result; U, uncertainty; s.l., significance level; c, certified value; obs., observed value.
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
information is incomplete, the Eurachem Guide approach should be used to transform the information in a way comparable to that obtained in the laboratory. If there is overlapping of the certi®ed and obtained con®dence intervals there is no reason to suspect that the method is not as accurate as the one used for the reference material certi®cation and it can be said that the method is accurate and valid for the determination under concern. In spiking assays, sample addition should produce a con®dence interval containing the expected value (see following sections). Such conclusion is only characteristic of the speci®ed equipment, operator and laboratory. Extrapolation to other conditions should be tested. 4. Quality control The validation of an analytical method does not necessarily imply that all analyses performed by that method and tested instrumentation produce valid results. A routine analysis must be performed with a quality control scheme in order to detect deviations produced by equipment, analyst or experimental procedure and guarantee the extrapolation of validation conclusions to the daily work. The daily performance of the analytical method can be tested through the analysis of a reference material. Secondary, home-made reference materials can also be certi®ed by the simultaneous validated analysis of a primary one. This approach allows for a detailed and frequent quality control without the need for large amounts of expensive reference materials. When an analytical method includes an instrumental procedure, control standards can be used in order to verify instrumental behaviour. Ideally, standards and samples should be presented randomly [3,11±13] to instruments. However, the instrument software normally separates calibration from sample analysis allowing immediate obtaining of sample contents. Sensitivity drifting should be controlled to avoid lack of accuracy due to the use of outdated calibrations. The periodic analysis of control standards certi®es the quality of sample signal interpolation. In this work, the comparison of expected and obtained values was based on the expression of results with uncertainty instead of the use of control charts.
171
Interpolation uncertainties behaved as a quality control criterion adjustable to the daily instrumental response stability, avoiding the disadvantage of using information from several days. Control charts can estimate, by excess or defect, daily instrumental response precision preventing the detection of signal drifting or the validation of low precision results, respectively. The use of results with a measure of their uncertainty based on ISO/Eurachem Guide principles, as presented, represents an approach of the analytical work different from that of control charts. These principles are also valid for global method quality control, ideally supported by the analysis of reference materials. The analyst needs to conjugate the various control tools that care for the instrumental method performance in order to avoid the extensive use of expensive certi®ed reference materials (CRM) [14]. 5. Application of the validation scheme to atomic spectrometry The above proposed validation scheme was applied to the determination of metals in lettuce leaves by atomic spectrometry. The results refer to a dry base, the dry content determination being parallel to the metal content quanti®cation. The samples were digested in a microwave irradiated closed system in the presence of nitric acid. The instrumental methods used for the quanti®cation step were electrothermal atomic absorption spectrometry (ETAAS) for manganese, ¯ame atomic absorption spectrometry (FAAS) for zinc and ¯ame atomic emission spectrometry for sodium (FAES). Description of the validation process is as follows. 5.1. Validation of the calibration model 5.1.1. Determination of Mn by ETAAS The instrumental quanti®cation was performed on a GBC 906 AA spectrometer with D2 lamp background correction at 279.5 nm. The graphite furnace was equipped with a L'vov platform. The drying stage was performed at two stages, respectively, 1308 and 2008C, ashing was done at 10008C, atomisation took place at 20008C and, ®nally, the cleaning stage was
172
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
effective at 26008C [15,16]. A mixture of palladium nitrate/magnesium nitrate (1.0 g lÿl/1.5 g lÿ1) was used as a matrix modi®er. The instrumental system behaved linearly and with constant variance over the calibration range 0±2±3±4± 5±6 mg lÿ1. Therefore, the linear unweighted model [17] was applied to the characterisation of the calibration graph. 5.1.2. Determination of Zn by FAAS The spectrometric determinations were done with the same instrument as above in the ¯ame mode (airacetylene) using the same background correction, and studying the absorption line at 213.9 nm. The instrument produced a response over the range 0±0.05±0.1± 0.15±0.2±0.25 mg lÿ1 satisfactorily described by the second degree unweighted model [8]. 5.1.3. Determination of Na by FAES The determination of sodium was implemented with the same instrumentation as previously but in the emission mode, at 589.0 nm. The ¯ame was airacetylene. The response ®tted the unweighted second degree model for the concentration range 2±4±5±6±8± 10 mg lÿ1, zero having to be excluded in order to obtain the adjustment. That choice changed a normal calibration graph to more like second degree bracketing. This approach loses the advantage of normal linear bracketing but takes the advantage of a broader concentration range. The blank solution was spiked at 2 mg Na lÿ1 in order to ®nd its in¯uence on the sample result. 5.2. Precision test A detailed description of the precision test uncertainty estimation is available in the literature [3]. 5.2.1. Determination of Mn by ETAAS The manganese repeatability test produced experimental results with two consecutive Grubbs test single outliers. The rejection of those values proved that the experimental standard deviation (0.82 mg kgÿ1 for 98 of freedom, df) was statistically equivalent to that estimated (0.73 mg kgÿ1 for 57 500 df). This conclusion is supported by an F-test using the same signi®cance level (95%) as for the other statistical tests performed.
The average relative expanded uncertainty, AREU, i.e. average expanded uncertainty divided by the average best estimation (middle value) obtained in the precision test was 3.2%. 5.2.2. Determination of Zn by FAAS The determination of zinc in lettuce leaves produced unsatisfactory results. The experimental dispersion (standard deviation, SD: 1.86 mg kgÿ1 for 7 df) was statistically different from that estimated (SD: 0.67 mg kgÿ1 for 5900 df). The success of the precision test in other systems, the detail of the uncertainty estimations and the results of the accuracy test supported the hypothesis of heterogeneity of the test sample with respect to zinc. The in¯uence of eventual contamination, frequent for low levels of zinc, was not observed in the accuracy test, as will be shown, suggesting the trueness of the hypothesis. The AREU for this determination was 3.2%. 5.2.3. Determination of Na by FAES Fig. 3 represents the results obtained for the sodium repeatability test. Results are presented as con®dence intervals with the best estimate (middle point) being associated with expanded uncertainty estimations [2]. Overlapping of all the con®dence intervals is observed, supporting the suggestion that they represent the same sample. The standard deviation associated with the middle points of that con®dence interval (experimental SD) was 0.032% (w/w) with 11 df. The estimated SD following the Eurachem Guide (represented in Fig. 3 in the expanded formÐcoverage factor of 1.96) was 0.049% for 1 1010 df. These two SD values are statistically equivalent at the 95% con®dence level. The test sample precision results obtained had an AREU of 3.2%. 5.3. Accuracy test After the validation of the uncertainty estimation, the accuracy of the method was evaluated through a speci®c test based on the analysis of a certi®ed reference material, CRM, (NIST 1972a). This is a metal content certi®ed spinach leaves CRM which has a proven similarity with lettuce leaves in terms of the proportion of proteins, ®bre, minerals and carbohydrates [18].
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
173
Fig. 3. Precision test for the determination of the sodium content of lettuce leaves.
Alternative sample spiking tests were performed. 5.3.1. Determination of Mn by ETAAS The analysis, in three replicates, of NIST 1972a produced results (best estimations and estimated expanded uncertainty con®dence intervals) equivalent to the 95% con®dence level certi®ed con®dence intervals. The overlapping of those four intervals indicates that they could represent measurements of the same entity. It can also be seen that the ETAAS based method used in this work was less precise than the one used by NIST. This observation is easily understood taking into account the bad reputation of the precision of the applied technique as well as the well-known precision of laser-excited atomic ¯uorescence spectrometry and
instrumental neutron activation analysis used by NIST [19]. 5.3.2. Determination of Zn by FAAS The accuracy test proved to be decisive for the evaluation of the method. The overlapping of the expected value (certi®ed con®dence interval) with three replicate results proved that there were no contamination problems from the procedure (Fig. 4). The homogeneity of the CRM appeared satisfactory for the ful®lment of that criterion. The precision of the FAAS method was very similar to that shown by the certi®ed con®dence interval width. Such observation is in agreement with the known high precision of this quanti®cation step.
Fig. 4. Accuracy test for the determination of the zinc content of lettuce leaves.
174
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
Fig. 5. Accuracy test for the determination of the sodium content of lettuce leaves.
5.3.3. Determination of Na by FAES The accuracy test for sodium was also successful. In order to suggest the recommended procedure for the cases where adequate CRM is not available, spiking assays were performed. The interpolation uncertainty estimation and regression were performed on the assumption that the uncertainty associated with the preparation of standards was negligible when compared with the instrumental reading component [17]. Therefore, the spiking of samples, performed in the same way as the preparation of standards, also has negligible uncertainty. Hence, the observed con®dence intervals (middle point and expanded uncertainty) for a spiked sample should overlap the observed interval for the non-spiked sample plus the added content. This addition is performed at the interval middle point maintaining the interval width. Fig. 5 represents the comparison of the observed and expected values (sample result plus addition) for 4 mg lÿ1 spiking of a sample. There is a clear overlap of the two intervals, supporting the success of the addition. 5.4. Quality control Routine analysis quality control was performed through the analysis of the CRM, after the calibration, validating the sample preparation and through readings of control standards equivalent to calibration standards for the purpose of testing samples signal
interpolation. Readings were taken with control standards after each group of ®ve samples excluding blanks and duplicates validating previous determinations. The routine quanti®cation of manganese (ETAAS), zinc (FAAS) and sodium (FAES) samples content complied with the quality control criterion for 30± 40, 70±85, 55±65 samples, respectively, without exception. Further determinations were performed after re-calibration of the instrument. 6. Conclusions The presented analytical methods, performing at different precisions, were considered valid by the validation scheme. The estimated uncertainties describe adequately the experimental dispersion of results supporting the accuracy of the methods. The estimation of obtained results uncertainty for the reference materials and interpolation uncertainty for the control standards proved to be an adequate tool to detect routine analysis deviations. The proposed quality control scheme was applied without the need for the elaboration of control charts. The independence of this approach from previous data makes it adequate for non-routine work, for the use of various concentration ranges in heteroscedastic instrumental responses and when daily method precision varies due to lack of instrumental stability.
R.J.N. Bettencourt da Silva et al. / Analytica Chimica Acta 393 (1999) 167±175
Acknowledgements Thanks are due to JNICT for ®nancial support, to CONTROLAB LDA for the instrumental facilities as well as to the teams of INETI and CONTROLAB LDA for their co-operative attitude. References [1] International Organisation for Standardisation, ISO, Guide to the Expression of Uncertainty in Measurement, Switzerland, 1993. [2] Eurachem, Quantifying Uncertainty in Analytical Measurement, Version 6, 1995. [3] R.J.N.B. Silva, M.F.G.F.C. CamoÄes, J.S. Barros, Accreditation and Quality Assurance 3 (1988) 155±160. [4] W. Penninckx, C. Hartmann, D.L. Massart, J. SmeyersVerbeke, J. Anal. At. Spectrom. 11 (1996) 237±246. [5] F.E. Grubbs, G. Beck, Technometrics 14 (1972) 847±854. [6] J.K. Taylor, Quality Assurance of Chemical Measurement, Version 5, 1994. [7] P.C. Kelly, J. Assoc. Off. Anal. Chem. 73 (1990) 58±64. [8] International Organisation for Standardisation, ISO International Standard 8466-2 Water Quality, Calibration and
[9] [10] [11] [12] [13] [14] [15] [16] [17]
[18] [19]
175
Evaluation of Analytical Methods and Estimation of Performance Characteristics. Part 2: Calibration Strategy for Nonlinear Second Order Function Geneva, 1993. D.L. Massart, B.G.M. Vandeginste, S.N. Deming, Y. Michotte, L. Kaufman, Chemometrics: a textbook, Elsevier, Amsterdam, 1988. J. Kargten, Analyst 119 (1994) 2161±2165. G. Staas, Fresenius J. Anal. Chem. 352 (1995) 413±419. J.K. Taylor, Anal. Chem. 55 (1983) 600A±608A. Analytical Methods Committee, Analyst, 119 (1994) 2363± 2366. R.J.N.B. Silva, M.F.G.F.C. CamoÄes, submitted to publication. R. Blust, A. Van der Linden, W. Decleir, At. Spectrosc. 6 (1985) 163±165. R. Chakraborty, A.K. Das, M.L. Cervera, M. de la Guardia, J. Anal. At. Spectrom. 10 (1995) 353±358. International Organisation for Standardisation, ISO International Standard 8466-2 Water Quality, Calibration and Evaluation of Analytical Methods and Estimation of Performance Characteristics. Part 1: Calibration Strategy for linear Function Geneva, 1993. W. Penninckx, J. Smeyers-Verbeke, P. Vankeerberghen, D.L. Massart, Anal. Chem. 68 (1996) 481±489. National Institute of Standards and Technology NIST, Certificate of Analysis, Standard Reference Material 1570a, Trace Elements In Spinach Leaves, USA, 1994.