- Email: [email protected]
Commutability and traceability: their repercussions on analytical bias and inaccuracy
Clinica Chimica Acta 280 (1999) 135–145
Commutability and traceability: their repercussions on analytical bias and inaccuracy c d ´ ´ a , *, R. Juvany a , M. Simon ´ b , A. Hernandez ´ C. Ricos , V. Alvarez , e e f ´ C.V. Jimenez , J. Minchinela , C. Perich a
Servei de Bioquimica, Hospitals Vall d’ Hebron, Po Vall d’ Hebron 119 -129, ES-08035 Barcelona, Spain b ´ ´ , Spain Laboratori Clınic Intercomarcal Vilafranca, c / Espirall s /n, ES-08720 Vilafranca del Penedes c ´ Laboratori Clınic de l’ Hospitalet, c / Lerida 50, ES-08901 -L’ Hospitalet de Liobregat, Barcelona, Spain d Laboratori Clinic Cornella´ , c / Bellaterra 41, ES-08940 -cornella´ , Barcelona, Spain e ` Nord, Pza. de la Medicina s /n, ES-08911 -Badalona, Spain Laboratori del Barcelones f C. A.P. bon Pastor, Laboratori, c / Mollerusa s /n, ES-08030 -Barcelona, Spain Received 23 March 1998; received in revised form 20 October 1998; accepted 28 October 1998
Abstract The commutability of calibrators and accuracy control materials affects the traceable link between patient sample results and standards. We sought to identify the repercussions of commutability on various aspects of laboratory practice (calibration, control of bias and accuracy assessment) and to discover the solutions that can reduce the problems produced by noncommutability with presently available resources. Ten serum constituents, ten comparison procedures and 37 analytical procedures were studied. The information concerning accuracy and bias provided from materials found to be commutable in previous works was challenged with native serum results for each routine and reference method compared, using Passing–Bablok regression and decision limits derived from biological variation. We found that: (1) Use of commutable control materials did not assure reliable information on the bias (systematic component of analytical error) of analytical procedures, and (2) Results from native serum and commutable controls were very highly concordant, indicating that these materials provide a good indication of the inaccuracy (total analytical error) of results. We suggest that the performance of individual laboratories would be better evaluated by occasional use of native sera with values assigned by reference methods in EQAS schemes. Moreover, our findings support the idea that manufacturers should assign values to calibrators using reference methods and native sera to reduce matrix effects and promote traceability. 1999 Elsevier Science B.V. All rights reserved. *Corresponding author. [email protected]
Tel.:
13493-274-61-00,
extn.
6976;
fax:
13493-274-68-31;
0009-8981 / 99 / $ – see front matter 1999 Elsevier Science B.V. All rights reserved. PII: S0009-8981( 98 )00187-9
e-mail:
136
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
Keywords: Commutability; Traceability; Biological variation
1. Introduction Commutability is the property of a stabilized material to produce results that, within the uncertainty of a measurement, show the same relationship between two different analytical procedures as do patient specimens [1,2]. Traceability is the property of the result of a measurement that can be related to national or international standards through an unbroken chain of comparisons all having stated uncertainties [3]. The trueness of laboratory results can be demonstrated only when the chain of traceability is intact. Reference methods are considered to be standards and, therefore, the metrologic relations between reference and routine methods should be established to attain traceability of results. The commutability of the materials used to establish this metrologic relation, calibrators and accuracy controls, thus becomes a key factor in the evaluation of laboratory performance and in achieving true results. In previous works, we examined the commutability of stabilized control materials and found that the commercial materials available to us are commutable for several analytical procedures but there is no single material that displays this property for all the constituents and materials we studied [4–6]. It was clear that the issue of commutability still has to be solved. A subsequent study that standardized a method to identify commutable control materials [7] was aimed toward facilitating this practice in the individual laboratories and toward promoting an awareness of the property of commutability among laboratory professionals. Our next concern was to determine to what extent the commutability of the stabilized controls materials used for quality assurance purposes affects the traceability of analytical results to standards. Two basic concepts related to attaining trueness of results, bias and accuracy, were studied. We investigated whether the plots of percent deviation regarding accuracy and bias generated by a commutable material coincided with the deviation plots obtained from native human serum, for each routine and comparison method examined. The limits used for accuracy and bias were derived from the widely accepted criterion of biological variation. In this way, the present study seeks to disclose the ways in which stabilized materials can be properly used and interpreted to demonstrate the trueness of laboratory results.
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
137
2. Materials and methods Ten serum constituents, ten comparison procedures and 37 analytical procedures were studied (Table 1). For each constituent analyzed, 25–30 fresh serum specimens were used to compare the various routine procedures against a comparison method. The regression lines for each pair of methods compared were calculated by the Passing–Babble method [8,9], a statistical technique that does not require normal distribution of the X and Y values or a constant coefficient of variation over the measurement range. Moreover, extreme values do not affect the final calculation. After applying this calculation, the differences between each regression line obtained and the ideal regression line obtained and the ideal regression ( y-intercept of zero and slope of 1) were used for determining the bias between each analytical procedure and the corresponding comparison method. The stabilized material that in our previous works [4–6] was shown to be commutable with fresh human serum for the maximum number of pairs of methods compared was Monitrol-I-X lot 615003 from Dade International (Miami, USA). We calculated the deviation of this material to see if it coincided with the bias shown by the regression line of the procedures and comparison methods. For each constituent and procedure studied, Monitrol-I-X was analyzed 20 times, and the difference between the averaged results ( y) and the
Table 1 Constituents and analytical procedures studied Constituent
Comparison procedure
Studied procedure
a-amylase Aspartate aminotransferase (AST) Cholesterol Creatine kinase Creatinine
G7-EOS (Hitachi-747) IFCC (Hitachi-747) CHOD-POD (Hit-747) DGKC (Hitachi-747) HPLC
Reflotron, Spotchem, Vitros Reflotron, Seralyzer, Spotchem
Glucose
GOD-POD
Potassium ion Triglyceride Urate
Indirect ISE (Hit-747) Lip / Glyc (Hitachi-747) Uricase-POD (Hit-747)
Urea
Urease (Hitachi-747)
Reflotron, Spotchem, Vitros Jaffe´ (Hitachi 747, Chem-1) Enzymatic (Hit 911, Reflotron, Vitros) Reflotron, Spotchem, Vitros HK (Hitachi 747) Reflotron, Seralyzer, Vitros Reflotron, Seralyzer, Vitros Chem-1, Reflotron, Seralyzer, Spotchem Cchem-1, Reflotron, Seralyzer, Spotchem, Vitros
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
138
corresponding expected value over the serum data regression line ( y9) expressed in percent term, was the deviation of the material. Various stabilised materials that had been shown to be computable with fresh human serum for each pair of methods compared in the previous works [4–6] were analysed in the present study, and the differences between the results obtained for a single analysis with the routine procedure and the expected value over the corresponding regression line were used to express inaccuracy of each analytical procedure. The acceptability limits used for interpreting the plots were derived from within an between-subject biological variation, according to the following formulas: Bias , 0.25(CV 2w 1 CV 2g )1 / 2 2
2 1/2
Inaccuracy , 2.33*0.5CVw 1 0.25(CV w 1 CV g )
3. Results In determining the trueness of results, inaccuracy refers to the percent difference between a single measured result and the value expected on the basis of the result from the comparison method [3]. To present the inaccuracy of results graphically, four representative constituents (AST, cholesterol, creatinine and glucose) were randomly chosen for illustrative purposes. In Fig. 1 the inaccuracy of the measurements using various computable control materials (triangles) is depicted. The dashed lines represent the confidence interval of the Passing–Babble regression from the serum sample result, and the continuous ´ et al. lines depict the acceptability limit for single analysis described by Ricos [10], derived from biological variation. When the serum sample results fell within the acceptability limits (indicating no error), the results from the commutable materials were also found within, and when serum results fell outside the limits (indicating error) the control material results followed suit. The information proceeding from the native serum and commutable control was highly concordant. This means that commutable controls provide a good indication of the inaccuracy of results. In the creatinine plot, the serum sample values show two tendencies that are concentration-dependent, and results from the commutable controls coincide for both tendencies. This same coherence was found for all the other analytes and procedures studied. The bias (or systematic error) of an analytical procedure refers to the relative difference between the mean of a number of measurements and the value expected on the basis of the result from the comparative method [3]. Using a similar presentation, Fig. 2 illustrates the bias for potassium, creatine kinase and
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos 139
Fig. 1. Inaccuracy of measurements of AST (Reflotron vs Hitachi-747), cholesterol (Chem-1 vs Hitachi-747), creatinine (Hitachi-747 vs HPLC) and glucose (Vitros DT-60 vs Hitachi-747).
140 ´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos Fig. 2. Bias of measurements of potassium ion (Seralyzer vs Hitachi-747), creatine kinase (Spotchem vs Hitachi-747) and triglyceride (Vitros DT-60 vs Hitachi-747).
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
141
triglycerides, analyzed by the Seralyzer, Spotchem and Vitros DT-60 instruments, respectively, vs the Hitachi 747 comparison procedure. For each analyte, the serum data and the confidence interval of the regression line (dashed line) are plotted together with the ideal regression line – indicating no bias – and its corresponding limit of acceptance (continuous line), derived from biology [11]. It can be seen that the three tests possess some bias, since the serum samples fall outside the acceptability limits. The commutable control material, however, (triangle) shows no bias. This lack of concordance between the analytical bias information provided by the commutable control and the serum regression lines was found for all the remaining analytes and procedures studied.
4. Discussion As confirmed by our findings, the demonstration of trueness of laboratory results is closely related with the stabilized materials used for calibration, control of bias, and control of accuracy. The repercussions of the commutability of these materials on each intended use is discussed here.
4.1. Calibration and traceability The traceability of the analytical process can be vertical, as attained by calibration, or horizontal, as attained by participation in external quality assessment programs. In the clinical laboratory primary standards do not function well as calibrators. Being aqueous or alcohol solutions containing a pure substance prepared by weight [12], they little resemble human samples. Secondary standards (also known as reference materials), solutions with a matrix similar to human specimens and containing constituents with values assigned by reference methods, are more practicable alternatives. However, these calibrators are stabilized, generally by means of a process of lyophilization and, consequently, their resemblance to human specimens may be compromised. Moreover, in the calibration process the metrologic characteristics of the calibrator and the patient samples are compared with the concentration assigned to the calibrator by the reference method. Thus, the variability of two analytical methods comes into play and if the calibrator is not commutable with specimens from patients, the chain of traceability to the standards is broken. The effect that the commutability of calibrators has on the accuracy of analytical results has been well demonstrated by Catozzo et al. [13]. These authors showed that direct potentiometry determinations of sodium and potassium ions were traceable to the reference method in more than 50% of laboratories participating in the study when provided with commutable cali-
142
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
brators, whereas less than 5% were traceable when non-commutable calibrators were used. As the works to date on commutability indicate, there are no materials on the market that are commutable for a large number of constituents and suitable for use in the routine laboratory as multicalibrators. It is not easy to determine why materials are non-commutable. The characteristics of stabilized materials that condition their commutability have still not been defined [14], but it is known that human origin is not a determinant factor [15]. It has been suggested that manufacturers of calibrators should use native human sera when assigning values by reference methods to correct the matrix effect of their products. This solution, proposed by Thienpont et al. [16] based on the findings of Lasky [17,18] would greatly reduce the negative repercussions of commutability. Horizontal traceability is attained by demonstrating that the results in a laboratory approximate the results obtained by other laboratories working in the same professional area. The way to achieve this is participation in external quality assessment programs. At regular intervals, a control material (generally stabilized) is distributed to a large number of laboratories which analyze them once. When the deviation between the result obtained by the laboratory and the theoretic value of the control falls within the limits allowed by the organizer, laboratory accuracy is accepted. If the theoretical value of the control has been assigned by a reference method (standard), The result, besides being accurate, is also considered to be traceable. It has been demonstrated here (Fig. 1) that if the control material distributed are computable with human specimens, the information obtained with the control will closely reflect the behaviour of the specimens; that is, the results obtained for patient samples may be considered traceable to standards.
4.2. Bias When the clinical laboratory is organized in such a way that two procedures are used to analyze the same constituent in the same patient at different times (e.g. hospital emergency and routine laboratories), the bias between the two procedures must be known to avoid false interpretation of changes in the patient’s condition. According to our findings, the use of commutable control materials does not assure reliable information on the bias of an analytical procedure (Fig. 2). A better way to control laboratory bias would be first to establish the regression line by analyzing a set of patient sera covering a wide interval of concentrations, using the study and comparison methods [7]. Once the bias is established, the two procedures can be routinely monitored by the control material, bearing in mind that even though the same material is used, the target value obtained for each procedure may be different. This is not a problem for the internal quality control of each analytical procedure, since the non-
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
143
commutability of a control does not affect its capacity to reliably reflect the robustness of a single analytical procedure. The same rationale can be applied to the internal quality control of analytical imprecision (agreement between the results of measurements of the same measurand) [3]: commutable materials are not required, because the aim is to evaluate the consistency of a single procedure over time.
4.3. External quality assessment Participation in external quality assessment programs provides objective evidence as to whether or not the laboratory is producing satisfactory results. The laboratory is compared with its homogeneous group, i.e. the set formed by all the laboratories using the same method, reagents, instrument and calibrator. Any control material will reliably reflect the performance of a single laboratory in the group since a single procedure is involved. Therefore, in this case the problem of commutability between stabilised materials and human specimens does not affect the comparison. Instead, efforts should be centered on correctly identifying the groups as homogeneous. It is clear that if the laboratory decides to use instruments, reagents, and even calibrators produced by different manufacturers, it will be difficult to find a group for quality assessment that uses the same procedure. Thus, the previous practice of freely choosing instruments, reagents and calibrators may need to be balanced with the policy of demonstrating quality of results. Moreover, the forthcoming EC directive on in vitro diagnostic medical devices, requiring that manufacturers assume the responsibility for defining the traceability of their products, will free the laboratories from this task and permit them to work with procedures that are more comparable. Libeer et. al. [19] proposed several designs for external quality assessment programs according to the objective desired. To know the bias of each participant laboratory and to establish the dispersion between laboratories, one stable control can be used without determining its commutability. However, Catozzo [13] has demonstrated that the use of commutable controls significantly reduces the divergent results obtained by different laboratories using the same methods. Since, as we have seen, controls that are commutable for a large number of constituents and analytical procedures are not presently available, we suggest the distribution of frozen human serum in external quality assurance programs. This would significantly improve interlaboratory comparisons. To evaluate the inaccuracy of a specific analytical method against the true value, native human serum or a commutable control should be circulated. Since this quality verification operation does not need to be done frequently, the cost of such programs would, presumably, be acceptable. In summary, the results of this work show the influence that commutability
144
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
has on various aspects of laboratory quality assurance. According to our findings, though use of commutable control materials provides a good indication of the accuracy of results, commutable controls do not assure reliable information on the bias of the analytical procedure. This finding prompts us to suggest that occasional distribution of native sera with values assigned by reference methods in EQAS programs would be useful for assuring the accuracy of results of individual laboratories. To promote traceability, calibrators should be prepared using reference methods and native sera to reduce matrix effects. Though the laboratory staff should be aware that commutability-related problems can affect their performance, the definitive resolution of this question can only be attained by the combined effort of laboratories, manufacturers and EQAS organizers.
Acknowledgements This work has been carried out with the support of a grant from the Fondo de Investigaciones Sanitarias de la Seguridad Social. C.R., R.J., M.S., A.H., V.A., C.V.J., J.M. and C.P. are members of the Quality Assurance and Laboratory Accreditation Committee from the Spanish Society of Clinical Biochemistry and Molecular Pathology (SEQC).
References [1] Clark P, Kricka LJ, Whitehead TP. Matrix effects in clinical analysis: commutability of control materials between the Ektachem, Beckman and SMA 12 / 60 glucose and urea methods. Clin Chim Acta 1981;113:293–303. [2] Franzini C. Commutability of reference materials in clinical chemistry. JIFCC 1993;5:186– 93. [3] BIPM, IFC, IIFCC, ISO, TUPAC, LUPAP, OIML. International vocabulary of basic and general terms in metrology. ISO 2nd ed. Switzerland, 1993. ´ ´ C, Juvany R, Alvarez [4] Ricos V, et al. Commutability between stabilized materials and human serum in serum creatinine testing. Quim Clin 1995;14:350–6. [5] Minchinela J, Perich C, Juvany R, et al. Commutability between stabilized materials and human serum cholesterol, glucose, urate and urea testing. Quim Clin 1995;14:357–63. ´ ´ C, Juvany R, Alvarez [6] Ricos V, et al. Commutability between stabilized materials and fresh human serum to improve laboratory performance. Clin Chim Acta 1997;263:225–38. ´ C, Juvany R, Jimenez ´ [7] Ricos CV, et al. Procedure for studying commutability validated by biological variation. Clin Chim Acta 1998;268:73–83. [8] Passing H, Bablok W. A new biometrical procedure for testing the equality of measurements from two different analytical methods. Application of linear regression procedures for method comparison studies in clinical chemistry, part 1. J Clin Chem Clin Biochem 1983;21:709–20.
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
145
[9] Passing H, Bablok W. Comparison of several regression procedures for method comparison studies and determination of sample sizes. Application of linear regression procedures for method comparison studies in clinical chemistry, part 2. J Clin Chem Clin Biochem 1984;22:431–45. ´ C, Baadenhuijsen H, Libeer JC, et al. External quality assessment: currently used [10] Ricos criteria for evaluating performance in European countries, and criteria for future harmonization. Eur J Clin Chem Clin Biochem 1996;34:159–65. [11] Gowans EMS, Hyltoft Petersen P, Blaabjerg O, Horder M. Analytical goals for the acceptance of common reference intervals for laboratories throughout a geographical area. Scan J Clin Lab Invest 1988;47:757–64. ´ M, Alsina MJ, Ricos ´ C, Ramon ´ F, Navarro JM. Garantia de la calidad. En: [12] Cortes ´ ´ ´ ´ y diagnostico: ´ ´ de Gonzalez-Sastre F, editor. Bioquımica clınica. Semiologıa interpretacion ´ los datos bioquımicos. Barcelona: Barcanova, 1993. [13] Catozzo G, Fabi A, Franzini C. Intermethod variability of sodium and potassium results: patients sera and commercially available control sera. Eur J Clin Chem Clin Biochem 1997;35:387–92. [14] Libeer JC. External quality assessment in clinical laboratories. European perspectives; today and tomorrow. Doctoral thesis. Antwerp, 1993. ´ ´ [15] Juvany R. Quımica seca versus quımica convencional. Impacto de la conmutabilidad sobre la ` transferibilidad de los resultados. Doctoral Thesis. Barcelona: Universitat Autonoma de Barcelona, Departament de Bioquimica i Biologia Molecular, 1996. [16] Thienpont L, Franzini C, Kratochvila J, et al. Analytical quality specifications for reference methods and operating specifications for networks of reference laboratories. Eur J Clin Chem Clin Biochem 1995;33:949–57. [17] Lasky FD. Achieving accuracy for routine clinical chemistry methods by using patient specimen correlations to assign calibrator values. Arch Pathol Lab Med 1993;117:412–9. [18] Lasky FD. Proficiency testing linked to the National References System for the clinical laboratory. A proposal for achieving accuracy. Clin Chem 1992;38:1260–7. [19] Libeer JC, Baadenhuijsen H, Fraser CG, et al. Characterization and classification of EQA according to objectives such as evaluation of method and participant bias and standard deviation. Eur J Clin Chem Clin Biochem 1996;34:665–78.
Commutability and traceability: their repercussions on analytical bias and inaccuracy c d ´ ´ a , *, R. Juvany a , M. Simon ´ b , A. Hernandez ´ C. Ricos , V. Alvarez , e e f ´ C.V. Jimenez , J. Minchinela , C. Perich a
Servei de Bioquimica, Hospitals Vall d’ Hebron, Po Vall d’ Hebron 119 -129, ES-08035 Barcelona, Spain b ´ ´ , Spain Laboratori Clınic Intercomarcal Vilafranca, c / Espirall s /n, ES-08720 Vilafranca del Penedes c ´ Laboratori Clınic de l’ Hospitalet, c / Lerida 50, ES-08901 -L’ Hospitalet de Liobregat, Barcelona, Spain d Laboratori Clinic Cornella´ , c / Bellaterra 41, ES-08940 -cornella´ , Barcelona, Spain e ` Nord, Pza. de la Medicina s /n, ES-08911 -Badalona, Spain Laboratori del Barcelones f C. A.P. bon Pastor, Laboratori, c / Mollerusa s /n, ES-08030 -Barcelona, Spain Received 23 March 1998; received in revised form 20 October 1998; accepted 28 October 1998
Abstract The commutability of calibrators and accuracy control materials affects the traceable link between patient sample results and standards. We sought to identify the repercussions of commutability on various aspects of laboratory practice (calibration, control of bias and accuracy assessment) and to discover the solutions that can reduce the problems produced by noncommutability with presently available resources. Ten serum constituents, ten comparison procedures and 37 analytical procedures were studied. The information concerning accuracy and bias provided from materials found to be commutable in previous works was challenged with native serum results for each routine and reference method compared, using Passing–Bablok regression and decision limits derived from biological variation. We found that: (1) Use of commutable control materials did not assure reliable information on the bias (systematic component of analytical error) of analytical procedures, and (2) Results from native serum and commutable controls were very highly concordant, indicating that these materials provide a good indication of the inaccuracy (total analytical error) of results. We suggest that the performance of individual laboratories would be better evaluated by occasional use of native sera with values assigned by reference methods in EQAS schemes. Moreover, our findings support the idea that manufacturers should assign values to calibrators using reference methods and native sera to reduce matrix effects and promote traceability. 1999 Elsevier Science B.V. All rights reserved. *Corresponding author. [email protected]
Tel.:
13493-274-61-00,
extn.
6976;
fax:
13493-274-68-31;
0009-8981 / 99 / $ – see front matter 1999 Elsevier Science B.V. All rights reserved. PII: S0009-8981( 98 )00187-9
e-mail:
136
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
Keywords: Commutability; Traceability; Biological variation
1. Introduction Commutability is the property of a stabilized material to produce results that, within the uncertainty of a measurement, show the same relationship between two different analytical procedures as do patient specimens [1,2]. Traceability is the property of the result of a measurement that can be related to national or international standards through an unbroken chain of comparisons all having stated uncertainties [3]. The trueness of laboratory results can be demonstrated only when the chain of traceability is intact. Reference methods are considered to be standards and, therefore, the metrologic relations between reference and routine methods should be established to attain traceability of results. The commutability of the materials used to establish this metrologic relation, calibrators and accuracy controls, thus becomes a key factor in the evaluation of laboratory performance and in achieving true results. In previous works, we examined the commutability of stabilized control materials and found that the commercial materials available to us are commutable for several analytical procedures but there is no single material that displays this property for all the constituents and materials we studied [4–6]. It was clear that the issue of commutability still has to be solved. A subsequent study that standardized a method to identify commutable control materials [7] was aimed toward facilitating this practice in the individual laboratories and toward promoting an awareness of the property of commutability among laboratory professionals. Our next concern was to determine to what extent the commutability of the stabilized controls materials used for quality assurance purposes affects the traceability of analytical results to standards. Two basic concepts related to attaining trueness of results, bias and accuracy, were studied. We investigated whether the plots of percent deviation regarding accuracy and bias generated by a commutable material coincided with the deviation plots obtained from native human serum, for each routine and comparison method examined. The limits used for accuracy and bias were derived from the widely accepted criterion of biological variation. In this way, the present study seeks to disclose the ways in which stabilized materials can be properly used and interpreted to demonstrate the trueness of laboratory results.
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
137
2. Materials and methods Ten serum constituents, ten comparison procedures and 37 analytical procedures were studied (Table 1). For each constituent analyzed, 25–30 fresh serum specimens were used to compare the various routine procedures against a comparison method. The regression lines for each pair of methods compared were calculated by the Passing–Babble method [8,9], a statistical technique that does not require normal distribution of the X and Y values or a constant coefficient of variation over the measurement range. Moreover, extreme values do not affect the final calculation. After applying this calculation, the differences between each regression line obtained and the ideal regression line obtained and the ideal regression ( y-intercept of zero and slope of 1) were used for determining the bias between each analytical procedure and the corresponding comparison method. The stabilized material that in our previous works [4–6] was shown to be commutable with fresh human serum for the maximum number of pairs of methods compared was Monitrol-I-X lot 615003 from Dade International (Miami, USA). We calculated the deviation of this material to see if it coincided with the bias shown by the regression line of the procedures and comparison methods. For each constituent and procedure studied, Monitrol-I-X was analyzed 20 times, and the difference between the averaged results ( y) and the
Table 1 Constituents and analytical procedures studied Constituent
Comparison procedure
Studied procedure
a-amylase Aspartate aminotransferase (AST) Cholesterol Creatine kinase Creatinine
G7-EOS (Hitachi-747) IFCC (Hitachi-747) CHOD-POD (Hit-747) DGKC (Hitachi-747) HPLC
Reflotron, Spotchem, Vitros Reflotron, Seralyzer, Spotchem
Glucose
GOD-POD
Potassium ion Triglyceride Urate
Indirect ISE (Hit-747) Lip / Glyc (Hitachi-747) Uricase-POD (Hit-747)
Urea
Urease (Hitachi-747)
Reflotron, Spotchem, Vitros Jaffe´ (Hitachi 747, Chem-1) Enzymatic (Hit 911, Reflotron, Vitros) Reflotron, Spotchem, Vitros HK (Hitachi 747) Reflotron, Seralyzer, Vitros Reflotron, Seralyzer, Vitros Chem-1, Reflotron, Seralyzer, Spotchem Cchem-1, Reflotron, Seralyzer, Spotchem, Vitros
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
138
corresponding expected value over the serum data regression line ( y9) expressed in percent term, was the deviation of the material. Various stabilised materials that had been shown to be computable with fresh human serum for each pair of methods compared in the previous works [4–6] were analysed in the present study, and the differences between the results obtained for a single analysis with the routine procedure and the expected value over the corresponding regression line were used to express inaccuracy of each analytical procedure. The acceptability limits used for interpreting the plots were derived from within an between-subject biological variation, according to the following formulas: Bias , 0.25(CV 2w 1 CV 2g )1 / 2 2
2 1/2
Inaccuracy , 2.33*0.5CVw 1 0.25(CV w 1 CV g )
3. Results In determining the trueness of results, inaccuracy refers to the percent difference between a single measured result and the value expected on the basis of the result from the comparison method [3]. To present the inaccuracy of results graphically, four representative constituents (AST, cholesterol, creatinine and glucose) were randomly chosen for illustrative purposes. In Fig. 1 the inaccuracy of the measurements using various computable control materials (triangles) is depicted. The dashed lines represent the confidence interval of the Passing–Babble regression from the serum sample result, and the continuous ´ et al. lines depict the acceptability limit for single analysis described by Ricos [10], derived from biological variation. When the serum sample results fell within the acceptability limits (indicating no error), the results from the commutable materials were also found within, and when serum results fell outside the limits (indicating error) the control material results followed suit. The information proceeding from the native serum and commutable control was highly concordant. This means that commutable controls provide a good indication of the inaccuracy of results. In the creatinine plot, the serum sample values show two tendencies that are concentration-dependent, and results from the commutable controls coincide for both tendencies. This same coherence was found for all the other analytes and procedures studied. The bias (or systematic error) of an analytical procedure refers to the relative difference between the mean of a number of measurements and the value expected on the basis of the result from the comparative method [3]. Using a similar presentation, Fig. 2 illustrates the bias for potassium, creatine kinase and
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos 139
Fig. 1. Inaccuracy of measurements of AST (Reflotron vs Hitachi-747), cholesterol (Chem-1 vs Hitachi-747), creatinine (Hitachi-747 vs HPLC) and glucose (Vitros DT-60 vs Hitachi-747).
140 ´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos Fig. 2. Bias of measurements of potassium ion (Seralyzer vs Hitachi-747), creatine kinase (Spotchem vs Hitachi-747) and triglyceride (Vitros DT-60 vs Hitachi-747).
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
141
triglycerides, analyzed by the Seralyzer, Spotchem and Vitros DT-60 instruments, respectively, vs the Hitachi 747 comparison procedure. For each analyte, the serum data and the confidence interval of the regression line (dashed line) are plotted together with the ideal regression line – indicating no bias – and its corresponding limit of acceptance (continuous line), derived from biology [11]. It can be seen that the three tests possess some bias, since the serum samples fall outside the acceptability limits. The commutable control material, however, (triangle) shows no bias. This lack of concordance between the analytical bias information provided by the commutable control and the serum regression lines was found for all the remaining analytes and procedures studied.
4. Discussion As confirmed by our findings, the demonstration of trueness of laboratory results is closely related with the stabilized materials used for calibration, control of bias, and control of accuracy. The repercussions of the commutability of these materials on each intended use is discussed here.
4.1. Calibration and traceability The traceability of the analytical process can be vertical, as attained by calibration, or horizontal, as attained by participation in external quality assessment programs. In the clinical laboratory primary standards do not function well as calibrators. Being aqueous or alcohol solutions containing a pure substance prepared by weight [12], they little resemble human samples. Secondary standards (also known as reference materials), solutions with a matrix similar to human specimens and containing constituents with values assigned by reference methods, are more practicable alternatives. However, these calibrators are stabilized, generally by means of a process of lyophilization and, consequently, their resemblance to human specimens may be compromised. Moreover, in the calibration process the metrologic characteristics of the calibrator and the patient samples are compared with the concentration assigned to the calibrator by the reference method. Thus, the variability of two analytical methods comes into play and if the calibrator is not commutable with specimens from patients, the chain of traceability to the standards is broken. The effect that the commutability of calibrators has on the accuracy of analytical results has been well demonstrated by Catozzo et al. [13]. These authors showed that direct potentiometry determinations of sodium and potassium ions were traceable to the reference method in more than 50% of laboratories participating in the study when provided with commutable cali-
142
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
brators, whereas less than 5% were traceable when non-commutable calibrators were used. As the works to date on commutability indicate, there are no materials on the market that are commutable for a large number of constituents and suitable for use in the routine laboratory as multicalibrators. It is not easy to determine why materials are non-commutable. The characteristics of stabilized materials that condition their commutability have still not been defined [14], but it is known that human origin is not a determinant factor [15]. It has been suggested that manufacturers of calibrators should use native human sera when assigning values by reference methods to correct the matrix effect of their products. This solution, proposed by Thienpont et al. [16] based on the findings of Lasky [17,18] would greatly reduce the negative repercussions of commutability. Horizontal traceability is attained by demonstrating that the results in a laboratory approximate the results obtained by other laboratories working in the same professional area. The way to achieve this is participation in external quality assessment programs. At regular intervals, a control material (generally stabilized) is distributed to a large number of laboratories which analyze them once. When the deviation between the result obtained by the laboratory and the theoretic value of the control falls within the limits allowed by the organizer, laboratory accuracy is accepted. If the theoretical value of the control has been assigned by a reference method (standard), The result, besides being accurate, is also considered to be traceable. It has been demonstrated here (Fig. 1) that if the control material distributed are computable with human specimens, the information obtained with the control will closely reflect the behaviour of the specimens; that is, the results obtained for patient samples may be considered traceable to standards.
4.2. Bias When the clinical laboratory is organized in such a way that two procedures are used to analyze the same constituent in the same patient at different times (e.g. hospital emergency and routine laboratories), the bias between the two procedures must be known to avoid false interpretation of changes in the patient’s condition. According to our findings, the use of commutable control materials does not assure reliable information on the bias of an analytical procedure (Fig. 2). A better way to control laboratory bias would be first to establish the regression line by analyzing a set of patient sera covering a wide interval of concentrations, using the study and comparison methods [7]. Once the bias is established, the two procedures can be routinely monitored by the control material, bearing in mind that even though the same material is used, the target value obtained for each procedure may be different. This is not a problem for the internal quality control of each analytical procedure, since the non-
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
143
commutability of a control does not affect its capacity to reliably reflect the robustness of a single analytical procedure. The same rationale can be applied to the internal quality control of analytical imprecision (agreement between the results of measurements of the same measurand) [3]: commutable materials are not required, because the aim is to evaluate the consistency of a single procedure over time.
4.3. External quality assessment Participation in external quality assessment programs provides objective evidence as to whether or not the laboratory is producing satisfactory results. The laboratory is compared with its homogeneous group, i.e. the set formed by all the laboratories using the same method, reagents, instrument and calibrator. Any control material will reliably reflect the performance of a single laboratory in the group since a single procedure is involved. Therefore, in this case the problem of commutability between stabilised materials and human specimens does not affect the comparison. Instead, efforts should be centered on correctly identifying the groups as homogeneous. It is clear that if the laboratory decides to use instruments, reagents, and even calibrators produced by different manufacturers, it will be difficult to find a group for quality assessment that uses the same procedure. Thus, the previous practice of freely choosing instruments, reagents and calibrators may need to be balanced with the policy of demonstrating quality of results. Moreover, the forthcoming EC directive on in vitro diagnostic medical devices, requiring that manufacturers assume the responsibility for defining the traceability of their products, will free the laboratories from this task and permit them to work with procedures that are more comparable. Libeer et. al. [19] proposed several designs for external quality assessment programs according to the objective desired. To know the bias of each participant laboratory and to establish the dispersion between laboratories, one stable control can be used without determining its commutability. However, Catozzo [13] has demonstrated that the use of commutable controls significantly reduces the divergent results obtained by different laboratories using the same methods. Since, as we have seen, controls that are commutable for a large number of constituents and analytical procedures are not presently available, we suggest the distribution of frozen human serum in external quality assurance programs. This would significantly improve interlaboratory comparisons. To evaluate the inaccuracy of a specific analytical method against the true value, native human serum or a commutable control should be circulated. Since this quality verification operation does not need to be done frequently, the cost of such programs would, presumably, be acceptable. In summary, the results of this work show the influence that commutability
144
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
has on various aspects of laboratory quality assurance. According to our findings, though use of commutable control materials provides a good indication of the accuracy of results, commutable controls do not assure reliable information on the bias of the analytical procedure. This finding prompts us to suggest that occasional distribution of native sera with values assigned by reference methods in EQAS programs would be useful for assuring the accuracy of results of individual laboratories. To promote traceability, calibrators should be prepared using reference methods and native sera to reduce matrix effects. Though the laboratory staff should be aware that commutability-related problems can affect their performance, the definitive resolution of this question can only be attained by the combined effort of laboratories, manufacturers and EQAS organizers.
Acknowledgements This work has been carried out with the support of a grant from the Fondo de Investigaciones Sanitarias de la Seguridad Social. C.R., R.J., M.S., A.H., V.A., C.V.J., J.M. and C.P. are members of the Quality Assurance and Laboratory Accreditation Committee from the Spanish Society of Clinical Biochemistry and Molecular Pathology (SEQC).
References [1] Clark P, Kricka LJ, Whitehead TP. Matrix effects in clinical analysis: commutability of control materials between the Ektachem, Beckman and SMA 12 / 60 glucose and urea methods. Clin Chim Acta 1981;113:293–303. [2] Franzini C. Commutability of reference materials in clinical chemistry. JIFCC 1993;5:186– 93. [3] BIPM, IFC, IIFCC, ISO, TUPAC, LUPAP, OIML. International vocabulary of basic and general terms in metrology. ISO 2nd ed. Switzerland, 1993. ´ ´ C, Juvany R, Alvarez [4] Ricos V, et al. Commutability between stabilized materials and human serum in serum creatinine testing. Quim Clin 1995;14:350–6. [5] Minchinela J, Perich C, Juvany R, et al. Commutability between stabilized materials and human serum cholesterol, glucose, urate and urea testing. Quim Clin 1995;14:357–63. ´ ´ C, Juvany R, Alvarez [6] Ricos V, et al. Commutability between stabilized materials and fresh human serum to improve laboratory performance. Clin Chim Acta 1997;263:225–38. ´ C, Juvany R, Jimenez ´ [7] Ricos CV, et al. Procedure for studying commutability validated by biological variation. Clin Chim Acta 1998;268:73–83. [8] Passing H, Bablok W. A new biometrical procedure for testing the equality of measurements from two different analytical methods. Application of linear regression procedures for method comparison studies in clinical chemistry, part 1. J Clin Chem Clin Biochem 1983;21:709–20.
´ et al. / Clinica Chimica Acta 280 (1999) 135 – 145 C. Ricos
145
[9] Passing H, Bablok W. Comparison of several regression procedures for method comparison studies and determination of sample sizes. Application of linear regression procedures for method comparison studies in clinical chemistry, part 2. J Clin Chem Clin Biochem 1984;22:431–45. ´ C, Baadenhuijsen H, Libeer JC, et al. External quality assessment: currently used [10] Ricos criteria for evaluating performance in European countries, and criteria for future harmonization. Eur J Clin Chem Clin Biochem 1996;34:159–65. [11] Gowans EMS, Hyltoft Petersen P, Blaabjerg O, Horder M. Analytical goals for the acceptance of common reference intervals for laboratories throughout a geographical area. Scan J Clin Lab Invest 1988;47:757–64. ´ M, Alsina MJ, Ricos ´ C, Ramon ´ F, Navarro JM. Garantia de la calidad. En: [12] Cortes ´ ´ ´ ´ y diagnostico: ´ ´ de Gonzalez-Sastre F, editor. Bioquımica clınica. Semiologıa interpretacion ´ los datos bioquımicos. Barcelona: Barcanova, 1993. [13] Catozzo G, Fabi A, Franzini C. Intermethod variability of sodium and potassium results: patients sera and commercially available control sera. Eur J Clin Chem Clin Biochem 1997;35:387–92. [14] Libeer JC. External quality assessment in clinical laboratories. European perspectives; today and tomorrow. Doctoral thesis. Antwerp, 1993. ´ ´ [15] Juvany R. Quımica seca versus quımica convencional. Impacto de la conmutabilidad sobre la ` transferibilidad de los resultados. Doctoral Thesis. Barcelona: Universitat Autonoma de Barcelona, Departament de Bioquimica i Biologia Molecular, 1996. [16] Thienpont L, Franzini C, Kratochvila J, et al. Analytical quality specifications for reference methods and operating specifications for networks of reference laboratories. Eur J Clin Chem Clin Biochem 1995;33:949–57. [17] Lasky FD. Achieving accuracy for routine clinical chemistry methods by using patient specimen correlations to assign calibrator values. Arch Pathol Lab Med 1993;117:412–9. [18] Lasky FD. Proficiency testing linked to the National References System for the clinical laboratory. A proposal for achieving accuracy. Clin Chem 1992;38:1260–7. [19] Libeer JC, Baadenhuijsen H, Fraser CG, et al. Characterization and classification of EQA according to objectives such as evaluation of method and participant bias and standard deviation. Eur J Clin Chem Clin Biochem 1996;34:665–78.