- Email: [email protected]
Observations on diagnostic kits for the determination of chloride
Clin. Biochem. 5, 214-221 (1972)
OBSERVATIONS ON DIAGNOSTIC KITS FOR THE DETERMINATION OF CHLORIDE
E. K. KIM, L. D. W A D D E L L and J. E. LOGAN
CIb~ical Laboratories, Canadia~ Communicable Disease CeTitre, Departme~t of Natio~al Health and Welfare, Ottawa, O~Ttario. (Received April 26, 1972)
SUM MARY
1. Diagnostic kits or reagent sets from eight m a n u f a c t u r e r s for the determination of chloride were assessed. 2. The kits were assessed for adequacy of literature supplied, the reproducibility of the procedures, the accuracy of results compared with those obtained by reference method, and for the recovery of chloride added to sera. 3. Only two kits were acceptable according to our criteria for a good laboratory test procedure. One kit just exceeded the allowable limits of error for precision, whereas another tended to produce some falsely elevated values. F o u r kits frequently yielded data for patient sera which could lead to incorrect clinical interpretations. 4. The cost analysis of the various kits showed no correlation between price and reagent constitution, particularly for the four kits based on the colorimetric principle.
T H E USE OF DIAGNOSTIC KITS AND REAGENTS in clinical laboratories for the determination of chloride was found to be extensive on both a routine and standby basis (1). In a survey conducted by Logan et al (1) 50 or 11.5~, of 435 laboratories across Canada indicated t h a t they were using chloride kits. Thus information on the reliability of those kits which are available on the Canadian m a r k e t should be useful. The results of an assessment of eight kits tested in this laboratory are presented.
Correspondence: Dr. J. E. Logan.
CHLORIDE DIAGNOSTIC KITS
215
METHODS The chloride kits which we have studied are listed as follows: 1. Sigma Chloride # 8 3 0 ; Sigma Chemical Co., St. Louis, Mo. 2. Medi-Chem Chloride; Medi-Chem Inc., S a n t a Monica, Calif. 3. Oxford Chloride Reagent # 6 3 0 ; Oxford Laboratories, San Mateo, Calif. 4. E.M.-Test Chloride; E. Merck A.G., Darmstadt, Germany, Canadian distributor - - B r i n k m a n n I n s t r u m e n t s (Canada) Ltd., Rexdale, Ontario. 5. Harleco Chloride; Harleco, Philadelphia, Pa. 6. Uni-Tech Chlori-Pak; Uni-Tech Chemical M a n u f a c t u r i n g Co., Sun Valley, Calif. 7. Omni Tech Chlori-Kit; Omni Tech Inc., Santa Monica, Calif. 8. American Monitor Chloride; American Monitor Corporation, Indianapolis, Ind. All the above kits are advertised in various publications and are sold in Canada. We purchased them f r o m local sales representatives and distributors or directly from the m a n u f a c t u r e r s . The instructions of the various m a n u f a c t u r e r s were rigidly followed in using the kits. The accompanying literature was assessed in accordance with the recommendations of the American Association of Clinical Chemists (2) and our r a t i n g system (3). Six of the 8 kits for chloride are based on Schales & Schales method (4) in which a color complex is formed at the end point with excess mercuric n i t r a t e and s-diphenyl carbazone in an acid medium. The two exceptions are Harleco and American Monitor kits which are based on thiocyanate and ferric ion complex color formation in the presence of mercuric ion as described in the work of Hamilton (5) and of Schoenfield and Lewellen (6), respectively. Both of these kits employ a colorimetric readout. The Uni-Tech and Omni Tech kits also utilize a colorimetric readout for the Schales & Schales reaction. The Omni Tech kit requires a f i l t r a t i o n step to remove protein. Sigma, Medi-Chem, Oxford and E. M. test kits are direct t i t r i m e t r i c methods. A Mettler Gravimetric T i t r a t o r with Micro Reagent Dispenser (Mettler DP16ON & D P l l ) was used to c a r r y out the titrations for these kits. This is much superior to the conventional volumetric technique (7) as it is possible to detect very small changes in t i t r a n t by recording the weight change. Thus the error normally introduced by t i t r i m e t r y would not be evident in these results.
216
KIM, WADDELL & LOGAN
As a reference procedure the AutoAnalyzer N-5b procedure of Skeggs
(8) was used. The College of American Pathologists' Certified S t a n d a r d was taken as a p r i m a r y s t a n d a r d and the reagents and working s t a n d a r d s were prepared f r o m reagent grade chemicals in this laboratory. F o r 2 y e a r s which includes this evaluation period, we have participated as a reference laboratory in the National Reference L a b o r a t o r y Network for the Proficiency Testing P r o g r a m conducted by the National Center for Disease Control, Atlanta, Ga. T h r o u g h o u t our participation, data by this automated chloride reference procedure produced " A " grade ratings. A method similar to t h a t of B a r n e t t (9) was followed in evaluating the kits. The day-to-day reproducibility of each was determined for a total of 10 days using lyophilized commercial control sera containing three d i f f e r e n t levels of chloride. Data were collected a f t e r sufficient familiarization with the various kit methods. The s t a n d a r d deviation and coefficient of variation were then calculated. In the patient sera study we compared results by the kits with those by the reference procedure. The bias and Student's statistical 't' value of each kit method when compared to the reference procedure were obtained on 40 patient sera. Results were also analyzed to determine the incidence of false clinical impressions created by the data based on B a r n e t t ' s criteria of statistical and medical significance (10). Recovery experiments were performed a f t e r adding aqueous chloride stock s t a n d a r d in three d i f f e r e n t levels to a low normal pool of patient sera. This base pool value had been established by the reference method. Determinations were made in triplicate on the base pool and on sera with the three levels of added chloride and percentage recoveries were calculated. The costs of the various kits and reagents were compared on a 100-test basis to determine whether they bore any relation to the constituents supplied in the units or to their performance.
RESULTS AND DISCUSSION
The adequacy of the literature provided with the kits by m a n u f a c t u r e r s in shown in Table 1. The four kits based on the t i t r i m e t r i c principle are scored out of 13 r a t h e r than 15 since two of the points are applicable only to colorimetric tests. The table indicates t h a t most of the kit manufacturers provided incomplete information and no comparison data as to expected kit performances. Such data should be produced in their research and development programs. F r o m our experience, we would recommend t h a t m a n u f a c t u r e r s f i r s t obtain approval by field trial in a reputable institution where precision, accuracy and stability of the products are validated before marketing. A scheme similar to t h a t
CHLORIDE DIAGNOSTIC KITS TABLE
217
1
COMPARISON OF THE ADEQUACY OF THE LITERATURE SUPPLIED WITH THE VARIOUS KITS BY AN ARBITRARY SCORING SYSTEM
Titrimetric Rating Factors References of the methods . . . . . . . . . . Principle of the t e s t s . Normal range by the kits . . . . . . . . . . . . . . . Accuracy and precision Composition of reagents . . . . . . . . . . . . Calibration and linearity . . . . . . . . . . . Storage and stability data . . . . . . . . . . . . . . Recovery data in detail . . . . . . . . . . . . . Comparative statistics Information on blank, etc . . . . . . . . . . Supplied standard and data . . . . . . . . . . . . . . Information for instruments . . . . . . . Formula for calculation . . . . . . . . . . . . . Adequate notes and precautions . . . . . . . . Data to validate the test . . . . . . . . . . . . . . . Overall Total Score
Sigma
MediChem
Colorimetric
Oxford E. Merck
Uni- Omni American Harleco Tech Tech Monitor
1 1
1 1
I 1
1 l
1 1
1 1
1 1
1 1
] 0
1 0
I 0
1 0
0 0
l 0
1 0
1/2 0
1 1/2
1/2
1
1/2
1/2
I
i/2
1
1/2
0
0 0
1/2
0
1
0
0 0
0 0
0 0
0 0
n/a
1
1
1
1
1
1
1
1
1
n/a
i
I
i
I
i
1
I
I
I
I
1
0
1
1
0
0 0
0 0
0 0
n/a
n/a
I
i
n/a
n/a
1
1
n/a 1 n/a
1/2
1/2
1
i
1/2
1/2
1
1
1
1/2
1/2
1
0
0
0
0
0
0
0
0
8 1/2
7 1/2
8
6 1/2
8 1/2
9
10
9
followed for new pharmaceutical products used in therapy might well be practised for diagnostic kits. The day-to-day reproducibility of the kits and reference procedures is given in Table 2, for 3 different concentration levels of chloride in commercial lyophilized sera. The mean values of ten determinations and coefficients of variation have been calculated. The criteria for acceptability of precision are the allowable limits of error (A.L.E.) suggested by Tonks (11). Thus the data in this table indicate that twice the coefficient of variation exceeds the suggested A.L.E. for chloride, i.e., ± 4?6 for Sigma, Uni-Tech, Omni Tech and American Monitor kits at all three levels, Medi-Chem at the medium and high levels, Oxford
218
KIM, WADDELL
& LOGAN
TABLE 2 DAY-TO-DAY REPRODUCIBILITY OF THE KITS AND THE COLORIMETRIC REFERENCE PROCEDURE FOR THREE CONCENTRATION LEVELS OF CHLORIDE IN CONTROL SERA Low K i t or M e t h o d Reference-AA ......... Sigma ............... Medi-Chem ........... Oxford ............... E. M e r c k . . . . . . . . . . . . Harleco . . . . . . . . . . . . . . Uni-Tech ............ Omni Tech ........... American Monitor ....
Intermediate
Mean mEq/1
C.V. (4- %)
Mean mEq/1
C.V. (4- %)
91.1 93.4 91.8 92.3 92.2 90.6 98.2 92.8 89.8
1.59 2.72* 1.76 2.94* 3.02" 2.28* 6.22* 2.68* 4.32*
99.4 103.5 102.8 101.5 102.0 99.7 106.1 101.0 96.3
1.18 2.63* 2.19" 0.70 2.36" 1.71 4.13" 3.93* 3.79*
High Mean mEq/1 110.1 113.2 112.4 111.0 111.6 109.0 121.7 109.4 103.1
C.V. (4- %) 0.9 2.7* 2.30* 1.95 1.75 2.25* 11.5" 4.16" 2.26*
*Outside t h e allowable limits of error
at the low level, E. Merck at low and medium levels and Harleco at low and high levels. Therefore all the kits exceeded allowable limits of precision at one or more levels. Accuracy and reproducibility are also assessed by plotting the mean ± 3 S.D. values of the kit and reference procedures as shown in Fig. 1. This figure shows t h a t some of the kits, in addition to very poor precision, produced mean values considerably offset f r o m the reference values. F r o m these data and the figure, it is evident t h a t the mean values obtained for the lyophilized sera by the Sigma and Uni-Tech kits are too high while those obtained by the American Monitor kit are lower t h a n the reference values. Poor precision for a number of kits, especially Uni-Tech and Omni Tech is also evident. Comparative studies of data on 40 patient sera have been carried out by each of the methods over a period of 6 to 8 weeks. Three samples of p a t i e n t sera were analyzed on each test day. The results are compared with those of the reference procedure in Table 3. The data indicate t h a t a statistically significant bias exists for Medi-Chem, Oxford, Omni Tech, Uni-Tech and American Monitor kits according to Student's 't' test. The question arises as to how accurate a chloride method should be for use in clinical diagnosis and in therapeutic m a n a g e m e n t of fluid and electrolyte balance in critically ill patients. In order to examine this question we added two more criteria, the frequency of large discrepancies, L.D., and false decision possibilities, F.D., to Table 3. We have designated as a large discrepancy a bias between the kit and reference values which
CHLORIDE meq
DIAGNOSTIC
KITS
219
/L LOW
MEDIUM
HIGH
150 R: S: M: OX: EhA: 30
Reference (A.A.) Sigma Medi-Chem. OxFord E. ~ e r c k H: Harieco U: Uni-Tech Ore: O m n i Tech Am: American Monitor
im
1°1*Ii iiiiliiii!liiit.."!iEililjiiiiiiiiil EM
R ~M Ehl 90
2
Am U
U Am 70
F i g . 1. T h e d a y - t o - d a y r e p r o d u c i b i l i t y for the determination of chloride.
of the various
TABLE
kit and reference
procedures
3
BIAS OF KIT METHODS VERSUS REFERENCE PROCEDURE No. of D i s c r e p a n c i e s > 5 mEq/l N a m e of K i t Sigma ................... Medi-Chem .............. Oxford . . . . . . . . . . . . . . . . . . . E. M e r c k . . . . . . . . . . . . . . . . Harleco . . . . . . . . . . . . . . . . . . Uni-Tech ................. Omni Tech ............... American Monitor ........
No. of Sera
Bias (mEq/1)
40 40 39 40 40 32 40 40
- 1.2 - 1.7 -0.8 +0.05 +0.03 - 8.2 -3.8 +4.3
*Statistically b u t n o t necessarily clinically s i g n i f i c a n t bias
"t" 1.4 3.9* 2.4* 0.1 0.1 6.7* 4.1" 4.2*
L.D. 14 3 4 0 7 23 12 18
F.D. 5 1
2 0 3 17 9 7
220
KIM, WADDELL
& LOGAN
TABLE 4 THE RECOVERIES OF CHLORIDE ADDED TO POOLED SERUM AS DETERMINED IN TRIPLICATE BY THE KIT AND REFERENCE PROCEDURES Base Pooled Serum K i t or Method Reference AA Sigma . . . . . . . Medi-Chem.. Oxford . . . . . . . E. Merck . . . . Harleco . . . . . . Omni Tech .. American Monitor...
F o u n d Value mEq/1
1st Level Added (10 m E q / 1 ) Found mEq/1
Recovery %
2nd Level Added (20 m E q / 1 ) Found mEq/1
3rd Level Added (30 m E q / 1 )
Recovery %
Found mEq/1
Recovery %
84.5 88.5 84.6 86.2 87.5 85.0 89.0
94.5 97.0 94.0 95.3 98.5 94.5 98.0
100 85 94 91 110 95 90
105.5 107.0 105.0 106.0 110.0 106.5 110.0
105 92.5 102 99 112.5 107.5 105
115.5 117.5 114.0 117.5 117.0 117.0 118.0
103.3 96.7 98 104.3 98.3 106.7 96.7
86.0
93.5
75
104.0
90
111.5
85
is more than three times the value of the medically significant standard deviation suggested by the Sub Committee on Criteria of Medical Usefulness of the College of American Pathologists (10). If the kit produces large discrepancies which lead to false interpretation more often than once in t w e n t y instances, particularly at the medically significant decision levels of 90 and 110 m E q / 1 as designated by F.D. in the Table, then it is obvious that the kit will not be suitable for use in laboratory medicine. Using these criteria it will be seen that the Sigma, Harleco, Omni Tech, Uni-Tech and American Monitor kits could frequently yield false inf o r m a t i o n to the physician. Recovery studies are tabulated in Table 4. Data were not obtained for the Uni-Tech kit. In general the remainder of the kits yielded quite acceptable recoveries. Mean recoveries were considered acceptable between 90 and 110% provided no individual recovery was less than 85% or g r e a t e r than 1 1 5 ~ . The single exception was the American Monitor kit in which the recovery w a s less than 90%. No instability was found in any of the kit reagents during the test period. The costs of the various kits when compared on a 100-test basis seemed to bear no relation to their constitution of reagents. The cost of the kits based on the titrimetric principle was less, v a r y i n g f r o m $10 to $16.50 p e r 100 tests, while the colorimetric type kits cost f r o m $17.50 to $30 f o r the same n u m b e r of tests.
CHLORIDE DIAGNOSTIC KITS
221
REFERENCES 1. LOGAN, J. E., RENTON, H. M., and ALLEN, R. H. The use of diagnostic kits and reagents in the clinical laboratory in Canada. Clin. Biochem. 3, 81-89 {1970). ~. A M E R I C A N ASSOCIATION of CLINICAL CHEMISTS, Committee on Standards and Controls. AACC policy regarding reagent sets and kits. Clin. Chem. 12, 43-44 (1966). 3. KIM, E. K., WADDELL, L. D., S U N D E R L A N D , M. L. E., and LOGAN, J. E. Observations on diagnostic kits for the determination of uric acid. Clin. Biochem. 4, 279-286 (1971). ~. SCHALE~, 0. and SCHALES, S. S. A simple and accurate method for the determination of chloride in biological fluids. J. Biol. Chem. 140, 879 (1941). 5. HAMILTON, R. H. Temple University, Philadelphia, Pa. Harleco product insert. (Paper submitted for publication) 6. S C H O E N F E L D , R. G. and L E W E L L E N , C. J. A colorimetric method for determination of serum chloride. Clin. Chem. 10, 533-539 (1964). 7. THOBURN, J. M. Weight titrations revived. J. Chem. Ed. 36, 616-618 (1959). 8. T E C H N I C O N I N S T R U M E N T S CORPORATION. "Chloride" Technicon Laboratory Method, File N-5b, Ardsley, New York (1965). 9. B A R N E T T , R. N. A scheme for the comparison of quantitative methods. Am. J. Clin. Path. 43, 562-569 (1965). 10. BARNETT, R. N. Medical significance of laboratory results. Am. J. Clin. Path. 50, 671-676 (1968). 11. TONKS, D. B. Quality control in clinical laboratories. 1970 Ed., Diagnostic Reagents Div., Warner-Chilcott Laboratories Co. Ltd., Scarborough, Ontario (1970).
OBSERVATIONS ON DIAGNOSTIC KITS FOR THE DETERMINATION OF CHLORIDE
E. K. KIM, L. D. W A D D E L L and J. E. LOGAN
CIb~ical Laboratories, Canadia~ Communicable Disease CeTitre, Departme~t of Natio~al Health and Welfare, Ottawa, O~Ttario. (Received April 26, 1972)
SUM MARY
1. Diagnostic kits or reagent sets from eight m a n u f a c t u r e r s for the determination of chloride were assessed. 2. The kits were assessed for adequacy of literature supplied, the reproducibility of the procedures, the accuracy of results compared with those obtained by reference method, and for the recovery of chloride added to sera. 3. Only two kits were acceptable according to our criteria for a good laboratory test procedure. One kit just exceeded the allowable limits of error for precision, whereas another tended to produce some falsely elevated values. F o u r kits frequently yielded data for patient sera which could lead to incorrect clinical interpretations. 4. The cost analysis of the various kits showed no correlation between price and reagent constitution, particularly for the four kits based on the colorimetric principle.
T H E USE OF DIAGNOSTIC KITS AND REAGENTS in clinical laboratories for the determination of chloride was found to be extensive on both a routine and standby basis (1). In a survey conducted by Logan et al (1) 50 or 11.5~, of 435 laboratories across Canada indicated t h a t they were using chloride kits. Thus information on the reliability of those kits which are available on the Canadian m a r k e t should be useful. The results of an assessment of eight kits tested in this laboratory are presented.
Correspondence: Dr. J. E. Logan.
CHLORIDE DIAGNOSTIC KITS
215
METHODS The chloride kits which we have studied are listed as follows: 1. Sigma Chloride # 8 3 0 ; Sigma Chemical Co., St. Louis, Mo. 2. Medi-Chem Chloride; Medi-Chem Inc., S a n t a Monica, Calif. 3. Oxford Chloride Reagent # 6 3 0 ; Oxford Laboratories, San Mateo, Calif. 4. E.M.-Test Chloride; E. Merck A.G., Darmstadt, Germany, Canadian distributor - - B r i n k m a n n I n s t r u m e n t s (Canada) Ltd., Rexdale, Ontario. 5. Harleco Chloride; Harleco, Philadelphia, Pa. 6. Uni-Tech Chlori-Pak; Uni-Tech Chemical M a n u f a c t u r i n g Co., Sun Valley, Calif. 7. Omni Tech Chlori-Kit; Omni Tech Inc., Santa Monica, Calif. 8. American Monitor Chloride; American Monitor Corporation, Indianapolis, Ind. All the above kits are advertised in various publications and are sold in Canada. We purchased them f r o m local sales representatives and distributors or directly from the m a n u f a c t u r e r s . The instructions of the various m a n u f a c t u r e r s were rigidly followed in using the kits. The accompanying literature was assessed in accordance with the recommendations of the American Association of Clinical Chemists (2) and our r a t i n g system (3). Six of the 8 kits for chloride are based on Schales & Schales method (4) in which a color complex is formed at the end point with excess mercuric n i t r a t e and s-diphenyl carbazone in an acid medium. The two exceptions are Harleco and American Monitor kits which are based on thiocyanate and ferric ion complex color formation in the presence of mercuric ion as described in the work of Hamilton (5) and of Schoenfield and Lewellen (6), respectively. Both of these kits employ a colorimetric readout. The Uni-Tech and Omni Tech kits also utilize a colorimetric readout for the Schales & Schales reaction. The Omni Tech kit requires a f i l t r a t i o n step to remove protein. Sigma, Medi-Chem, Oxford and E. M. test kits are direct t i t r i m e t r i c methods. A Mettler Gravimetric T i t r a t o r with Micro Reagent Dispenser (Mettler DP16ON & D P l l ) was used to c a r r y out the titrations for these kits. This is much superior to the conventional volumetric technique (7) as it is possible to detect very small changes in t i t r a n t by recording the weight change. Thus the error normally introduced by t i t r i m e t r y would not be evident in these results.
216
KIM, WADDELL & LOGAN
As a reference procedure the AutoAnalyzer N-5b procedure of Skeggs
(8) was used. The College of American Pathologists' Certified S t a n d a r d was taken as a p r i m a r y s t a n d a r d and the reagents and working s t a n d a r d s were prepared f r o m reagent grade chemicals in this laboratory. F o r 2 y e a r s which includes this evaluation period, we have participated as a reference laboratory in the National Reference L a b o r a t o r y Network for the Proficiency Testing P r o g r a m conducted by the National Center for Disease Control, Atlanta, Ga. T h r o u g h o u t our participation, data by this automated chloride reference procedure produced " A " grade ratings. A method similar to t h a t of B a r n e t t (9) was followed in evaluating the kits. The day-to-day reproducibility of each was determined for a total of 10 days using lyophilized commercial control sera containing three d i f f e r e n t levels of chloride. Data were collected a f t e r sufficient familiarization with the various kit methods. The s t a n d a r d deviation and coefficient of variation were then calculated. In the patient sera study we compared results by the kits with those by the reference procedure. The bias and Student's statistical 't' value of each kit method when compared to the reference procedure were obtained on 40 patient sera. Results were also analyzed to determine the incidence of false clinical impressions created by the data based on B a r n e t t ' s criteria of statistical and medical significance (10). Recovery experiments were performed a f t e r adding aqueous chloride stock s t a n d a r d in three d i f f e r e n t levels to a low normal pool of patient sera. This base pool value had been established by the reference method. Determinations were made in triplicate on the base pool and on sera with the three levels of added chloride and percentage recoveries were calculated. The costs of the various kits and reagents were compared on a 100-test basis to determine whether they bore any relation to the constituents supplied in the units or to their performance.
RESULTS AND DISCUSSION
The adequacy of the literature provided with the kits by m a n u f a c t u r e r s in shown in Table 1. The four kits based on the t i t r i m e t r i c principle are scored out of 13 r a t h e r than 15 since two of the points are applicable only to colorimetric tests. The table indicates t h a t most of the kit manufacturers provided incomplete information and no comparison data as to expected kit performances. Such data should be produced in their research and development programs. F r o m our experience, we would recommend t h a t m a n u f a c t u r e r s f i r s t obtain approval by field trial in a reputable institution where precision, accuracy and stability of the products are validated before marketing. A scheme similar to t h a t
CHLORIDE DIAGNOSTIC KITS TABLE
217
1
COMPARISON OF THE ADEQUACY OF THE LITERATURE SUPPLIED WITH THE VARIOUS KITS BY AN ARBITRARY SCORING SYSTEM
Titrimetric Rating Factors References of the methods . . . . . . . . . . Principle of the t e s t s . Normal range by the kits . . . . . . . . . . . . . . . Accuracy and precision Composition of reagents . . . . . . . . . . . . Calibration and linearity . . . . . . . . . . . Storage and stability data . . . . . . . . . . . . . . Recovery data in detail . . . . . . . . . . . . . Comparative statistics Information on blank, etc . . . . . . . . . . Supplied standard and data . . . . . . . . . . . . . . Information for instruments . . . . . . . Formula for calculation . . . . . . . . . . . . . Adequate notes and precautions . . . . . . . . Data to validate the test . . . . . . . . . . . . . . . Overall Total Score
Sigma
MediChem
Colorimetric
Oxford E. Merck
Uni- Omni American Harleco Tech Tech Monitor
1 1
1 1
I 1
1 l
1 1
1 1
1 1
1 1
] 0
1 0
I 0
1 0
0 0
l 0
1 0
1/2 0
1 1/2
1/2
1
1/2
1/2
I
i/2
1
1/2
0
0 0
1/2
0
1
0
0 0
0 0
0 0
0 0
n/a
1
1
1
1
1
1
1
1
1
n/a
i
I
i
I
i
1
I
I
I
I
1
0
1
1
0
0 0
0 0
0 0
n/a
n/a
I
i
n/a
n/a
1
1
n/a 1 n/a
1/2
1/2
1
i
1/2
1/2
1
1
1
1/2
1/2
1
0
0
0
0
0
0
0
0
8 1/2
7 1/2
8
6 1/2
8 1/2
9
10
9
followed for new pharmaceutical products used in therapy might well be practised for diagnostic kits. The day-to-day reproducibility of the kits and reference procedures is given in Table 2, for 3 different concentration levels of chloride in commercial lyophilized sera. The mean values of ten determinations and coefficients of variation have been calculated. The criteria for acceptability of precision are the allowable limits of error (A.L.E.) suggested by Tonks (11). Thus the data in this table indicate that twice the coefficient of variation exceeds the suggested A.L.E. for chloride, i.e., ± 4?6 for Sigma, Uni-Tech, Omni Tech and American Monitor kits at all three levels, Medi-Chem at the medium and high levels, Oxford
218
KIM, WADDELL
& LOGAN
TABLE 2 DAY-TO-DAY REPRODUCIBILITY OF THE KITS AND THE COLORIMETRIC REFERENCE PROCEDURE FOR THREE CONCENTRATION LEVELS OF CHLORIDE IN CONTROL SERA Low K i t or M e t h o d Reference-AA ......... Sigma ............... Medi-Chem ........... Oxford ............... E. M e r c k . . . . . . . . . . . . Harleco . . . . . . . . . . . . . . Uni-Tech ............ Omni Tech ........... American Monitor ....
Intermediate
Mean mEq/1
C.V. (4- %)
Mean mEq/1
C.V. (4- %)
91.1 93.4 91.8 92.3 92.2 90.6 98.2 92.8 89.8
1.59 2.72* 1.76 2.94* 3.02" 2.28* 6.22* 2.68* 4.32*
99.4 103.5 102.8 101.5 102.0 99.7 106.1 101.0 96.3
1.18 2.63* 2.19" 0.70 2.36" 1.71 4.13" 3.93* 3.79*
High Mean mEq/1 110.1 113.2 112.4 111.0 111.6 109.0 121.7 109.4 103.1
C.V. (4- %) 0.9 2.7* 2.30* 1.95 1.75 2.25* 11.5" 4.16" 2.26*
*Outside t h e allowable limits of error
at the low level, E. Merck at low and medium levels and Harleco at low and high levels. Therefore all the kits exceeded allowable limits of precision at one or more levels. Accuracy and reproducibility are also assessed by plotting the mean ± 3 S.D. values of the kit and reference procedures as shown in Fig. 1. This figure shows t h a t some of the kits, in addition to very poor precision, produced mean values considerably offset f r o m the reference values. F r o m these data and the figure, it is evident t h a t the mean values obtained for the lyophilized sera by the Sigma and Uni-Tech kits are too high while those obtained by the American Monitor kit are lower t h a n the reference values. Poor precision for a number of kits, especially Uni-Tech and Omni Tech is also evident. Comparative studies of data on 40 patient sera have been carried out by each of the methods over a period of 6 to 8 weeks. Three samples of p a t i e n t sera were analyzed on each test day. The results are compared with those of the reference procedure in Table 3. The data indicate t h a t a statistically significant bias exists for Medi-Chem, Oxford, Omni Tech, Uni-Tech and American Monitor kits according to Student's 't' test. The question arises as to how accurate a chloride method should be for use in clinical diagnosis and in therapeutic m a n a g e m e n t of fluid and electrolyte balance in critically ill patients. In order to examine this question we added two more criteria, the frequency of large discrepancies, L.D., and false decision possibilities, F.D., to Table 3. We have designated as a large discrepancy a bias between the kit and reference values which
CHLORIDE meq
DIAGNOSTIC
KITS
219
/L LOW
MEDIUM
HIGH
150 R: S: M: OX: EhA: 30
Reference (A.A.) Sigma Medi-Chem. OxFord E. ~ e r c k H: Harieco U: Uni-Tech Ore: O m n i Tech Am: American Monitor
im
1°1*Ii iiiiliiii!liiit.."!iEililjiiiiiiiiil EM
R ~M Ehl 90
2
Am U
U Am 70
F i g . 1. T h e d a y - t o - d a y r e p r o d u c i b i l i t y for the determination of chloride.
of the various
TABLE
kit and reference
procedures
3
BIAS OF KIT METHODS VERSUS REFERENCE PROCEDURE No. of D i s c r e p a n c i e s > 5 mEq/l N a m e of K i t Sigma ................... Medi-Chem .............. Oxford . . . . . . . . . . . . . . . . . . . E. M e r c k . . . . . . . . . . . . . . . . Harleco . . . . . . . . . . . . . . . . . . Uni-Tech ................. Omni Tech ............... American Monitor ........
No. of Sera
Bias (mEq/1)
40 40 39 40 40 32 40 40
- 1.2 - 1.7 -0.8 +0.05 +0.03 - 8.2 -3.8 +4.3
*Statistically b u t n o t necessarily clinically s i g n i f i c a n t bias
"t" 1.4 3.9* 2.4* 0.1 0.1 6.7* 4.1" 4.2*
L.D. 14 3 4 0 7 23 12 18
F.D. 5 1
2 0 3 17 9 7
220
KIM, WADDELL
& LOGAN
TABLE 4 THE RECOVERIES OF CHLORIDE ADDED TO POOLED SERUM AS DETERMINED IN TRIPLICATE BY THE KIT AND REFERENCE PROCEDURES Base Pooled Serum K i t or Method Reference AA Sigma . . . . . . . Medi-Chem.. Oxford . . . . . . . E. Merck . . . . Harleco . . . . . . Omni Tech .. American Monitor...
F o u n d Value mEq/1
1st Level Added (10 m E q / 1 ) Found mEq/1
Recovery %
2nd Level Added (20 m E q / 1 ) Found mEq/1
3rd Level Added (30 m E q / 1 )
Recovery %
Found mEq/1
Recovery %
84.5 88.5 84.6 86.2 87.5 85.0 89.0
94.5 97.0 94.0 95.3 98.5 94.5 98.0
100 85 94 91 110 95 90
105.5 107.0 105.0 106.0 110.0 106.5 110.0
105 92.5 102 99 112.5 107.5 105
115.5 117.5 114.0 117.5 117.0 117.0 118.0
103.3 96.7 98 104.3 98.3 106.7 96.7
86.0
93.5
75
104.0
90
111.5
85
is more than three times the value of the medically significant standard deviation suggested by the Sub Committee on Criteria of Medical Usefulness of the College of American Pathologists (10). If the kit produces large discrepancies which lead to false interpretation more often than once in t w e n t y instances, particularly at the medically significant decision levels of 90 and 110 m E q / 1 as designated by F.D. in the Table, then it is obvious that the kit will not be suitable for use in laboratory medicine. Using these criteria it will be seen that the Sigma, Harleco, Omni Tech, Uni-Tech and American Monitor kits could frequently yield false inf o r m a t i o n to the physician. Recovery studies are tabulated in Table 4. Data were not obtained for the Uni-Tech kit. In general the remainder of the kits yielded quite acceptable recoveries. Mean recoveries were considered acceptable between 90 and 110% provided no individual recovery was less than 85% or g r e a t e r than 1 1 5 ~ . The single exception was the American Monitor kit in which the recovery w a s less than 90%. No instability was found in any of the kit reagents during the test period. The costs of the various kits when compared on a 100-test basis seemed to bear no relation to their constitution of reagents. The cost of the kits based on the titrimetric principle was less, v a r y i n g f r o m $10 to $16.50 p e r 100 tests, while the colorimetric type kits cost f r o m $17.50 to $30 f o r the same n u m b e r of tests.
CHLORIDE DIAGNOSTIC KITS
221
REFERENCES 1. LOGAN, J. E., RENTON, H. M., and ALLEN, R. H. The use of diagnostic kits and reagents in the clinical laboratory in Canada. Clin. Biochem. 3, 81-89 {1970). ~. A M E R I C A N ASSOCIATION of CLINICAL CHEMISTS, Committee on Standards and Controls. AACC policy regarding reagent sets and kits. Clin. Chem. 12, 43-44 (1966). 3. KIM, E. K., WADDELL, L. D., S U N D E R L A N D , M. L. E., and LOGAN, J. E. Observations on diagnostic kits for the determination of uric acid. Clin. Biochem. 4, 279-286 (1971). ~. SCHALE~, 0. and SCHALES, S. S. A simple and accurate method for the determination of chloride in biological fluids. J. Biol. Chem. 140, 879 (1941). 5. HAMILTON, R. H. Temple University, Philadelphia, Pa. Harleco product insert. (Paper submitted for publication) 6. S C H O E N F E L D , R. G. and L E W E L L E N , C. J. A colorimetric method for determination of serum chloride. Clin. Chem. 10, 533-539 (1964). 7. THOBURN, J. M. Weight titrations revived. J. Chem. Ed. 36, 616-618 (1959). 8. T E C H N I C O N I N S T R U M E N T S CORPORATION. "Chloride" Technicon Laboratory Method, File N-5b, Ardsley, New York (1965). 9. B A R N E T T , R. N. A scheme for the comparison of quantitative methods. Am. J. Clin. Path. 43, 562-569 (1965). 10. BARNETT, R. N. Medical significance of laboratory results. Am. J. Clin. Path. 50, 671-676 (1968). 11. TONKS, D. B. Quality control in clinical laboratories. 1970 Ed., Diagnostic Reagents Div., Warner-Chilcott Laboratories Co. Ltd., Scarborough, Ontario (1970).