- Email: [email protected]
Accuracy of bedside point of care testing in critical emergency department patients
YAJEM-56963; No of Pages 4 American Journal of Emergency Medicine xxx (2017) xxx–xxx
Contents lists available at ScienceDirect
American Journal of Emergency Medicine journal homepage: www.elsevier.com/locate/ajem
Accuracy of bedside point of care testing in critical emergency department patients☆ Braden W. McIntosh, MD a, Jerina Vasek, RN b, Maria Taylor, RN a, Deborah Le Blanc, RN a, Henry C. Thode, PhD a, Adam J. Singer, MD a,⁎ a b
Department of Emergency Medicine, Stony Brook University, Stony Brook, NY, United States Department of Clinical Pathology, Stony Brook University, Stony Brook, NY, United States
a r t i c l e
i n f o
Article history: Received 21 August 2017 Received in revised form 12 September 2017 Accepted 13 September 2017 Available online xxxx Keywords: Point-of-care testing Accuracy Emergency department
a b s t r a c t Background: Point-of-care (POC) testing reduces laboratory turn-around having the potential to improve timely diagnosis and management. We compared the accuracy of nurse performed POC and core laboratory testing and determined whether deviations between the two were clinically meaningful. Methods: We performed a prospective, observational study on a convenience sample of 50 critical care ED patients in whom a POC chemistry and hematocrit was ordered. Blood samples were divided into 2 aliquots; one sample was tested by the treating nurse using a handheld POC device and the other sample was tested in the core laboratory. Paired comparisons of test results were performed using Pearson's correlation coefficients, Lin concordance coefficients, and Bland Altman plots. Results: Mean patient age was 67, 50% were male, 82% were admitted. Pearson's correlation and Lin concordance coefficients were excellent (0.84–1.00) for all 8 analytes. Mean (95%CI) paired differences between POC and core laboratory measurements were Na+ 0.30 (−0.22 to 0.82) mmol/L, K+ − 0.12 (−0.14 to - 0.09) mmol/L, Cl− 2.10 (1.41 to 2.78) mmol/L, TCO2–1.68 (−2.06 to −1.30) mmol/L, glucose 2.46 (1.46 to 3.46) mg/dL, BUN, 1.69 (0.95 to 2.42) mg/dL, creatinine 0.13 (0.08 to 0.17) mg/dL, and hematocrit −0.39 (−0.93 to 0.15) %. In 3 of 400 measurements, the difference between POC and core lab exceeded the maximal clinically acceptable deviation based on physician surveys. Conclusions: Bedside POC by ED nurses is reliable and accurate and does not deviate significantly from core laboratory testing by trained technicians. © 2017 Elsevier Inc. All rights reserved.
1. Introduction A significant proportion of clinical decisions are made based on the results of laboratory testing [1]. While all analytes are important, acute disturbances in potassium (K+) levels have the greatest potential to cause patient harm. Thus, use of analytical methods that minimize errors in measuring K+ levels is of major concern. A large number of rapid point-of-care (POC) tests and devices are now available, which have the potential to significantly reduce test turn around times and emergency department (ED) length of stay [2,3]. However, some health care practitioners may be skeptical about the results of bedside POC testing, and hesitate acting on them without confirmation using core laboratory testing. Rapid measurement of analytes using whole blood specimens is fast and convenient. However, a major disadvantage of using whole blood
☆ Presented at the Annual Meeting of the Society for Academic Emergency Medicine, May 2017, New Orleans, LA. ⁎ Corresponding author at: Department of Emergency Medicine, HSC-L4-080, 8350 SUNY, Stony Brook, NY 11794-8350, United States. E-mail address: [email protected] (A.J. Singer).
samples is the inability to detect hemolysis. A study of 610 blood samples found mild hemolysis in 18%, moderate hemolysis in 3.6%, and severe hemolysis in 0.4% [4]. In this study, Hawkins estimated that the difference in K+ measurement attributable to hemolysis was N0.5 mmol/L in 8% of the samples. A prior study by the same author estimated a hemolysis rate of 3.4% [5]. As a result, when measuring analytes such as K+ using whole blood samples, there is a risk of obtaining an artificially elevated K+ level that can lead to misdiagnosis and inappropriate treatment. Mechanical forces leading to hemolysis include use of tourniquets, small gauge needles, and fist clenching to name but a few. Other causes of spuriously elevated K+ levels (pseudohyperkalemia) include excessive numbers of platelets or neoplastic WBCs. These same factors that lead to pseudohyperkalemia may also sometimes mask hypokalemia [6]. The objective of the present study was to determine the reliability of a whole blood based POC device in measuring eight commonly measured analytes in critically ill or injured emergency department patients. We hypothesized that the correlations between POC and core lab results would be excellent and that the bias (the difference between the two measurements) would not be clinically meaningful.
http://dx.doi.org/10.1016/j.ajem.2017.09.018 0735-6757/© 2017 Elsevier Inc. All rights reserved.
Please cite this article as: McIntosh BW, et al, Accuracy of bedside point of care testing in critical emergency department patients, American Journal of Emergency Medicine (2017), http://dx.doi.org/10.1016/j.ajem.2017.09.018
2
B.W. McIntosh et al. / American Journal of Emergency Medicine xxx (2017) xxx–xxx
2. Methods 2.1. Study design We conducted a prospective, observational, study to test the study hypothesis. Waiver of informed consent was obtained from our Institutional Review Board. 2.2. Patients and setting The study was conducted at a tertiary care, suburban, academic medical center with an annual ED census of approximately 110,000. Fifty consecutive critically ill or injured adult patients in whom a physician ordered a POC (i-STAT, Abbott Point of Care, Princeton, NJ) limited metabolic panel and hematocrit level (requiring a CHEM8+ cartridge) were enrolled. 2.3. Blood samples and measurements Venous blood samples were collected from patients using a vacutainer and placed in blood collection glass tubes containing lithium heparin. The blood samples were split into two aliquots, one of which was analyzed by a trained clinical nurse at the bedside using the POC device. The other sample was transported to the core lab for further testing. The remaining aliquot was centrifuged at 3000 ×g for 15 min at 4 °C to harvest plasma. The plasma sample was then analyzed by a trained laboratory technician using core laboratory devices (Roche COBAS 6000 [Roche Diagnostics USA, Indianapolis, IN] and Sysmex XN [Sysmex America, Inc., Licolnshire, IL] for metabolic panel and hemoglobin respectively). 2.4. Physician surveys A convenience sample of 25 emergency physicians were asked to indicate the maximal clinically acceptable deviation between POC and core laboratory results that would maintain their confidence in the POC results in ED patients for each of the eight analytes (Na +, K +, Cl−, TCO2, BUN, creatinine, glucose, and hematocrit). 2.5. Data analysis Continuous data are summarized as means and 95% confidence intervals (CI). Binary data are summarized as numbers and percentages frequency of occurrence. Regression plots and Pearson's correlation coefficients were used to measure the agreement between paired POC and core laboratory results. The differences (bias) between paired measurements were then determined by Bland-Altman plot analysis [7]. Lin's concordance correlation (pc) that describes the relationship between paired measurements was also used [8]. Unlike the Pearson correlation, which measures the strength of a linear relationship but may not pass through the origin and have a slope not equal to unity, pc compares agreement (between two sets of measurements) by assessing the variation from the 45° line through the origin. The number of test results in which the difference between paired samples was greater than the median minimal clinically acceptable deviation based on physician surveys was also determined. Assuming a correlation of 0.90 between pairs of measurements and a standard deviation (SD) of 0.6 for each type of measurement (POC or core lab), then the SD of the difference is 0.27. A sample size of 29 patients would be needed to have a confidence interval of the difference of ±0.1 around the point estimate. 3. Results Samples from 50 ED critical care patients were analyzed. Mean patient age (range) was 67 (24–97) years, 50% were male, 82% were
admitted, and of those, 30% went to the intensive care unit (ICU). General categories of patient chief complaints leading to an ED visit included 14 cardiac (chest pain or arrhythmia), 12 neurological (possible stroke or seizure), 11 traumatic (motor vehicle collision or fall), 5 altered mental status, and 5 miscellaneous. The Pearson's correlation coefficients between paired samples, one measured with the POC device and the other measured with the core laboratory device, ranged from 0.84 to 0.99 (Table 1), and were lowest for Cl− (0.89) and Na+ (0.84). Lin concordance coefficients ranged from 0.82 to 0.99 (Table 1), and were lowest for Cl− (0.82) and Na+ (0.84). The mean (range) paired differences between the POC and core laboratory samples were Na+ 0.30 (−6 to 4) mmol/L, K+ −0.12 (−0.40 to 0.15) mmol/L, Cl− 2.10 (−3 to 10) mmol/L, TCO2 1.68 (−4 to 2) mmol/ L, glucose 2.46 (−9 to 9) mg/dL, BUN, 1.69 (−4 to 9) mg/dL, creatinine 0.13 (−0.1 to 0.9) mg/dL, and hematocrit −0.39 (−7.2 to 4.2) % (Table 1). The Bland-Altman plots are presented in Fig. 1. The mean and median maximal clinically acceptable deviations between POC and core laboratory results that would not impact clinical diagnosis and management based on physician surveys is presented in Table 2. The number of test results in which the difference between POC and core laboratory results exceeded the median clinically acceptable difference were TCO2 none, Cl− none, K+ none, BUN one (27 vs. 18 mg/dL), creatinine one (5.5 vs. 4.6 mg/dL), glucose none, hematocrit one (35 vs. 42%), and Na+ one (133 vs. 139 mmol/L). Thus, only in 4 out of 400 paired samples did the difference exceed the clinically acceptable deviation. In all four cases, these differences did not alter clinical management. 4. Discussion In this observational study of 50 critically ill or injured ED patients, there was high to excellent correlation between paired measurements of POC and core laboratory tests for all eight of the tested analytes. Excellent correlations were seen both with the Pearson's and the Lin concordance tests. The bias between the two tests was generally small, with the 95% CIs falling within the maximal clinically acceptable deviations according to the physician surveys. In addition, the differences between the two tests rarely exceeded the maximal clinically acceptable deviation that would be considered to effect clinical diagnosis and management. These findings suggest that rapid beside POC testing using the iSTAT device can be used interchangeably with core laboratory testing. This should give clinicians confidence when making clinical decisions based on POC testing results alone. As always, whenever a laboratory result does not make clinical sense, it should be repeated. This is true for all laboratory tests, regardless of the device or platform used including POC and traditional core laboratory tests. Table 1 Agreement between point of care and core laboratory results. Pearson's correlation coefficient
Mean (95%CI) paired difference
Lin concordance coefficient
TCO2
0.94
0.86
CI−
0.89
K+
0.999
BUN
0.96
Creatinine
0.999
Glucose
0.999
Hematocrit
0.95
−1.68 −2.06–1.30 2.10 1.41–2.78 −0.12 −0.14–0.09 1.69 0.95–2.42 0.13 0.08–0.17 2.46 1.46–3.46 −0.39 −0.93–0.15 0.30 −0.22–0.82
Na
+
0.85
0.82 0.97 0.94 0.98 0.999 0.95 0.84
Please cite this article as: McIntosh BW, et al, Accuracy of bedside point of care testing in critical emergency department patients, American Journal of Emergency Medicine (2017), http://dx.doi.org/10.1016/j.ajem.2017.09.018
B.W. McIntosh et al. / American Journal of Emergency Medicine xxx (2017) xxx–xxx
Fig. 1. Bland Altman Plots for the eight analytes. Left column from top to bottom: TCO2, K+, creatinine, hematocrit. Right column from top to bottom: Cl−, BUN, glucose, Na+.
Please cite this article as: McIntosh BW, et al, Accuracy of bedside point of care testing in critical emergency department patients, American Journal of Emergency Medicine (2017), http://dx.doi.org/10.1016/j.ajem.2017.09.018
3
4
B.W. McIntosh et al. / American Journal of Emergency Medicine xxx (2017) xxx–xxx
Table 2 Surveyed physicians' maximal acceptable deviations between point-of-care and core laboratory results.
TCO2, mmol/L Cl−, mmol/L K+, mmol/L BUN, mmol/L Creatinine, mg/dL Glucose, mg/dL Hematocrit, % Na+, mmol/L
Mean
Median
3.7 9.5 0.5 7.1 0.4 11 4.3 4.2
4.0 10.0 0.5 8.0 0.5 10 4.0 5.0
A number of prior studies have evaluated whether rapid measurements of K+ using whole blood are as accurate as core laboratory testing using plasma samples [9-11]. In one study, discrepancies of up to 1 mmol/L were noted between whole blood and plasma measurements, especially when potassium levels were below 3 mmol/L. [9] Another study reported that nearly 1 in 3 hypokalemic samples was missed when whole blood was analyzed [10]. A recent study by Yildrim et al. assessed the reliability of a POC device (i-STAT) for the determination of K+, Na+, and Cl− whole blood concentrations in 98 cattle [12]. POC plasma concentrations of K+ (4.39 vs. 4.2 mmol/L; P b 0.001) and Cl− (100.30 vs. 99.4 mmol/L; P b 0.04) were higher than their concentration in core laboratory plasma. The mean bias (whole blood concentration-plasma concentration) in mmol/L was − 0.20 for K+, − 0.87 for Cl−, and − 0.16 for Na+. These differences do not appear to be clinically meaningful. A study by Leino et al. found that the blood gases, K+, Na+, Ca2+, glucose and lactate measured with stat POC and core laboratory analyzers were comparable with minimal bias. However, at low hemoglobin levels, the stat POC test over-diagnosed anemia [9]. In contrast, a Dutch study of 48 blood samples from patients undergoing cardiac surgery and another 42 blood samples from ICU patients found that there was an underestimation in hematocrit of up to 2.2% (P b 0.001) in ICU patients with hematocrits below 25% [10]. Finally an Indian study compared whole blood and plasma samples from ICU patients analyzed by arterial blood gas and core laboratory analyzers, respectively [11]. In this study the reported bias in K+ levels for whole blood samples was greater (95% levels of agreement between [LOA] − 1.0 and − 0.13 mmol/L) for K+ levels under 3.0 mmol/L than for higher K+ levels (mean difference − 0.2; 95% LOA -0.48 to 0.06). These studies suggest that the degree of concordance between POC and core laboratory measurements may not be uniform across the entire range of values. With all prior studies, the assumption always was that the results of the core laboratory were most accurate and therefore considered the criterion standard. However, with today's advanced technology, this may not always be the case. In fact, for some anlaytes, such as lactate, any delays in analysis may lead to spurious increases in test results. In this case, a POC measurement may actually be more accurate than an often-delayed core laboratory analysis. In our study, we did not assume that the core laboratory measurements were more accurate than the POC measurements. A certain degree of variation in test results is to be expected with all devices and analyzers whenever a sample is tested and retested using the same or another device. Our results suggest that
the variation between POC and the core laboratory measurements is rarely of clinical significance. This should give ED practitioners confidence in the POC results, recognizing that in up to 2–3% of samples, hemolysis may be present, artificially elevating the K+ levels. 5. Limitations This study has certain limitations. First, our results are limited to one commercially available POC device and eight specific analytes and cannot be generalized to other POC devices and analytes. Second, while adequately powered, our sample size was relatively small, especially with regards to severe laboratory abnormalities. Thus, we may have missed rare but more significant discrepancies between the two methods of analysis. Finally, our results are limited to a single center with highly experienced nurses who routinely perform bedside POC testing. As a result, the accuracy of POC may be lower with less experienced operators. 6. Conclusions Bedside POC testing by clinical nurses utilizing the i-STAT platform is as reliable and accurate as core laboratory testing by trained technicians. This suggests that clinicians should be confident when using POC results to make clinical decisions. Conflict of interest AJS is on the speaker's bureau of Abbott Point of Care and has received prior research funding from Abbott Point of Care. The current study was investigator initiated and the authors received no financial support from Abbott Point of Care for the current study. References [1] Forsman RW. Why is the laboratory an afterthought for managed care organizations? Clin Chem 1996;42:813–6. [2] Jarvis PRE. Improving emergency department flow. Clin Exp Emerg Med 2016;3: 63–8. [3] Rooney KD, Schilling UM. Point-of-care testing in the over-crowded emergency department: can it make a difference? Crit Care 2014;18:692. [4] Hawkins R. Measurement of whole-blood potassium: is it clinically safe? Clin Chem 2003;49:2105–6. [5] Hawkins R. Discrepancy between visual and spectrophotometric assessment of a sample haemolysis. Ann Clin Biochem 2002;39:521–2. [6] Asirvatham JR, Moses V, Bjornson L. Errors in potassium measurement: a laboratory perspective for the clinician. N Am J Med Sci 2013;5:255–9. [7] Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999;8:135–60. [8] Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989;45:255–68. [9] Leino A, Kurvinen K. Interchangeability of blood gas, electrolyte, and metabolite results with point-of-care, blood gas and core laboratory analyzers. Clin Chem Lab Med 2011;49:1187–91. [10] Steinfelder-Visscher J, Teerenstra S, Gunnewiek JM, Weerwind PW. Evaluation of the i-STAT point-of-care analyzer in critically ill adult patients. J Extra Corp Technol 2008;40:57–60. [11] Chacko B, Peter JV, Patole S, Fleming JJ, Selvakumar R. Electrolytes assessed by pointof-care testing—are the values comparable with results obtained from central laboratory? Indian J Crit Care Med 2011;15:24–9. [12] Yildrim E, Karapinar T, Hayirli A. Reliability of the i-STAT for the determination of blood electrolyte (K+, Na+, Cl−) concentration in cattle. J Vet Intern Med 2015;29: 388–94.
Please cite this article as: McIntosh BW, et al, Accuracy of bedside point of care testing in critical emergency department patients, American Journal of Emergency Medicine (2017), http://dx.doi.org/10.1016/j.ajem.2017.09.018
Contents lists available at ScienceDirect
American Journal of Emergency Medicine journal homepage: www.elsevier.com/locate/ajem
Accuracy of bedside point of care testing in critical emergency department patients☆ Braden W. McIntosh, MD a, Jerina Vasek, RN b, Maria Taylor, RN a, Deborah Le Blanc, RN a, Henry C. Thode, PhD a, Adam J. Singer, MD a,⁎ a b
Department of Emergency Medicine, Stony Brook University, Stony Brook, NY, United States Department of Clinical Pathology, Stony Brook University, Stony Brook, NY, United States
a r t i c l e
i n f o
Article history: Received 21 August 2017 Received in revised form 12 September 2017 Accepted 13 September 2017 Available online xxxx Keywords: Point-of-care testing Accuracy Emergency department
a b s t r a c t Background: Point-of-care (POC) testing reduces laboratory turn-around having the potential to improve timely diagnosis and management. We compared the accuracy of nurse performed POC and core laboratory testing and determined whether deviations between the two were clinically meaningful. Methods: We performed a prospective, observational study on a convenience sample of 50 critical care ED patients in whom a POC chemistry and hematocrit was ordered. Blood samples were divided into 2 aliquots; one sample was tested by the treating nurse using a handheld POC device and the other sample was tested in the core laboratory. Paired comparisons of test results were performed using Pearson's correlation coefficients, Lin concordance coefficients, and Bland Altman plots. Results: Mean patient age was 67, 50% were male, 82% were admitted. Pearson's correlation and Lin concordance coefficients were excellent (0.84–1.00) for all 8 analytes. Mean (95%CI) paired differences between POC and core laboratory measurements were Na+ 0.30 (−0.22 to 0.82) mmol/L, K+ − 0.12 (−0.14 to - 0.09) mmol/L, Cl− 2.10 (1.41 to 2.78) mmol/L, TCO2–1.68 (−2.06 to −1.30) mmol/L, glucose 2.46 (1.46 to 3.46) mg/dL, BUN, 1.69 (0.95 to 2.42) mg/dL, creatinine 0.13 (0.08 to 0.17) mg/dL, and hematocrit −0.39 (−0.93 to 0.15) %. In 3 of 400 measurements, the difference between POC and core lab exceeded the maximal clinically acceptable deviation based on physician surveys. Conclusions: Bedside POC by ED nurses is reliable and accurate and does not deviate significantly from core laboratory testing by trained technicians. © 2017 Elsevier Inc. All rights reserved.
1. Introduction A significant proportion of clinical decisions are made based on the results of laboratory testing [1]. While all analytes are important, acute disturbances in potassium (K+) levels have the greatest potential to cause patient harm. Thus, use of analytical methods that minimize errors in measuring K+ levels is of major concern. A large number of rapid point-of-care (POC) tests and devices are now available, which have the potential to significantly reduce test turn around times and emergency department (ED) length of stay [2,3]. However, some health care practitioners may be skeptical about the results of bedside POC testing, and hesitate acting on them without confirmation using core laboratory testing. Rapid measurement of analytes using whole blood specimens is fast and convenient. However, a major disadvantage of using whole blood
☆ Presented at the Annual Meeting of the Society for Academic Emergency Medicine, May 2017, New Orleans, LA. ⁎ Corresponding author at: Department of Emergency Medicine, HSC-L4-080, 8350 SUNY, Stony Brook, NY 11794-8350, United States. E-mail address: [email protected] (A.J. Singer).
samples is the inability to detect hemolysis. A study of 610 blood samples found mild hemolysis in 18%, moderate hemolysis in 3.6%, and severe hemolysis in 0.4% [4]. In this study, Hawkins estimated that the difference in K+ measurement attributable to hemolysis was N0.5 mmol/L in 8% of the samples. A prior study by the same author estimated a hemolysis rate of 3.4% [5]. As a result, when measuring analytes such as K+ using whole blood samples, there is a risk of obtaining an artificially elevated K+ level that can lead to misdiagnosis and inappropriate treatment. Mechanical forces leading to hemolysis include use of tourniquets, small gauge needles, and fist clenching to name but a few. Other causes of spuriously elevated K+ levels (pseudohyperkalemia) include excessive numbers of platelets or neoplastic WBCs. These same factors that lead to pseudohyperkalemia may also sometimes mask hypokalemia [6]. The objective of the present study was to determine the reliability of a whole blood based POC device in measuring eight commonly measured analytes in critically ill or injured emergency department patients. We hypothesized that the correlations between POC and core lab results would be excellent and that the bias (the difference between the two measurements) would not be clinically meaningful.
http://dx.doi.org/10.1016/j.ajem.2017.09.018 0735-6757/© 2017 Elsevier Inc. All rights reserved.
Please cite this article as: McIntosh BW, et al, Accuracy of bedside point of care testing in critical emergency department patients, American Journal of Emergency Medicine (2017), http://dx.doi.org/10.1016/j.ajem.2017.09.018
2
B.W. McIntosh et al. / American Journal of Emergency Medicine xxx (2017) xxx–xxx
2. Methods 2.1. Study design We conducted a prospective, observational, study to test the study hypothesis. Waiver of informed consent was obtained from our Institutional Review Board. 2.2. Patients and setting The study was conducted at a tertiary care, suburban, academic medical center with an annual ED census of approximately 110,000. Fifty consecutive critically ill or injured adult patients in whom a physician ordered a POC (i-STAT, Abbott Point of Care, Princeton, NJ) limited metabolic panel and hematocrit level (requiring a CHEM8+ cartridge) were enrolled. 2.3. Blood samples and measurements Venous blood samples were collected from patients using a vacutainer and placed in blood collection glass tubes containing lithium heparin. The blood samples were split into two aliquots, one of which was analyzed by a trained clinical nurse at the bedside using the POC device. The other sample was transported to the core lab for further testing. The remaining aliquot was centrifuged at 3000 ×g for 15 min at 4 °C to harvest plasma. The plasma sample was then analyzed by a trained laboratory technician using core laboratory devices (Roche COBAS 6000 [Roche Diagnostics USA, Indianapolis, IN] and Sysmex XN [Sysmex America, Inc., Licolnshire, IL] for metabolic panel and hemoglobin respectively). 2.4. Physician surveys A convenience sample of 25 emergency physicians were asked to indicate the maximal clinically acceptable deviation between POC and core laboratory results that would maintain their confidence in the POC results in ED patients for each of the eight analytes (Na +, K +, Cl−, TCO2, BUN, creatinine, glucose, and hematocrit). 2.5. Data analysis Continuous data are summarized as means and 95% confidence intervals (CI). Binary data are summarized as numbers and percentages frequency of occurrence. Regression plots and Pearson's correlation coefficients were used to measure the agreement between paired POC and core laboratory results. The differences (bias) between paired measurements were then determined by Bland-Altman plot analysis [7]. Lin's concordance correlation (pc) that describes the relationship between paired measurements was also used [8]. Unlike the Pearson correlation, which measures the strength of a linear relationship but may not pass through the origin and have a slope not equal to unity, pc compares agreement (between two sets of measurements) by assessing the variation from the 45° line through the origin. The number of test results in which the difference between paired samples was greater than the median minimal clinically acceptable deviation based on physician surveys was also determined. Assuming a correlation of 0.90 between pairs of measurements and a standard deviation (SD) of 0.6 for each type of measurement (POC or core lab), then the SD of the difference is 0.27. A sample size of 29 patients would be needed to have a confidence interval of the difference of ±0.1 around the point estimate. 3. Results Samples from 50 ED critical care patients were analyzed. Mean patient age (range) was 67 (24–97) years, 50% were male, 82% were
admitted, and of those, 30% went to the intensive care unit (ICU). General categories of patient chief complaints leading to an ED visit included 14 cardiac (chest pain or arrhythmia), 12 neurological (possible stroke or seizure), 11 traumatic (motor vehicle collision or fall), 5 altered mental status, and 5 miscellaneous. The Pearson's correlation coefficients between paired samples, one measured with the POC device and the other measured with the core laboratory device, ranged from 0.84 to 0.99 (Table 1), and were lowest for Cl− (0.89) and Na+ (0.84). Lin concordance coefficients ranged from 0.82 to 0.99 (Table 1), and were lowest for Cl− (0.82) and Na+ (0.84). The mean (range) paired differences between the POC and core laboratory samples were Na+ 0.30 (−6 to 4) mmol/L, K+ −0.12 (−0.40 to 0.15) mmol/L, Cl− 2.10 (−3 to 10) mmol/L, TCO2 1.68 (−4 to 2) mmol/ L, glucose 2.46 (−9 to 9) mg/dL, BUN, 1.69 (−4 to 9) mg/dL, creatinine 0.13 (−0.1 to 0.9) mg/dL, and hematocrit −0.39 (−7.2 to 4.2) % (Table 1). The Bland-Altman plots are presented in Fig. 1. The mean and median maximal clinically acceptable deviations between POC and core laboratory results that would not impact clinical diagnosis and management based on physician surveys is presented in Table 2. The number of test results in which the difference between POC and core laboratory results exceeded the median clinically acceptable difference were TCO2 none, Cl− none, K+ none, BUN one (27 vs. 18 mg/dL), creatinine one (5.5 vs. 4.6 mg/dL), glucose none, hematocrit one (35 vs. 42%), and Na+ one (133 vs. 139 mmol/L). Thus, only in 4 out of 400 paired samples did the difference exceed the clinically acceptable deviation. In all four cases, these differences did not alter clinical management. 4. Discussion In this observational study of 50 critically ill or injured ED patients, there was high to excellent correlation between paired measurements of POC and core laboratory tests for all eight of the tested analytes. Excellent correlations were seen both with the Pearson's and the Lin concordance tests. The bias between the two tests was generally small, with the 95% CIs falling within the maximal clinically acceptable deviations according to the physician surveys. In addition, the differences between the two tests rarely exceeded the maximal clinically acceptable deviation that would be considered to effect clinical diagnosis and management. These findings suggest that rapid beside POC testing using the iSTAT device can be used interchangeably with core laboratory testing. This should give clinicians confidence when making clinical decisions based on POC testing results alone. As always, whenever a laboratory result does not make clinical sense, it should be repeated. This is true for all laboratory tests, regardless of the device or platform used including POC and traditional core laboratory tests. Table 1 Agreement between point of care and core laboratory results. Pearson's correlation coefficient
Mean (95%CI) paired difference
Lin concordance coefficient
TCO2
0.94
0.86
CI−
0.89
K+
0.999
BUN
0.96
Creatinine
0.999
Glucose
0.999
Hematocrit
0.95
−1.68 −2.06–1.30 2.10 1.41–2.78 −0.12 −0.14–0.09 1.69 0.95–2.42 0.13 0.08–0.17 2.46 1.46–3.46 −0.39 −0.93–0.15 0.30 −0.22–0.82
Na
+
0.85
0.82 0.97 0.94 0.98 0.999 0.95 0.84
Please cite this article as: McIntosh BW, et al, Accuracy of bedside point of care testing in critical emergency department patients, American Journal of Emergency Medicine (2017), http://dx.doi.org/10.1016/j.ajem.2017.09.018
B.W. McIntosh et al. / American Journal of Emergency Medicine xxx (2017) xxx–xxx
Fig. 1. Bland Altman Plots for the eight analytes. Left column from top to bottom: TCO2, K+, creatinine, hematocrit. Right column from top to bottom: Cl−, BUN, glucose, Na+.
Please cite this article as: McIntosh BW, et al, Accuracy of bedside point of care testing in critical emergency department patients, American Journal of Emergency Medicine (2017), http://dx.doi.org/10.1016/j.ajem.2017.09.018
3
4
B.W. McIntosh et al. / American Journal of Emergency Medicine xxx (2017) xxx–xxx
Table 2 Surveyed physicians' maximal acceptable deviations between point-of-care and core laboratory results.
TCO2, mmol/L Cl−, mmol/L K+, mmol/L BUN, mmol/L Creatinine, mg/dL Glucose, mg/dL Hematocrit, % Na+, mmol/L
Mean
Median
3.7 9.5 0.5 7.1 0.4 11 4.3 4.2
4.0 10.0 0.5 8.0 0.5 10 4.0 5.0
A number of prior studies have evaluated whether rapid measurements of K+ using whole blood are as accurate as core laboratory testing using plasma samples [9-11]. In one study, discrepancies of up to 1 mmol/L were noted between whole blood and plasma measurements, especially when potassium levels were below 3 mmol/L. [9] Another study reported that nearly 1 in 3 hypokalemic samples was missed when whole blood was analyzed [10]. A recent study by Yildrim et al. assessed the reliability of a POC device (i-STAT) for the determination of K+, Na+, and Cl− whole blood concentrations in 98 cattle [12]. POC plasma concentrations of K+ (4.39 vs. 4.2 mmol/L; P b 0.001) and Cl− (100.30 vs. 99.4 mmol/L; P b 0.04) were higher than their concentration in core laboratory plasma. The mean bias (whole blood concentration-plasma concentration) in mmol/L was − 0.20 for K+, − 0.87 for Cl−, and − 0.16 for Na+. These differences do not appear to be clinically meaningful. A study by Leino et al. found that the blood gases, K+, Na+, Ca2+, glucose and lactate measured with stat POC and core laboratory analyzers were comparable with minimal bias. However, at low hemoglobin levels, the stat POC test over-diagnosed anemia [9]. In contrast, a Dutch study of 48 blood samples from patients undergoing cardiac surgery and another 42 blood samples from ICU patients found that there was an underestimation in hematocrit of up to 2.2% (P b 0.001) in ICU patients with hematocrits below 25% [10]. Finally an Indian study compared whole blood and plasma samples from ICU patients analyzed by arterial blood gas and core laboratory analyzers, respectively [11]. In this study the reported bias in K+ levels for whole blood samples was greater (95% levels of agreement between [LOA] − 1.0 and − 0.13 mmol/L) for K+ levels under 3.0 mmol/L than for higher K+ levels (mean difference − 0.2; 95% LOA -0.48 to 0.06). These studies suggest that the degree of concordance between POC and core laboratory measurements may not be uniform across the entire range of values. With all prior studies, the assumption always was that the results of the core laboratory were most accurate and therefore considered the criterion standard. However, with today's advanced technology, this may not always be the case. In fact, for some anlaytes, such as lactate, any delays in analysis may lead to spurious increases in test results. In this case, a POC measurement may actually be more accurate than an often-delayed core laboratory analysis. In our study, we did not assume that the core laboratory measurements were more accurate than the POC measurements. A certain degree of variation in test results is to be expected with all devices and analyzers whenever a sample is tested and retested using the same or another device. Our results suggest that
the variation between POC and the core laboratory measurements is rarely of clinical significance. This should give ED practitioners confidence in the POC results, recognizing that in up to 2–3% of samples, hemolysis may be present, artificially elevating the K+ levels. 5. Limitations This study has certain limitations. First, our results are limited to one commercially available POC device and eight specific analytes and cannot be generalized to other POC devices and analytes. Second, while adequately powered, our sample size was relatively small, especially with regards to severe laboratory abnormalities. Thus, we may have missed rare but more significant discrepancies between the two methods of analysis. Finally, our results are limited to a single center with highly experienced nurses who routinely perform bedside POC testing. As a result, the accuracy of POC may be lower with less experienced operators. 6. Conclusions Bedside POC testing by clinical nurses utilizing the i-STAT platform is as reliable and accurate as core laboratory testing by trained technicians. This suggests that clinicians should be confident when using POC results to make clinical decisions. Conflict of interest AJS is on the speaker's bureau of Abbott Point of Care and has received prior research funding from Abbott Point of Care. The current study was investigator initiated and the authors received no financial support from Abbott Point of Care for the current study. References [1] Forsman RW. Why is the laboratory an afterthought for managed care organizations? Clin Chem 1996;42:813–6. [2] Jarvis PRE. Improving emergency department flow. Clin Exp Emerg Med 2016;3: 63–8. [3] Rooney KD, Schilling UM. Point-of-care testing in the over-crowded emergency department: can it make a difference? Crit Care 2014;18:692. [4] Hawkins R. Measurement of whole-blood potassium: is it clinically safe? Clin Chem 2003;49:2105–6. [5] Hawkins R. Discrepancy between visual and spectrophotometric assessment of a sample haemolysis. Ann Clin Biochem 2002;39:521–2. [6] Asirvatham JR, Moses V, Bjornson L. Errors in potassium measurement: a laboratory perspective for the clinician. N Am J Med Sci 2013;5:255–9. [7] Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999;8:135–60. [8] Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989;45:255–68. [9] Leino A, Kurvinen K. Interchangeability of blood gas, electrolyte, and metabolite results with point-of-care, blood gas and core laboratory analyzers. Clin Chem Lab Med 2011;49:1187–91. [10] Steinfelder-Visscher J, Teerenstra S, Gunnewiek JM, Weerwind PW. Evaluation of the i-STAT point-of-care analyzer in critically ill adult patients. J Extra Corp Technol 2008;40:57–60. [11] Chacko B, Peter JV, Patole S, Fleming JJ, Selvakumar R. Electrolytes assessed by pointof-care testing—are the values comparable with results obtained from central laboratory? Indian J Crit Care Med 2011;15:24–9. [12] Yildrim E, Karapinar T, Hayirli A. Reliability of the i-STAT for the determination of blood electrolyte (K+, Na+, Cl−) concentration in cattle. J Vet Intern Med 2015;29: 388–94.
Please cite this article as: McIntosh BW, et al, Accuracy of bedside point of care testing in critical emergency department patients, American Journal of Emergency Medicine (2017), http://dx.doi.org/10.1016/j.ajem.2017.09.018