- Email: [email protected]
CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays
CLB-09022; No. of pages: 10; 4C: Clinical Biochemistry xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
Clinical Biochemistry journal homepage: www.elsevier.com/locate/clinbiochem
F
8
a r t i c l e
9 10 11 12
Article history: Received 8 April 2015 Accepted 4 May 2015 Available online xxxx
13 14 15 16 17 18 19
Keywords: Routine chemistry markers Reference intervals Abbott Beckman Coulter AU CALIPER Reference Intervals
b
CALIPER Program, Department of Pediatric Laboratory Medicine, The Hospital of Sick Children, Toronto, ON, Canada Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada Eastern Health, St. Johns, NL, Canada
i n f o
a b s t r a c t
Objective: The CALIPER program has established a comprehensive database of pediatric reference intervals using largely the Abbott ARCHITECT biochemical assays. To expand clinical application of CALIPER reference standards, the present study is aimed at transferring CALIPER reference intervals from the Abbott ARCHITECT to Beckman Coulter AU assays. Design and methods: Transference of CALIPER reference intervals was performed based on the CLSI guidelines C28-A3 and EP9-A2. The new reference intervals were directly verified using 100 reference samples from the healthy CALIPER cohort. Results: We found a strong correlation between Abbott ARCHITECT and Beckman Coulter AU biochemical assays, allowing the transference of the vast majority (94%; 30 out of 32 assays) of CALIPER reference intervals previously established using Abbott assays. Transferred reference intervals were, in general, similar to previously published CALIPER reference intervals, with some exceptions. Most of the transferred reference intervals were sex-specific and were verified using healthy reference samples from the CALIPER biobank based on CLSI criteria. It is important to note that the comparisons performed between the Abbott and Beckman Coulter assays make no assumptions as to assay accuracy or which system is more correct/accurate. Conclusion: The majority of CALIPER reference intervals were transferrable to Beckman Coulter AU assays, allowing the establishment of a new database of pediatric reference intervals. This further expands the utility of the CALIPER database to clinical laboratories using the AU assays; however, each laboratory should validate these intervals for their analytical platform and local population as recommended by the CLSI. © 2015 The Canadian Society of Clinical Chemists. Published by Elsevier Inc. All rights reserved.
R
E
C
c
O
a
R O
5 6 7
P
4
Mohamed Abou El Hassan a,b, Alexandra Stoianov a, Petra A.T. Araújo a, Tara Sadeghieh a, Man Khun Chan a, Yunqi Chen a, Edward Randell c, Michelle Niewesteeg a, Khosrow Adeli a,b,⁎
D
3Q2
E
2
CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays
T
1Q1
R
38
44 45 46 47 48 49 50 51 52 53
Introduction
Reference intervals serve as health-associated benchmarks with which a patient’s individual test results are compared and are essential for accurate interpretation of laboratory test results. The levels of many biomarkers may vary with age, sex, and ethnicity; thus, age- and sexspecific reference values are critical for the appropriate interpretation of test results. This is particularly important for the pediatric population, where the high rates of growth and development can alter the otherwise predicted levels of many analytes [1–5]. The use of adult reference intervals for the interpretation of pediatric test results, a common practice in many healthcare centers worldwide, is inappropriate, and
U
41 43
N C O
42 40 39
Abbreviations: CALIPER, Canadian Laboratory Initiative in Pediatric Reference Intervals; CLSI, Clinical Laboratory Standards Institute; Q–Q, quantile–quantile; CAP, College of American Pathologists. ⁎ Corresponding author at: Clinical Biochemistry, The Hospital for Sick Children, 555 University Avenue, Toronto, ON, M5G 1X8 Canada. E-mail address: [email protected] (K. Adeli).
represents a major source of post-analytical error that can lead to patient misdiagnoses and inappropriate treatment decisions [6–9]. The lack of pediatric-specific reference intervals has remained a major problem in the pediatric setting due to the difficulty in collecting samples from healthy community children, particularly from young children and infants, where only a small blood volume can be obtained. In an effort to address this important evidence gap, the CALIPER (Canadian Laboratory Initiative in Pediatric Reference Intervals) project was established as a collaborative initiative between several pediatric centers across Canada [10,11]. In 2006, the CALIPER project released the first pilot data describing pediatric reference intervals for several basic chemistry analytes [12]. Since then, CALIPER has produced a comprehensive database of pediatric reference intervals for many specialty, immunoassay, and chemistry analytes [1,11,13–21], and has effectively filled many of the previous gaps in pediatric reference intervals [10,11]. A major caveat was that CALIPER reference intervals were established using the Abbott ARCHITECT assays, which limits the application of these reference intervals to pediatric centers that use Abbott analytical platforms.
http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002 0009-9120/© 2015 The Canadian Society of Clinical Chemists. Published by Elsevier Inc. All rights reserved.
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72
102
Method comparison
103
112
The present study was approved by the Institutional Review Board (IRB) at the Hospital for Sick Children (Toronto, Canada) and the review boards of collaborating hospitals. Approximately 200 pediatric pooled patient serum specimens (The Hospital for Sick Children, Toronto, Ontario) were analyzed on the Abbott ARCHITECT c8000 (at Eastern Health Authority, St. John's, Newfoundland) and the Beckman Coulter AU Systems (at Beckman Coulter, Brea, California). Samples represented a wide range of analyte concentration or activity. A full list of analyzed markers, assay specifications and analytical parameters are provided in Supplemental Tables 1 and 2.
113 114
98 99
104 105 106 107 108 109 110 111
115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133
134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155
Verification of transferred reference intervals using samples from the 156 CALIPER biorepository 157 Next, the transferred reference intervals were verified in accordance with the CLSI C28-A3 guidelines [22]. For this purpose, approximately 100 CALIPER reference specimens were analyzed on the Beckman Coulter AU platform. Selected samples spanned as many pediatric age range and gender partitions as possible. A conservative approach was adopted to define and remove outliers. In brief, the results were visually inspected for gross outliers, which were defined as being several-fold higher/lower than the next highest/lowest data point. If these points were considered outliers by the Tukey test (a method for finding outliers using the interquartile range to filter out very large or very small numbers), they were then excluded. The total percentage of samples that fell within the appropriate reference limits was then calculated (total verification across all pediatric age and gender partitions). This process was repeated for the lower and upper reference limits inclusive of the 95% confidence intervals. Reference intervals were considered verified when N 90% of reference samples fell within the confidence intervals of the transferred reference intervals.
158 159
Transference protocols and statistical analysis
Results
175
Data analysis and reference interval transference were done in accordance with CLSI C28-A3 and EP9-A2 guidelines [22,23]. Statistical analysis was performed using Excel (Microsoft) and R statistical computing program [25]. For each assay, the Abbott ARCHITECT results were plotted as a function of the corresponding results obtained with Beckman Coulter AU. Graphs were visually examined for outliers, and gross outliers, which occurred in rare cases, were removed. Results below the lower end of the reportable range were excluded. A simple linear regression assumes that the X variable is known (without error). If the R2 ≥ 0.95, any error in X is adequately compensated by the range of data, and simple linear regression using the least squares approach can be used to estimate the slope and intercept [23]. Therefore, this regression method was used to determine the line of best fit and its corresponding equation in cases where R2 ≥ 0.95. However, if 0.70 b R2 b 0.95, linear regression does not accurately estimate the slope or the Y intercept (where the error in X is not adequately compensated by the range of the data). In this case, Deming regression was used, which allows both methods to have measurement error. In cases of poor correlation (R2 b 0.70), the corresponding reference intervals were deemed non-transferable.
First, correlation between assays on the two platforms was carefully assessed using 200 pediatric serum samples. The results from 32 Abbott ARCHITECT assays were correlated with 60 corresponding assays on the Beckman Coulter AU system, where Beckman Coulter offered more than one assay for the majority of the tested analytes, with the exception of apolipoprotein B (APOB), antistreptolysin O (ASO), complement C3 (C3), complement C4 (C4), Creactive protein-high sensitivity (CRPHS), haptoglobin (HAPT), immunoglobulin A (IgA), immunoglobulin G (IgG), immunoglobulin M (IgM), prealbumin (PAB), and transferrin (TRN), for which only one assay was tested on the Beckman Coulter AU. Results from the vast majority of Beckman Coulter AU assays (51/60) strongly correlated (R2 N 0.7) with those from Abbott (Fig. 1A, Supplementary Fig. 1), except for 9 assays that did not sufficiently correlate between the two platforms (Supplementary Fig. 2), including: carbon dioxide (2/2 assays; R2 ranged from 0.15 to 0.2), calcium (2/2 assays; R2 from 0.32 to 0.52), albumin (1/2 assays, R2 = 0.69), alanine aminotransferase (1/3 assays; R2 = 0.52), aspartate aminotransferase (1/3 assays; R 2 = 0.4), magnesium (1/2 assays; R2 = 0.61), and total protein (1/2 assays; R2 = 0.66). As a result, the reference intervals
176
E
96 97
T
94 95
C
92 93
E
90 91
R
88 89
R
86 87
O
84 85
C
82 83
N
80 81
U
79
F
Materials and methods
77 78
O
101
75 76
In order to assess the appropriateness of a linear model with normally distributed data points, we generated standardized residual, Bland–Altman, and quantile–quantile (Q–Q) plots. The standardized residual and Bland–Altman plots were visually examined to confirm that the data points did not cluster into distinct patterns. Q–Q plots served to assess whether the residuals followed a normal distribution. A Q-Q plot shows the distance between a point and the regression line (i.e. the standardized residual) on the y-axis as a function of what that distance would be if the residuals were normally distributed (i.e. the theoretical quantile) on the x-axis. We visually examined Q–Q plots to verify that the data followed a straight line of the equation y = x, indicative of normally distributed residuals. In cases where all these criteria were met, the equation of the line of best fit was used to transfer the CALIPER reference intervals established using the Abbott ARCHITECT platform [1] to the corresponding Beckman Coulter AU assay. In cases where the criteria were not met, the corresponding reference intervals were deemed non-transferable. The root of the mean-squared error (RMSE) was used to determine 95% confidence intervals around each lower and upper reference limit, calculated as the reference limit ± 1.96 ∗ RMSE. The 95% confidence intervals were used as secondary limits with which to verify the transferred reference intervals.
R O
100
Establishing new reference intervals is a complex, costly, and daunting task, involving recruitment and sample collection from a large number of healthy individuals. As a result, the Clinical and Laboratory Standards Institute (CLSI) has issued guidelines on transferring reference intervals established in one laboratory (the “donor” laboratory) to other (“receiving”) laboratories [22]. This process involves transference and verification steps to ensure validity of the transferred reference intervals. First, transference of reference intervals can only occur if (1) there is a good correlation between methods used in “donor” and “receiving” laboratories and (2) the results produced by both methods are normally distributed [23]. If these criteria are met, a mathematical equation that governs the relationship between the results produced by both platforms is determined, and this equation is used to convert the original reference intervals into transferred reference intervals that can be applied to the platform in the receiving laboratory [22]. Next, the receiving laboratory must verify the transferred reference intervals using specimens from a small number of reference individuals recruited from a healthy population. We have previously adopted this approach to transfer reference intervals from the Abbott ARCHITECT analyzer to a multitude of assays on four other commonly used clinical chemistry analyzers [24]. There is, however, still a need to transfer reference intervals to additional systems that are commonly used in major pediatric hospitals across Canada, and worldwide. Here, in order to broaden the utility of the CALIPER reference interval database [1], we report transference of pediatric reference intervals to the widely used Beckman Coulter AU assays, which will further enhance the global utility of the CALIPER database.
D
73 74
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx
P
2
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
160 161 162 163 164 165 166 167 168 169 170 171 172 173 174
177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx
5
10
15
2 1 0 -4
10
y= -0.025 + 1.081 *x R-squared= 0.98 RMSE= 0.589
-2
15
20
Stanrdized residuals
25
3
(B)
5
Beckman Coulter AU Assay (µmol/L)
(A)
5
20
10
20
25
O
3 2 1
R O
0
P
-2
Standardized residuals 25
-3
-2
-1 0 1 Theoretical quantiles
2
3
D
10 15 20 Abbott concentration (µmol/L)
-4
20 10 0 -10 -20
% difference
(D)
5
15
Abbott concentration (µmol/L)
F
Abbott Assay (µmol/L)
(C)
3
202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228
C
E
R
200 201
R
198 199
for albumin, alanine aminotransferase, aspartate aminotransferase, magnesium, and total protein were transferred to only those Beckman Coulter AU assays showing strong correlation (R2 N 0.7), while no transference was possible to any of the carbon dioxide or calcium assays on the Beckman Coulter AU platform. Next, standardized residual, Bland–Altman, and Q–Q plots were generated for the 51 assays that correlated well with the Abbott ARCHITECT (R2 N 0.70) to assess the appropriateness of a linear model with normally distributed data points. Visual inspection of graphs revealed that both residuals and data points were randomly distributed with no evidence of data clustering across the studied range for all assays, and that the Q–Q plots followed a straight line (Fig. 1B–D; Supplementary Fig. 1). Thus, the linear model was considered appropriate and the derived regression equations were used to transfer the Abbott ARCHITECT reference intervals to these 51 Beckman Coulter AU assays. The new pediatric reference intervals, stratified by age and sex, for the tested Beckman Coulter AU assays are summarized in Table 1. Overall, the vast majority of CALIPER pediatric reference intervals (30/32) were transferrable from the Abbott ARCHITECT to the Beckman Coulter AU analyzer (Table 2). Finally, the reference intervals for all transferrable assays were verified using 100 healthy reference samples from the CALIPER cohort [1], a considerably higher sample size than is recommended in the CLSI C28-A3 guidelines [22], which provides a more stringent verification of the new Beckman Coulter AU reference intervals. The transferred reference intervals were considered verified if N 90% of the CALIPER samples fell within the reference intervals, inclusive of the 95% confidence intervals around the upper and lower limit of each reference interval. Percent verification of reference intervals for all transferred assays are summarized in Table 3. Overall, our analysis revealed that 82.4% (42 out of 51) of the transferred reference intervals were verified, based on our more stringent verification cut-off of 90% of CALIPER samples falling within the transferred reference interval, inclusive of
N C O
197
U
196
T
E
Fig. 1. Statistical criteria employed to assess appropriateness of reference interval transference. Shown is a representative example. (A) Scatter, (B) standardized residual, (C) Bland–Altman, and (D) Q–Q plots for iron (FE), where transference of reference intervals between analyzers was deemed appropriate.
the 95% confidence intervals. If the cut-off value for verification was reduced to 80% of CALIPER samples falling within the confidence intervals of the transferred reference intervals, 98.0% (50 out of 51) of the transferred reference intervals were found to be verified (Table 3).
229
Discussion
233
Establishing new reference intervals is particularly challenging for pediatric populations where sample collection is technically difficult and the need to obtain parental consent poses additional obstacles. The CALIPER program has recruited a large population of healthy children and adolescents and established a new database of pediatric reference intervals for laboratory biomarkers using Abbott ARCHITECT assays. To broaden the application of CALIPER reference standards, we have initiated a series of transference studies to establish a similar database of pediatric reference intervals for biochemical assays on other clinical chemistry platforms. These transference studies have been designed based on CLSI guidelines [22,23] and were recently used to transfer CALIPER reference values to biochemical assays on 4 other clinical chemistry platforms namely, Beckman Coulter DxC800, Ortho Vitros 5600, Roche Cobas 6000, and Siemens Vista 1500 [24]. In the present study, we report transference of CALIPER reference intervals to biochemical assays on the Beckman Coulter AU platform, one of the most commonly used clinical chemistry platforms, based on recent College of American Pathologists (CAP) surveys. Transference of reference intervals is a multistep process that involves studying the correlation between the donor (i.e. Abbott ARCHITECT in this study) and the receiving (i.e. Beckman Coulter AU) methods, establishing the suitability of a linear equation for the transference of reference intervals between the different methods, and finally, verification of the new reference intervals using samples from a healthy population. Each of these steps poses its own challenges. First, a lack of correlation between the different assays can represent a
234
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
230 231 232
235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259
4
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx Table 1 Age- and sex-specific pediatric reference intervals for biochemical markers measured with Beckman Coulter AU assays.
U
N
C
O
R
R
E
C
T
E
D
P
R O
O
F
t1:1 t1:2
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx Table 1 (continued)
U
N C O
R
R
E
C
T
E
D
P
R O
O
F
t1:4
5
(continued on next page)
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
6
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx Table 1 (continued)
U
N
C
O
R
R
E
C
T
E
D
P
R O
O
F
t1:6
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx Table 1 (continued)
t1:10
• Pink and blue highlights indicate female and male-specific reference intervals, respectively. • Assay specific characteristics and analytical parameters are summarized in Supplementary Tables 1 and 2.
7
U
N C O
R
R
E
C
T
E
D
P
R O
O
F
t1:8
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
8
Table 2 Transference of CALIPER reference intervals from Abbott ARCHITECT to Beckman Coulter AU assays. Table lists the Abbott ARCHITECT assays for which reference intervals were transferable to at least one Beckman Coulter AU assay (see Methods).
instability of CO2, which can easily evaporate from the sample during storage, transport, and handling [27]. Thus, analytes with poor stability may not be suitable candidates for transference studies, especially when performing studies between distant laboratories. Another analyte that did not exhibit acceptable correlation between the Abbott ARCHITECHT and the Beckman Coulter AU assays was total calcium (Ca), where the R2 ranged between 0.32 and 0.52. While Abbott has one Ca assay which is Arsenazo-based, Beckman Coulter AU has two Arsenazo-based assays (CaA and CAAO). Similar to the CO2 assays, the lower correlation between Ca assays on both platforms could not be attributed to a difference in the assay principle, as both platforms utilize Arsenazo-based assays. The lack of correlation between assays may be due to using different reagent lots on both platforms or due to using calibrators with different traceability, where Beckman Coulter and Abbott ARCHITECT Ca assay calibrators are traceable to NIST SRM 956 and SRM 909b, respectively. Verification of the transferred reference intervals can pose additional challenges in transference studies. The CLSI guidelines recommend using 20 reference samples from healthy individuals for this purpose, and considers a reference interval valid if N90% of the reference samples fall within the transferred reference interval. Here, we employed a more rigorous criteria by (1) using a minimum of 200 patients to derive the linear equation for transference, (2) applying a more rigorous statistical criteria to examine data clustering and ensure the validity of a linear relationship over the entire data range, (3) determining linear or Deming regression, as appropriate, to accurately describe the correlation between Abbott and Beckman Coulter assays, and finally, (4) testing 100 healthy reference samples for the verification of the transferred reference intervals. CLSI mandates some of these methods for either the validation of an existing manufacturer reference interval or for generating a new reference interval using patient data. Here, we have combined these criteria for a more robust calculation and verification of the new reference intervals. Thus, we selected a verification cutoff of 90% of CALIPER samples falling within the reference interval, inclusive of the 95% confidence intervals, which revealed a verification rate of 82.4% of all transferred reference intervals. The transferred Beckman Coulter AU reference intervals largely agreed with the Abbott ARCHITECT reference intervals. For some biomarkers, there was a high degree of overlap between male and female reference intervals on both analyzers such as urea and creatinine, which reflect the high levels of standardization of these assays. Similar agreement was observed for immunochemical assays such as IgA, ApoB, and high sensitivity CRP. In contrast, notable differences were observed between Abbott reference intervals and the transferred reference intervals for a few analytes, namely, total and direct bilirubin, amylase, lactate dehydrogenase, and ASO. Of these, direct bilirubin showed the largest difference between both analyzers, where Beckman Coulter AU reference intervals for both males and females were lower than the Abbott intervals for all age partitions by at least 60% (Table 1). In summary, we have established the mathematical relationships between Abbott ARCHITECT and Beckman Coulter AU assays based on patient sample comparisons, applied stringent statistical criteria to assess assay-to-assay correlation, calculated AU reference intervals for those assays meeting established criteria, and then verified the new intervals using reference samples from the healthy CALIPER cohort. The new database of Beckman Coulter pediatric reference intervals expands the utility of the CALIPER reference interval database to clinical laboratories using Beckman Coulter AU assays. It is important to clarify that this study was designed to transfer reference intervals to specific assays and does not validate reference intervals for individual analyzers, specific populations, or geographic locations. Individual analyzers may have instrument-specific biases compared to those used in our study. Since local populations may differ in ethnicity, life style, and environmental conditions compared to the multiethnic CALIPER population, it is highly recommended that assay-specific CALIPER reference intervals reported in the present study be validated on each analytical platform
t2:6
260 261 262 263 264 265 266 267 268 269 270 271
U
N
C
O
R
R
E
C
T
E
D
P
R O
O
F
t2:1 t2:2 t2:3 t2:4
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx
major obstacle in transference studies. Here, 9 out of 60 Beckman Coulter AU assays did not correlate well with the corresponding Abbott ARCHIITECT assays. A prominent example is carbon dioxide (CO2), which had two assays on the Beckman Coulter AU system (CO2A and CO2B). Both assays did not correlate with the CO2 assay on the Abbott ARCHITECT system (R2 b 0.2). It is noteworthy to mention that poor correlation does not reflect the accuracy of either the Abbott or the Beckman Coulter assays. This poor correlation is not unique to the Beckman Coulter system since we observed similar poor correlation between the Abbott assay and CO2 assays on other major analyzers [24]. The lack of correlation between the CO2 assays cannot be attributed to a difference in the assay principles and is likely a result of the
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx Table 3 Verification of Beckman Coulter AU pediatric reference intervals using CALIPER reference samplesa.
Acknowledgements
344
This study was supported by an operating grant from the CIHR (Canadian Institutes of Health Research) and Beckman Coulter, Inc. We thank all of the CALIPER participants and their families; this study would not have been possible without their participation. We also thank the CALIPER coordinators, and all of the CALIPER volunteers for their hard work in participant recruitment and sample collection. Lastly, we acknowledge Abbott and Beckman Coulter, for providing reagents, and Dr. Jack Zakowski for his critical review of the manuscript.
345 346
References
353
[1] Colantonio DA, Kyriakopoulou L, Chan MK, Daly CH, Brinc D, Venner AA, et al. Closing the gaps in pediatric laboratory reference intervals: a CALIPER database of 40 biochemical markers in a healthy and multiethnic population of children. Clin Chem 2012;58:854–68. [2] Mansoub S, Chan MK, Adeli K. Gap analysis of pediatric reference intervals for risk biomarkers of cardiovascular disease and the metabolic syndrome. Clin Biochem 2006;39:569–87. [3] Delvin EE, Laxmi Grey V, Vergee Z, CALIPER Working Group. Gap analysis of pediatric reference intervals related to thyroid hormones and the growth hormone-insulin growth factor axis. Clin Biochem 2006;39:588–94. [4] Lepage N, Li D, Kavsak PA, Bamforth F, Callahan J, Dooley K, et al. Incomplete pediatric reference intervals for the management of patients with inborn errors of metabolism. Clin Biochem 2006;39:595–9. [5] Yang L, Grey V. Pediatric reference intervals for bone markers. Clin Biochem 2006; 39:561–8. [6] American Academy of Pediatrics AAP Section on Endocrinology and Committee on Genetics, American Thyroid Association Committee on Public Health. Newborn screening for congenital hypothyroidism: recommended guidelines. Pediatrics 1993;91:1203–9. [7] Cheillan D, Vercherat M, Chevalier-Porst F, Charcosset M, Rolland MO, Dorche C. False-positive results in neonatal screening for cystic fibrosis based on a threestage protocol (IRT/DNA/IRT): should we adjust IRT cut-off to ethnic origin? J Inherit Metab Dis 2005;28:813–8. [8] Mir TS, Flato M, Falkenberg J, Haddad M, Budden R, Weil J, et al. Plasma concentrations of N-terminal brain natriuretic peptide in healthy children, adolescents, and young adults: effect of age and gender. Pediatr Cardiol 2006;27:73–7. [9] Ogunkeye OO, Roluga AI, Khan FA. Resetting the detection level of cord blood thyroid stimulating hormone (TSH) for the diagnosis of congenital hypothyroidism. J Trop Pediatr 2008;54:74–7. [10] Pediatric reference intervals: critical gap analysis and establishment of a national initiative. Clin Biochem 2006;39:559–60. [11] Adeli K. Closing the gaps in pediatric reference intervals: the CALIPER initiative. Clin Biochem 2011;44:480–2. [12] Yip PM, Chan MK, Nelken J, Lepage N, Brotea G, Adeli K. Pediatric reference intervals for lipids and apolipoproteins on the VITROS 5,1 FS Chemistry System. Clin Biochem 2006;39:978–83. [13] Brinc D, Chan MK, Venner AA, Pasic MD, Colantonio D, Kyriakopolou L, et al. Longterm stability of biochemical markers in pediatric serum specimens stored at −80 °C: a CALIPER Substudy. Clin Biochem 2012;45:816–26. [14] Pasic MD, Colantonio DA, Chan MK, Venner AA, Brinc D, Adeli K. Influence of fasting and sample collection time on 38 biochemical markers in healthy children: a CALIPER substudy. Clin Biochem 2012;45:1125–30. [15] Kyriakopoulou L, Yazdanpanah M, Colantonio DA, Chan MK, Daly CH, Adeli K. A sensitive and rapid mass spectrometric method for the simultaneous measurement of eight steroid hormones and CALIPER pediatric reference intervals. Clin Biochem 2013;46:642–51. [16] Konforte D, Shea JL, Kyriakopoulou L, Colantonio D, Cohen AH, Shaw J, et al. Complex biological pattern of fertility hormones in children and adolescents: a study of healthy children from the CALIPER cohort and establishment of pediatric reference intervals. Clin Chem 2013;59:1215–27. [17] Bailey D, Colantonio D, Kyriakopoulou L, Cohen AH, Chan MK, Armbruster D, et al. Marked biological variance in endocrine and biochemical markers in childhood: establishment of pediatric reference intervals using healthy community children from the CALIPER cohort. Clin Chem 2013;59:1393–405. [18] Shaw JLV, Cohen A, Konforte D, Binesh-Marvasti T, Colantonio DA, Adeli K. Validity of establishing pediatric reference intervals based on hospital patient data: a comparison of the modified Hoffmann approach to CALIPER reference intervals obtained in healthy children. Clin Biochem 2014;47:166–72. [19] Bailey D, Bevilacqua V, Colantonio DA, Pasic MD, Perumal N, Chan MK, et al. Pediatric within-day biological variation and quality specifications for 38 biochemical markers in the CALIPER cohort. Clin Chem 2014;60:518–29. [20] Raizman JE, Cohen AH, Teodoro-Morrison T, Wan B, Khun-Chen M, Wilkenson C, et al. Pediatric reference value distributions for vitamins A and E in the CALIPER cohort and establishment of age-stratified reference intervals. Clin Biochem 2014;47:812–5. [21] Adeli K. Closing the gaps in pediatric reference intervals: an update on the CALIPER project. Clin Biochem 2014;47:737–9. [22] CLSI. Defining, establishing and verifying referene intervals in the clinical laboratory; aproved guidlines. CLSI Document C28-A3c 3rd ed. Wayne, PA: Clinical and Laboratory Standards Institute; 2008.
354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422
338 339
a
% verification indicates the percentage of CALIPER samples (n = 100) that fell within the appropriate partitioned upper and lower reference limits inclusive of the 95% confidence intervals (see Methods). Numbers within parentheses are percentage of samples that fell within the appropriate partitioned upper and lower reference limits of the Beckman Coulter AU assays, not including the 95% confidence intervals. Assays are highlighted with different colors based on the % verification: white (verification N90%), grey (verification N80%), and black (verification b80%).
U
t3:5 t3:6 t3:7 t3:8 t3:9 t3:10 t3:11
N C O
R
R
E
C
T
E
D
P
R O
O
F
t3:1 t3:2 t3:3
9
340 341
using reference specimens from healthy children in the local population as recommended by the CLSI. Supplementary data to this article can be found online at http://dx. doi.org/10.1016/j.clinbiochem.2015.05.002.
342 Q4
Uncited reference
343
[26]
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
347 348 349 350 351 352
10
[25] R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing; 2013(http://www.R-projct.org/). [26] Anscombe F. Graphs in Statistical Analysis. 1973;27:17–22. [27] Bandi ZL. Estimation, prevention, and quality control of carbon dioxide loss during aerobic sample processing. Clin Chem 1981;27:1676–81.
N
C
O
R
R
E
C
T
E
D
P
R O
O
F
[23] CLSI. Measurement Procedure Comparison and Bias Estimation Using Patient Samples: Approved Guideline. CLSI Document EP09-A33rd ed. Wayne, PA: Clinical and Laboratory Standards Institute; 2013. [24] Estey MP, Cohen AH, Colantonio DA, Chan MK, Marvasti TB, Randell E, et al. CLSIbased transference of the CALIPER database of pediatric reference intervals from Abbott to Beckman, Ortho, Roche and Siemens Clinical Chemistry Assays: direct validation using reference samples from the CALIPER cohort. Clin Biochem 2013; 46:1197–219.
U
423 424 425 426 427 428 429 430
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
431 432 433 434 435
Contents lists available at ScienceDirect
Clinical Biochemistry journal homepage: www.elsevier.com/locate/clinbiochem
F
8
a r t i c l e
9 10 11 12
Article history: Received 8 April 2015 Accepted 4 May 2015 Available online xxxx
13 14 15 16 17 18 19
Keywords: Routine chemistry markers Reference intervals Abbott Beckman Coulter AU CALIPER Reference Intervals
b
CALIPER Program, Department of Pediatric Laboratory Medicine, The Hospital of Sick Children, Toronto, ON, Canada Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada Eastern Health, St. Johns, NL, Canada
i n f o
a b s t r a c t
Objective: The CALIPER program has established a comprehensive database of pediatric reference intervals using largely the Abbott ARCHITECT biochemical assays. To expand clinical application of CALIPER reference standards, the present study is aimed at transferring CALIPER reference intervals from the Abbott ARCHITECT to Beckman Coulter AU assays. Design and methods: Transference of CALIPER reference intervals was performed based on the CLSI guidelines C28-A3 and EP9-A2. The new reference intervals were directly verified using 100 reference samples from the healthy CALIPER cohort. Results: We found a strong correlation between Abbott ARCHITECT and Beckman Coulter AU biochemical assays, allowing the transference of the vast majority (94%; 30 out of 32 assays) of CALIPER reference intervals previously established using Abbott assays. Transferred reference intervals were, in general, similar to previously published CALIPER reference intervals, with some exceptions. Most of the transferred reference intervals were sex-specific and were verified using healthy reference samples from the CALIPER biobank based on CLSI criteria. It is important to note that the comparisons performed between the Abbott and Beckman Coulter assays make no assumptions as to assay accuracy or which system is more correct/accurate. Conclusion: The majority of CALIPER reference intervals were transferrable to Beckman Coulter AU assays, allowing the establishment of a new database of pediatric reference intervals. This further expands the utility of the CALIPER database to clinical laboratories using the AU assays; however, each laboratory should validate these intervals for their analytical platform and local population as recommended by the CLSI. © 2015 The Canadian Society of Clinical Chemists. Published by Elsevier Inc. All rights reserved.
R
E
C
c
O
a
R O
5 6 7
P
4
Mohamed Abou El Hassan a,b, Alexandra Stoianov a, Petra A.T. Araújo a, Tara Sadeghieh a, Man Khun Chan a, Yunqi Chen a, Edward Randell c, Michelle Niewesteeg a, Khosrow Adeli a,b,⁎
D
3Q2
E
2
CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays
T
1Q1
R
38
44 45 46 47 48 49 50 51 52 53
Introduction
Reference intervals serve as health-associated benchmarks with which a patient’s individual test results are compared and are essential for accurate interpretation of laboratory test results. The levels of many biomarkers may vary with age, sex, and ethnicity; thus, age- and sexspecific reference values are critical for the appropriate interpretation of test results. This is particularly important for the pediatric population, where the high rates of growth and development can alter the otherwise predicted levels of many analytes [1–5]. The use of adult reference intervals for the interpretation of pediatric test results, a common practice in many healthcare centers worldwide, is inappropriate, and
U
41 43
N C O
42 40 39
Abbreviations: CALIPER, Canadian Laboratory Initiative in Pediatric Reference Intervals; CLSI, Clinical Laboratory Standards Institute; Q–Q, quantile–quantile; CAP, College of American Pathologists. ⁎ Corresponding author at: Clinical Biochemistry, The Hospital for Sick Children, 555 University Avenue, Toronto, ON, M5G 1X8 Canada. E-mail address: [email protected] (K. Adeli).
represents a major source of post-analytical error that can lead to patient misdiagnoses and inappropriate treatment decisions [6–9]. The lack of pediatric-specific reference intervals has remained a major problem in the pediatric setting due to the difficulty in collecting samples from healthy community children, particularly from young children and infants, where only a small blood volume can be obtained. In an effort to address this important evidence gap, the CALIPER (Canadian Laboratory Initiative in Pediatric Reference Intervals) project was established as a collaborative initiative between several pediatric centers across Canada [10,11]. In 2006, the CALIPER project released the first pilot data describing pediatric reference intervals for several basic chemistry analytes [12]. Since then, CALIPER has produced a comprehensive database of pediatric reference intervals for many specialty, immunoassay, and chemistry analytes [1,11,13–21], and has effectively filled many of the previous gaps in pediatric reference intervals [10,11]. A major caveat was that CALIPER reference intervals were established using the Abbott ARCHITECT assays, which limits the application of these reference intervals to pediatric centers that use Abbott analytical platforms.
http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002 0009-9120/© 2015 The Canadian Society of Clinical Chemists. Published by Elsevier Inc. All rights reserved.
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72
102
Method comparison
103
112
The present study was approved by the Institutional Review Board (IRB) at the Hospital for Sick Children (Toronto, Canada) and the review boards of collaborating hospitals. Approximately 200 pediatric pooled patient serum specimens (The Hospital for Sick Children, Toronto, Ontario) were analyzed on the Abbott ARCHITECT c8000 (at Eastern Health Authority, St. John's, Newfoundland) and the Beckman Coulter AU Systems (at Beckman Coulter, Brea, California). Samples represented a wide range of analyte concentration or activity. A full list of analyzed markers, assay specifications and analytical parameters are provided in Supplemental Tables 1 and 2.
113 114
98 99
104 105 106 107 108 109 110 111
115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133
134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155
Verification of transferred reference intervals using samples from the 156 CALIPER biorepository 157 Next, the transferred reference intervals were verified in accordance with the CLSI C28-A3 guidelines [22]. For this purpose, approximately 100 CALIPER reference specimens were analyzed on the Beckman Coulter AU platform. Selected samples spanned as many pediatric age range and gender partitions as possible. A conservative approach was adopted to define and remove outliers. In brief, the results were visually inspected for gross outliers, which were defined as being several-fold higher/lower than the next highest/lowest data point. If these points were considered outliers by the Tukey test (a method for finding outliers using the interquartile range to filter out very large or very small numbers), they were then excluded. The total percentage of samples that fell within the appropriate reference limits was then calculated (total verification across all pediatric age and gender partitions). This process was repeated for the lower and upper reference limits inclusive of the 95% confidence intervals. Reference intervals were considered verified when N 90% of reference samples fell within the confidence intervals of the transferred reference intervals.
158 159
Transference protocols and statistical analysis
Results
175
Data analysis and reference interval transference were done in accordance with CLSI C28-A3 and EP9-A2 guidelines [22,23]. Statistical analysis was performed using Excel (Microsoft) and R statistical computing program [25]. For each assay, the Abbott ARCHITECT results were plotted as a function of the corresponding results obtained with Beckman Coulter AU. Graphs were visually examined for outliers, and gross outliers, which occurred in rare cases, were removed. Results below the lower end of the reportable range were excluded. A simple linear regression assumes that the X variable is known (without error). If the R2 ≥ 0.95, any error in X is adequately compensated by the range of data, and simple linear regression using the least squares approach can be used to estimate the slope and intercept [23]. Therefore, this regression method was used to determine the line of best fit and its corresponding equation in cases where R2 ≥ 0.95. However, if 0.70 b R2 b 0.95, linear regression does not accurately estimate the slope or the Y intercept (where the error in X is not adequately compensated by the range of the data). In this case, Deming regression was used, which allows both methods to have measurement error. In cases of poor correlation (R2 b 0.70), the corresponding reference intervals were deemed non-transferable.
First, correlation between assays on the two platforms was carefully assessed using 200 pediatric serum samples. The results from 32 Abbott ARCHITECT assays were correlated with 60 corresponding assays on the Beckman Coulter AU system, where Beckman Coulter offered more than one assay for the majority of the tested analytes, with the exception of apolipoprotein B (APOB), antistreptolysin O (ASO), complement C3 (C3), complement C4 (C4), Creactive protein-high sensitivity (CRPHS), haptoglobin (HAPT), immunoglobulin A (IgA), immunoglobulin G (IgG), immunoglobulin M (IgM), prealbumin (PAB), and transferrin (TRN), for which only one assay was tested on the Beckman Coulter AU. Results from the vast majority of Beckman Coulter AU assays (51/60) strongly correlated (R2 N 0.7) with those from Abbott (Fig. 1A, Supplementary Fig. 1), except for 9 assays that did not sufficiently correlate between the two platforms (Supplementary Fig. 2), including: carbon dioxide (2/2 assays; R2 ranged from 0.15 to 0.2), calcium (2/2 assays; R2 from 0.32 to 0.52), albumin (1/2 assays, R2 = 0.69), alanine aminotransferase (1/3 assays; R2 = 0.52), aspartate aminotransferase (1/3 assays; R 2 = 0.4), magnesium (1/2 assays; R2 = 0.61), and total protein (1/2 assays; R2 = 0.66). As a result, the reference intervals
176
E
96 97
T
94 95
C
92 93
E
90 91
R
88 89
R
86 87
O
84 85
C
82 83
N
80 81
U
79
F
Materials and methods
77 78
O
101
75 76
In order to assess the appropriateness of a linear model with normally distributed data points, we generated standardized residual, Bland–Altman, and quantile–quantile (Q–Q) plots. The standardized residual and Bland–Altman plots were visually examined to confirm that the data points did not cluster into distinct patterns. Q–Q plots served to assess whether the residuals followed a normal distribution. A Q-Q plot shows the distance between a point and the regression line (i.e. the standardized residual) on the y-axis as a function of what that distance would be if the residuals were normally distributed (i.e. the theoretical quantile) on the x-axis. We visually examined Q–Q plots to verify that the data followed a straight line of the equation y = x, indicative of normally distributed residuals. In cases where all these criteria were met, the equation of the line of best fit was used to transfer the CALIPER reference intervals established using the Abbott ARCHITECT platform [1] to the corresponding Beckman Coulter AU assay. In cases where the criteria were not met, the corresponding reference intervals were deemed non-transferable. The root of the mean-squared error (RMSE) was used to determine 95% confidence intervals around each lower and upper reference limit, calculated as the reference limit ± 1.96 ∗ RMSE. The 95% confidence intervals were used as secondary limits with which to verify the transferred reference intervals.
R O
100
Establishing new reference intervals is a complex, costly, and daunting task, involving recruitment and sample collection from a large number of healthy individuals. As a result, the Clinical and Laboratory Standards Institute (CLSI) has issued guidelines on transferring reference intervals established in one laboratory (the “donor” laboratory) to other (“receiving”) laboratories [22]. This process involves transference and verification steps to ensure validity of the transferred reference intervals. First, transference of reference intervals can only occur if (1) there is a good correlation between methods used in “donor” and “receiving” laboratories and (2) the results produced by both methods are normally distributed [23]. If these criteria are met, a mathematical equation that governs the relationship between the results produced by both platforms is determined, and this equation is used to convert the original reference intervals into transferred reference intervals that can be applied to the platform in the receiving laboratory [22]. Next, the receiving laboratory must verify the transferred reference intervals using specimens from a small number of reference individuals recruited from a healthy population. We have previously adopted this approach to transfer reference intervals from the Abbott ARCHITECT analyzer to a multitude of assays on four other commonly used clinical chemistry analyzers [24]. There is, however, still a need to transfer reference intervals to additional systems that are commonly used in major pediatric hospitals across Canada, and worldwide. Here, in order to broaden the utility of the CALIPER reference interval database [1], we report transference of pediatric reference intervals to the widely used Beckman Coulter AU assays, which will further enhance the global utility of the CALIPER database.
D
73 74
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx
P
2
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
160 161 162 163 164 165 166 167 168 169 170 171 172 173 174
177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx
5
10
15
2 1 0 -4
10
y= -0.025 + 1.081 *x R-squared= 0.98 RMSE= 0.589
-2
15
20
Stanrdized residuals
25
3
(B)
5
Beckman Coulter AU Assay (µmol/L)
(A)
5
20
10
20
25
O
3 2 1
R O
0
P
-2
Standardized residuals 25
-3
-2
-1 0 1 Theoretical quantiles
2
3
D
10 15 20 Abbott concentration (µmol/L)
-4
20 10 0 -10 -20
% difference
(D)
5
15
Abbott concentration (µmol/L)
F
Abbott Assay (µmol/L)
(C)
3
202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228
C
E
R
200 201
R
198 199
for albumin, alanine aminotransferase, aspartate aminotransferase, magnesium, and total protein were transferred to only those Beckman Coulter AU assays showing strong correlation (R2 N 0.7), while no transference was possible to any of the carbon dioxide or calcium assays on the Beckman Coulter AU platform. Next, standardized residual, Bland–Altman, and Q–Q plots were generated for the 51 assays that correlated well with the Abbott ARCHITECT (R2 N 0.70) to assess the appropriateness of a linear model with normally distributed data points. Visual inspection of graphs revealed that both residuals and data points were randomly distributed with no evidence of data clustering across the studied range for all assays, and that the Q–Q plots followed a straight line (Fig. 1B–D; Supplementary Fig. 1). Thus, the linear model was considered appropriate and the derived regression equations were used to transfer the Abbott ARCHITECT reference intervals to these 51 Beckman Coulter AU assays. The new pediatric reference intervals, stratified by age and sex, for the tested Beckman Coulter AU assays are summarized in Table 1. Overall, the vast majority of CALIPER pediatric reference intervals (30/32) were transferrable from the Abbott ARCHITECT to the Beckman Coulter AU analyzer (Table 2). Finally, the reference intervals for all transferrable assays were verified using 100 healthy reference samples from the CALIPER cohort [1], a considerably higher sample size than is recommended in the CLSI C28-A3 guidelines [22], which provides a more stringent verification of the new Beckman Coulter AU reference intervals. The transferred reference intervals were considered verified if N 90% of the CALIPER samples fell within the reference intervals, inclusive of the 95% confidence intervals around the upper and lower limit of each reference interval. Percent verification of reference intervals for all transferred assays are summarized in Table 3. Overall, our analysis revealed that 82.4% (42 out of 51) of the transferred reference intervals were verified, based on our more stringent verification cut-off of 90% of CALIPER samples falling within the transferred reference interval, inclusive of
N C O
197
U
196
T
E
Fig. 1. Statistical criteria employed to assess appropriateness of reference interval transference. Shown is a representative example. (A) Scatter, (B) standardized residual, (C) Bland–Altman, and (D) Q–Q plots for iron (FE), where transference of reference intervals between analyzers was deemed appropriate.
the 95% confidence intervals. If the cut-off value for verification was reduced to 80% of CALIPER samples falling within the confidence intervals of the transferred reference intervals, 98.0% (50 out of 51) of the transferred reference intervals were found to be verified (Table 3).
229
Discussion
233
Establishing new reference intervals is particularly challenging for pediatric populations where sample collection is technically difficult and the need to obtain parental consent poses additional obstacles. The CALIPER program has recruited a large population of healthy children and adolescents and established a new database of pediatric reference intervals for laboratory biomarkers using Abbott ARCHITECT assays. To broaden the application of CALIPER reference standards, we have initiated a series of transference studies to establish a similar database of pediatric reference intervals for biochemical assays on other clinical chemistry platforms. These transference studies have been designed based on CLSI guidelines [22,23] and were recently used to transfer CALIPER reference values to biochemical assays on 4 other clinical chemistry platforms namely, Beckman Coulter DxC800, Ortho Vitros 5600, Roche Cobas 6000, and Siemens Vista 1500 [24]. In the present study, we report transference of CALIPER reference intervals to biochemical assays on the Beckman Coulter AU platform, one of the most commonly used clinical chemistry platforms, based on recent College of American Pathologists (CAP) surveys. Transference of reference intervals is a multistep process that involves studying the correlation between the donor (i.e. Abbott ARCHITECT in this study) and the receiving (i.e. Beckman Coulter AU) methods, establishing the suitability of a linear equation for the transference of reference intervals between the different methods, and finally, verification of the new reference intervals using samples from a healthy population. Each of these steps poses its own challenges. First, a lack of correlation between the different assays can represent a
234
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
230 231 232
235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259
4
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx Table 1 Age- and sex-specific pediatric reference intervals for biochemical markers measured with Beckman Coulter AU assays.
U
N
C
O
R
R
E
C
T
E
D
P
R O
O
F
t1:1 t1:2
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx Table 1 (continued)
U
N C O
R
R
E
C
T
E
D
P
R O
O
F
t1:4
5
(continued on next page)
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
6
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx Table 1 (continued)
U
N
C
O
R
R
E
C
T
E
D
P
R O
O
F
t1:6
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx Table 1 (continued)
t1:10
• Pink and blue highlights indicate female and male-specific reference intervals, respectively. • Assay specific characteristics and analytical parameters are summarized in Supplementary Tables 1 and 2.
7
U
N C O
R
R
E
C
T
E
D
P
R O
O
F
t1:8
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
8
Table 2 Transference of CALIPER reference intervals from Abbott ARCHITECT to Beckman Coulter AU assays. Table lists the Abbott ARCHITECT assays for which reference intervals were transferable to at least one Beckman Coulter AU assay (see Methods).
instability of CO2, which can easily evaporate from the sample during storage, transport, and handling [27]. Thus, analytes with poor stability may not be suitable candidates for transference studies, especially when performing studies between distant laboratories. Another analyte that did not exhibit acceptable correlation between the Abbott ARCHITECHT and the Beckman Coulter AU assays was total calcium (Ca), where the R2 ranged between 0.32 and 0.52. While Abbott has one Ca assay which is Arsenazo-based, Beckman Coulter AU has two Arsenazo-based assays (CaA and CAAO). Similar to the CO2 assays, the lower correlation between Ca assays on both platforms could not be attributed to a difference in the assay principle, as both platforms utilize Arsenazo-based assays. The lack of correlation between assays may be due to using different reagent lots on both platforms or due to using calibrators with different traceability, where Beckman Coulter and Abbott ARCHITECT Ca assay calibrators are traceable to NIST SRM 956 and SRM 909b, respectively. Verification of the transferred reference intervals can pose additional challenges in transference studies. The CLSI guidelines recommend using 20 reference samples from healthy individuals for this purpose, and considers a reference interval valid if N90% of the reference samples fall within the transferred reference interval. Here, we employed a more rigorous criteria by (1) using a minimum of 200 patients to derive the linear equation for transference, (2) applying a more rigorous statistical criteria to examine data clustering and ensure the validity of a linear relationship over the entire data range, (3) determining linear or Deming regression, as appropriate, to accurately describe the correlation between Abbott and Beckman Coulter assays, and finally, (4) testing 100 healthy reference samples for the verification of the transferred reference intervals. CLSI mandates some of these methods for either the validation of an existing manufacturer reference interval or for generating a new reference interval using patient data. Here, we have combined these criteria for a more robust calculation and verification of the new reference intervals. Thus, we selected a verification cutoff of 90% of CALIPER samples falling within the reference interval, inclusive of the 95% confidence intervals, which revealed a verification rate of 82.4% of all transferred reference intervals. The transferred Beckman Coulter AU reference intervals largely agreed with the Abbott ARCHITECT reference intervals. For some biomarkers, there was a high degree of overlap between male and female reference intervals on both analyzers such as urea and creatinine, which reflect the high levels of standardization of these assays. Similar agreement was observed for immunochemical assays such as IgA, ApoB, and high sensitivity CRP. In contrast, notable differences were observed between Abbott reference intervals and the transferred reference intervals for a few analytes, namely, total and direct bilirubin, amylase, lactate dehydrogenase, and ASO. Of these, direct bilirubin showed the largest difference between both analyzers, where Beckman Coulter AU reference intervals for both males and females were lower than the Abbott intervals for all age partitions by at least 60% (Table 1). In summary, we have established the mathematical relationships between Abbott ARCHITECT and Beckman Coulter AU assays based on patient sample comparisons, applied stringent statistical criteria to assess assay-to-assay correlation, calculated AU reference intervals for those assays meeting established criteria, and then verified the new intervals using reference samples from the healthy CALIPER cohort. The new database of Beckman Coulter pediatric reference intervals expands the utility of the CALIPER reference interval database to clinical laboratories using Beckman Coulter AU assays. It is important to clarify that this study was designed to transfer reference intervals to specific assays and does not validate reference intervals for individual analyzers, specific populations, or geographic locations. Individual analyzers may have instrument-specific biases compared to those used in our study. Since local populations may differ in ethnicity, life style, and environmental conditions compared to the multiethnic CALIPER population, it is highly recommended that assay-specific CALIPER reference intervals reported in the present study be validated on each analytical platform
t2:6
260 261 262 263 264 265 266 267 268 269 270 271
U
N
C
O
R
R
E
C
T
E
D
P
R O
O
F
t2:1 t2:2 t2:3 t2:4
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx
major obstacle in transference studies. Here, 9 out of 60 Beckman Coulter AU assays did not correlate well with the corresponding Abbott ARCHIITECT assays. A prominent example is carbon dioxide (CO2), which had two assays on the Beckman Coulter AU system (CO2A and CO2B). Both assays did not correlate with the CO2 assay on the Abbott ARCHITECT system (R2 b 0.2). It is noteworthy to mention that poor correlation does not reflect the accuracy of either the Abbott or the Beckman Coulter assays. This poor correlation is not unique to the Beckman Coulter system since we observed similar poor correlation between the Abbott assay and CO2 assays on other major analyzers [24]. The lack of correlation between the CO2 assays cannot be attributed to a difference in the assay principles and is likely a result of the
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx Table 3 Verification of Beckman Coulter AU pediatric reference intervals using CALIPER reference samplesa.
Acknowledgements
344
This study was supported by an operating grant from the CIHR (Canadian Institutes of Health Research) and Beckman Coulter, Inc. We thank all of the CALIPER participants and their families; this study would not have been possible without their participation. We also thank the CALIPER coordinators, and all of the CALIPER volunteers for their hard work in participant recruitment and sample collection. Lastly, we acknowledge Abbott and Beckman Coulter, for providing reagents, and Dr. Jack Zakowski for his critical review of the manuscript.
345 346
References
353
[1] Colantonio DA, Kyriakopoulou L, Chan MK, Daly CH, Brinc D, Venner AA, et al. Closing the gaps in pediatric laboratory reference intervals: a CALIPER database of 40 biochemical markers in a healthy and multiethnic population of children. Clin Chem 2012;58:854–68. [2] Mansoub S, Chan MK, Adeli K. Gap analysis of pediatric reference intervals for risk biomarkers of cardiovascular disease and the metabolic syndrome. Clin Biochem 2006;39:569–87. [3] Delvin EE, Laxmi Grey V, Vergee Z, CALIPER Working Group. Gap analysis of pediatric reference intervals related to thyroid hormones and the growth hormone-insulin growth factor axis. Clin Biochem 2006;39:588–94. [4] Lepage N, Li D, Kavsak PA, Bamforth F, Callahan J, Dooley K, et al. Incomplete pediatric reference intervals for the management of patients with inborn errors of metabolism. Clin Biochem 2006;39:595–9. [5] Yang L, Grey V. Pediatric reference intervals for bone markers. Clin Biochem 2006; 39:561–8. [6] American Academy of Pediatrics AAP Section on Endocrinology and Committee on Genetics, American Thyroid Association Committee on Public Health. Newborn screening for congenital hypothyroidism: recommended guidelines. Pediatrics 1993;91:1203–9. [7] Cheillan D, Vercherat M, Chevalier-Porst F, Charcosset M, Rolland MO, Dorche C. False-positive results in neonatal screening for cystic fibrosis based on a threestage protocol (IRT/DNA/IRT): should we adjust IRT cut-off to ethnic origin? J Inherit Metab Dis 2005;28:813–8. [8] Mir TS, Flato M, Falkenberg J, Haddad M, Budden R, Weil J, et al. Plasma concentrations of N-terminal brain natriuretic peptide in healthy children, adolescents, and young adults: effect of age and gender. Pediatr Cardiol 2006;27:73–7. [9] Ogunkeye OO, Roluga AI, Khan FA. Resetting the detection level of cord blood thyroid stimulating hormone (TSH) for the diagnosis of congenital hypothyroidism. J Trop Pediatr 2008;54:74–7. [10] Pediatric reference intervals: critical gap analysis and establishment of a national initiative. Clin Biochem 2006;39:559–60. [11] Adeli K. Closing the gaps in pediatric reference intervals: the CALIPER initiative. Clin Biochem 2011;44:480–2. [12] Yip PM, Chan MK, Nelken J, Lepage N, Brotea G, Adeli K. Pediatric reference intervals for lipids and apolipoproteins on the VITROS 5,1 FS Chemistry System. Clin Biochem 2006;39:978–83. [13] Brinc D, Chan MK, Venner AA, Pasic MD, Colantonio D, Kyriakopolou L, et al. Longterm stability of biochemical markers in pediatric serum specimens stored at −80 °C: a CALIPER Substudy. Clin Biochem 2012;45:816–26. [14] Pasic MD, Colantonio DA, Chan MK, Venner AA, Brinc D, Adeli K. Influence of fasting and sample collection time on 38 biochemical markers in healthy children: a CALIPER substudy. Clin Biochem 2012;45:1125–30. [15] Kyriakopoulou L, Yazdanpanah M, Colantonio DA, Chan MK, Daly CH, Adeli K. A sensitive and rapid mass spectrometric method for the simultaneous measurement of eight steroid hormones and CALIPER pediatric reference intervals. Clin Biochem 2013;46:642–51. [16] Konforte D, Shea JL, Kyriakopoulou L, Colantonio D, Cohen AH, Shaw J, et al. Complex biological pattern of fertility hormones in children and adolescents: a study of healthy children from the CALIPER cohort and establishment of pediatric reference intervals. Clin Chem 2013;59:1215–27. [17] Bailey D, Colantonio D, Kyriakopoulou L, Cohen AH, Chan MK, Armbruster D, et al. Marked biological variance in endocrine and biochemical markers in childhood: establishment of pediatric reference intervals using healthy community children from the CALIPER cohort. Clin Chem 2013;59:1393–405. [18] Shaw JLV, Cohen A, Konforte D, Binesh-Marvasti T, Colantonio DA, Adeli K. Validity of establishing pediatric reference intervals based on hospital patient data: a comparison of the modified Hoffmann approach to CALIPER reference intervals obtained in healthy children. Clin Biochem 2014;47:166–72. [19] Bailey D, Bevilacqua V, Colantonio DA, Pasic MD, Perumal N, Chan MK, et al. Pediatric within-day biological variation and quality specifications for 38 biochemical markers in the CALIPER cohort. Clin Chem 2014;60:518–29. [20] Raizman JE, Cohen AH, Teodoro-Morrison T, Wan B, Khun-Chen M, Wilkenson C, et al. Pediatric reference value distributions for vitamins A and E in the CALIPER cohort and establishment of age-stratified reference intervals. Clin Biochem 2014;47:812–5. [21] Adeli K. Closing the gaps in pediatric reference intervals: an update on the CALIPER project. Clin Biochem 2014;47:737–9. [22] CLSI. Defining, establishing and verifying referene intervals in the clinical laboratory; aproved guidlines. CLSI Document C28-A3c 3rd ed. Wayne, PA: Clinical and Laboratory Standards Institute; 2008.
354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422
338 339
a
% verification indicates the percentage of CALIPER samples (n = 100) that fell within the appropriate partitioned upper and lower reference limits inclusive of the 95% confidence intervals (see Methods). Numbers within parentheses are percentage of samples that fell within the appropriate partitioned upper and lower reference limits of the Beckman Coulter AU assays, not including the 95% confidence intervals. Assays are highlighted with different colors based on the % verification: white (verification N90%), grey (verification N80%), and black (verification b80%).
U
t3:5 t3:6 t3:7 t3:8 t3:9 t3:10 t3:11
N C O
R
R
E
C
T
E
D
P
R O
O
F
t3:1 t3:2 t3:3
9
340 341
using reference specimens from healthy children in the local population as recommended by the CLSI. Supplementary data to this article can be found online at http://dx. doi.org/10.1016/j.clinbiochem.2015.05.002.
342 Q4
Uncited reference
343
[26]
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
347 348 349 350 351 352
10
[25] R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing; 2013(http://www.R-projct.org/). [26] Anscombe F. Graphs in Statistical Analysis. 1973;27:17–22. [27] Bandi ZL. Estimation, prevention, and quality control of carbon dioxide loss during aerobic sample processing. Clin Chem 1981;27:1676–81.
N
C
O
R
R
E
C
T
E
D
P
R O
O
F
[23] CLSI. Measurement Procedure Comparison and Bias Estimation Using Patient Samples: Approved Guideline. CLSI Document EP09-A33rd ed. Wayne, PA: Clinical and Laboratory Standards Institute; 2013. [24] Estey MP, Cohen AH, Colantonio DA, Chan MK, Marvasti TB, Randell E, et al. CLSIbased transference of the CALIPER database of pediatric reference intervals from Abbott to Beckman, Ortho, Roche and Siemens Clinical Chemistry Assays: direct validation using reference samples from the CALIPER cohort. Clin Biochem 2013; 46:1197–219.
U
423 424 425 426 427 428 429 430
M.A. El Hassan et al. / Clinical Biochemistry xxx (2015) xxx–xxx
Please cite this article as: El Hassan MA, et al, CLSI-based transference of CALIPER pediatric reference intervals to Beckman Coulter AU biochemical assays, Clin Biochem (2015), http://dx.doi.org/10.1016/j.clinbiochem.2015.05.002
431 432 433 434 435