- Email: [email protected]
Author Reply: “Correlation of Aspiration Results with Periprosthetic Sepsis in Revision Total Hip Arthroplasty”
The Journal of Arthroplasty 29 (2014) 1329–1330
Contents lists available at ScienceDirect
The Journal of Arthroplasty journal homepage: www.arthroplastyjournal.org
Letters to the Editor Letter to the Editor Dear Editor, We are reading with interest the article by Chalmers et al. regarding correlation of aspiration results with periprosthetic sepsis in revision total hip arthroplasty [1]. Interestingly the cutoff for the segmented cell differential percentile at 73.6% was chosen correctly (red line of ROC curve) with a sensitivity of 90% and a specificity of 76%. However the cutoff for the synovial WBC count (grey line of ROC curve) was not chosen at the optimal cutoff point with 745 WBC/μl. The big arrow shows the point, where the cutoff 745 achieves a sensitivity of 96% and a 1-specificity of 55% (Fig. 1). The closest point (dotted line)to perfect discrimination (0,1) and the Youden point (blue line), which maximises the Youden Index (sens + spec-1), is at 82% sensitivity and 90% specificity in the depicted ROC curve, which will be between readings of 1000–1500 WBC/μl [2,3]. The authors can check the performance at each reading, when they look at the coordinates of the ROC plot, which is an available option for ROC curves on SPSS 18. This explains why the AAOS threshold of 1700 WBC/μl for TKR infection had such a good performance in their validation cohort. The ROC plot for the synovial WBC count of 57 hip revision arthroplasties (THA) in our hospital, of which 17 were infected, showed the best performance at 1485 WBC/μl with a sensitivity of 88% and specificity of 92.5% for the diagnosis of infection. The area under the curve was excellent with 0.96 in our validation cohort. This cut-off
was consistent with our previous publication on 27 THAs, when we determined at an optimal cut-off of 1425/μl a sensitivity of 89% and specificity of 91% [4]. The ROC plot for synovial neutrophilia in our recent cohort of 57 THAs found a cut-off of 65%, which is lower than the 73.6% by Chalmers et al. and consistent with the AAOS cut-off. Andrea Guyot Department of Microbiology, Frimley Park Hospital, Frimley, UK William MacClutchie Department of Orthopaedics, Frimley Park Hospital, Frimley, UK Fabiana Gordon Statistical Advisory Service; Imperial College South Kensington Campus, London, UK http://dx.doi.org.10.1016/j.arth.2013.11.029 References 1. Chalmers PN, et al. Correlation of aspiration results with periprosthetic sepsis in revision total hip arthroplasty. J Arthroplasty 2014;29(2):438. 2. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143(1):29. 3. Zweig MH, Campbell G. Receiver operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry 1993;39(4):561. 4. Dineen A, et al. Synovial fluid white cell and differential count in the diagnosis or exclusion of prosthetic joint infection. Bone Joint J 2013;95-B:554.
Author Reply: “Correlation of Aspiration Results with Periprosthetic Sepsis in Revision Total Hip Arthroplasty” In Reply:
Fig. 1. Receiver operating-characteristic (ROC) curves for the diagnostic accuracy of each synovial aspirate.
0883-5403/© 2014 Elsevier Inc. All rights reserved.
We thank you for the opportunity to reply to the comments on our paper entitled “Correlation of Aspiration Results with Periprosthetic Sepsis in Revision Total Hip Arthroplasty” [1], and we are grateful to the authors of the letter for their interest in our study. We are also interested to hear that similar results in support of the current American Academy of Orthopaedic Surgeons Clinical Practice Guidelines (AAOS CPG) [2] have been obtained at their institution in both published [3] and unpublished results. When continuous laboratory data must be converted to a binary diagnosis, any selected “cut-off” value will necessarily be a compromise between sensitivity and specificity. The optimal cut-off thus depends upon the clinical situation in which the test is utilized and will depend upon the prevalence of the condition tested, the costs and consequences of a false positive, and the costs and consequences of a false negative diagnosis. No single mathematical formula can therefore be used to determine the “optimal” cutoff across all clinical tests—an understanding of the clinical situation in which the test is applied must also be considered.
1330
Letters to the Editor / The Journal of Arthroplasty 29 (2014) 1329–1330
The most commonly used [4,5] methods of selecting a cut-off involves maximizing Youden’s Index (sensitivity + specificity − 1) [6], or balanced accuracy [7], both of which maximize the probability that the cutoff will correctly diagnose periprosthetic sepsis assuming equal distributions of “infected” and “not infected” arthroplasties. The authors of the letter regarding our study are absolutely correct that our study did not use the Youden’s statistic. Using this method within our dataset returns a cutoff value of 3936 WBC for a sensitivity of 85%, a specificity of 93%, and an accuracy of 86%. Surgeons who prefer a more specific test may prefer this methodology. For our study, we a priori choose to maximize accuracy, instead of maximizing the Youden Index, as this method more fully accounts for the infection prevalence, and thus better balances the effects of false negatives. As stated in our article, our data fully support the use of the current AAOS CPG WBC cutoff of 1700 as it returns a high accuracy within our data set. However, surgeons should be aware that this cutoff provides a sensitivity of 88–92% in our data set and that of the authors of the letter regarding our study [3], and thus use of these criteria will result in an 8–12% false negative rate [1]. Use of the Youden Index, which over-estimates the prevalence of infection and underestimates the prevalence of other causes of failure, worsens this problem. For periprosthetic sepsis the consequences of a false negative and a concomitant misdiagnosis of aseptic loosening are significant for the patient and should not be underestimated.
Peter N. Chalmers, MD Scott M. Sporer, MD Brett R. Levine, MD Department of Orthopedic Surgery Rush University Medical Center Chicago, IL, USA http://dx.doi.org.10.1016/j.arth.2013.12.007
References 1. Chalmers P, Sporer SM, Levine BR. Correlation of aspiration results with periprosthetic sepsis in revision total hip arthroplasty. J Arthroplasty 2013 [Epub ahead of print]. 2. Valle Della C, Parvizi J, Bauer TW, et al. American Academy of Orthopaedic Surgeons clinical practice guideline on: the diagnosis of periprosthetic joint infections of the hip and knee. The Journal of Bone and Joint Surgery 2011;93:1355. 3. Dinneen A, Guyot A, Clements J, et al. Synovial fluid white cell and differential count in the diagnosis or exclusion of prosthetic joint infection. Bone Joint J 2013;95-B:554. 4. Cipriano CA, Brown NM, Michael AM, et al. Serum and synovial fluid analysis for diagnosing chronic periprosthetic infection in patients with inflammatory arthritis. J Bone Joint Surg 2012;94:594. 5. Schinsky MF, Valle Della CJ, Sporer SM, et al. Perioperative testing for joint infection in patients undergoing revision total hip arthroplasty. J Bone Joint Surg 2008;90:1869. 6. Youden WJ. Index for rating diagnostic tests. Cancer 1950;3:32. 7. Chalmers P, Sporer SM, Levine BR. Correlation of aspiration results with aseptic loosening in total hip arthroplasty. J Arthroplasty 2013;28:1671.
Contents lists available at ScienceDirect
The Journal of Arthroplasty journal homepage: www.arthroplastyjournal.org
Letters to the Editor Letter to the Editor Dear Editor, We are reading with interest the article by Chalmers et al. regarding correlation of aspiration results with periprosthetic sepsis in revision total hip arthroplasty [1]. Interestingly the cutoff for the segmented cell differential percentile at 73.6% was chosen correctly (red line of ROC curve) with a sensitivity of 90% and a specificity of 76%. However the cutoff for the synovial WBC count (grey line of ROC curve) was not chosen at the optimal cutoff point with 745 WBC/μl. The big arrow shows the point, where the cutoff 745 achieves a sensitivity of 96% and a 1-specificity of 55% (Fig. 1). The closest point (dotted line)to perfect discrimination (0,1) and the Youden point (blue line), which maximises the Youden Index (sens + spec-1), is at 82% sensitivity and 90% specificity in the depicted ROC curve, which will be between readings of 1000–1500 WBC/μl [2,3]. The authors can check the performance at each reading, when they look at the coordinates of the ROC plot, which is an available option for ROC curves on SPSS 18. This explains why the AAOS threshold of 1700 WBC/μl for TKR infection had such a good performance in their validation cohort. The ROC plot for the synovial WBC count of 57 hip revision arthroplasties (THA) in our hospital, of which 17 were infected, showed the best performance at 1485 WBC/μl with a sensitivity of 88% and specificity of 92.5% for the diagnosis of infection. The area under the curve was excellent with 0.96 in our validation cohort. This cut-off
was consistent with our previous publication on 27 THAs, when we determined at an optimal cut-off of 1425/μl a sensitivity of 89% and specificity of 91% [4]. The ROC plot for synovial neutrophilia in our recent cohort of 57 THAs found a cut-off of 65%, which is lower than the 73.6% by Chalmers et al. and consistent with the AAOS cut-off. Andrea Guyot Department of Microbiology, Frimley Park Hospital, Frimley, UK William MacClutchie Department of Orthopaedics, Frimley Park Hospital, Frimley, UK Fabiana Gordon Statistical Advisory Service; Imperial College South Kensington Campus, London, UK http://dx.doi.org.10.1016/j.arth.2013.11.029 References 1. Chalmers PN, et al. Correlation of aspiration results with periprosthetic sepsis in revision total hip arthroplasty. J Arthroplasty 2014;29(2):438. 2. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143(1):29. 3. Zweig MH, Campbell G. Receiver operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry 1993;39(4):561. 4. Dineen A, et al. Synovial fluid white cell and differential count in the diagnosis or exclusion of prosthetic joint infection. Bone Joint J 2013;95-B:554.
Author Reply: “Correlation of Aspiration Results with Periprosthetic Sepsis in Revision Total Hip Arthroplasty” In Reply:
Fig. 1. Receiver operating-characteristic (ROC) curves for the diagnostic accuracy of each synovial aspirate.
0883-5403/© 2014 Elsevier Inc. All rights reserved.
We thank you for the opportunity to reply to the comments on our paper entitled “Correlation of Aspiration Results with Periprosthetic Sepsis in Revision Total Hip Arthroplasty” [1], and we are grateful to the authors of the letter for their interest in our study. We are also interested to hear that similar results in support of the current American Academy of Orthopaedic Surgeons Clinical Practice Guidelines (AAOS CPG) [2] have been obtained at their institution in both published [3] and unpublished results. When continuous laboratory data must be converted to a binary diagnosis, any selected “cut-off” value will necessarily be a compromise between sensitivity and specificity. The optimal cut-off thus depends upon the clinical situation in which the test is utilized and will depend upon the prevalence of the condition tested, the costs and consequences of a false positive, and the costs and consequences of a false negative diagnosis. No single mathematical formula can therefore be used to determine the “optimal” cutoff across all clinical tests—an understanding of the clinical situation in which the test is applied must also be considered.
1330
Letters to the Editor / The Journal of Arthroplasty 29 (2014) 1329–1330
The most commonly used [4,5] methods of selecting a cut-off involves maximizing Youden’s Index (sensitivity + specificity − 1) [6], or balanced accuracy [7], both of which maximize the probability that the cutoff will correctly diagnose periprosthetic sepsis assuming equal distributions of “infected” and “not infected” arthroplasties. The authors of the letter regarding our study are absolutely correct that our study did not use the Youden’s statistic. Using this method within our dataset returns a cutoff value of 3936 WBC for a sensitivity of 85%, a specificity of 93%, and an accuracy of 86%. Surgeons who prefer a more specific test may prefer this methodology. For our study, we a priori choose to maximize accuracy, instead of maximizing the Youden Index, as this method more fully accounts for the infection prevalence, and thus better balances the effects of false negatives. As stated in our article, our data fully support the use of the current AAOS CPG WBC cutoff of 1700 as it returns a high accuracy within our data set. However, surgeons should be aware that this cutoff provides a sensitivity of 88–92% in our data set and that of the authors of the letter regarding our study [3], and thus use of these criteria will result in an 8–12% false negative rate [1]. Use of the Youden Index, which over-estimates the prevalence of infection and underestimates the prevalence of other causes of failure, worsens this problem. For periprosthetic sepsis the consequences of a false negative and a concomitant misdiagnosis of aseptic loosening are significant for the patient and should not be underestimated.
Peter N. Chalmers, MD Scott M. Sporer, MD Brett R. Levine, MD Department of Orthopedic Surgery Rush University Medical Center Chicago, IL, USA http://dx.doi.org.10.1016/j.arth.2013.12.007
References 1. Chalmers P, Sporer SM, Levine BR. Correlation of aspiration results with periprosthetic sepsis in revision total hip arthroplasty. J Arthroplasty 2013 [Epub ahead of print]. 2. Valle Della C, Parvizi J, Bauer TW, et al. American Academy of Orthopaedic Surgeons clinical practice guideline on: the diagnosis of periprosthetic joint infections of the hip and knee. The Journal of Bone and Joint Surgery 2011;93:1355. 3. Dinneen A, Guyot A, Clements J, et al. Synovial fluid white cell and differential count in the diagnosis or exclusion of prosthetic joint infection. Bone Joint J 2013;95-B:554. 4. Cipriano CA, Brown NM, Michael AM, et al. Serum and synovial fluid analysis for diagnosing chronic periprosthetic infection in patients with inflammatory arthritis. J Bone Joint Surg 2012;94:594. 5. Schinsky MF, Valle Della CJ, Sporer SM, et al. Perioperative testing for joint infection in patients undergoing revision total hip arthroplasty. J Bone Joint Surg 2008;90:1869. 6. Youden WJ. Index for rating diagnostic tests. Cancer 1950;3:32. 7. Chalmers P, Sporer SM, Levine BR. Correlation of aspiration results with aseptic loosening in total hip arthroplasty. J Arthroplasty 2013;28:1671.