- Email: [email protected]
Next-Generation Sequencing vs Culture-Based Methods for Diagnosing Periprosthetic Joint Infection After Total Knee Arthroplasty: A Cost-Effectiveness Analysis
The Journal of Arthroplasty 34 (2019) 1333e1341
Contents lists available at ScienceDirect
The Journal of Arthroplasty journal homepage: www.arthroplastyjournal.org
Health Policy & Economics
Next-Generation Sequencing vs Culture-Based Methods for Diagnosing Periprosthetic Joint Infection After Total Knee Arthroplasty: A Cost-Effectiveness Analysis Michael T. Torchia, MD, MS a, *, Daniel C. Austin, MD, MS a, Samuel T. Kunkel, MD, MS a, Kevin W. Dwyer, MD, MS a, b, Wayne E. Moschetti, MD, MS a, b a b
Department of Orthopaedics, Dartmouth-Hitchcock Medical Center, Lebanon, NH Department of Orthopaedics, Geisel School of Medicine, Dartmouth College, Lebanon, NH
a r t i c l e i n f o
a b s t r a c t
Article history: Received 16 January 2019 Received in revised form 17 February 2019 Accepted 8 March 2019 Available online 19 March 2019
Background: Periprosthetic joint infection (PJI) after total knee arthroplasty is challenging to diagnose. Compared with culture-based techniques, next-generation sequencing (NGS) is more sensitive for identifying organisms but is also less specific and more expensive. To date, there has been no study comparing the cost-effectiveness of these two methods to diagnose PJI after total knee arthroplasty. Methods: A Markov, state-transition model projecting lifetime costs and quality-adjusted life years (QALYs) was constructed to determine the cost-effectiveness from a societal perspective. The primary outcome was incremental cost-effectiveness ratio, with a willingness-to-pay threshold of $100,000/QALY. Sensitivity analyses were performed to evaluate parameter assumptions. Results: At our base case values, culture was not determined to be cost-effective compared to NGS, with an incremental cost-effectiveness ratio of $422,784 per QALY. One-way sensitivity analyses found NGS to be the cost-effective choice above a pretest probability of 45.5% for PJI. In addition, NGS was costeffective if its sensitivity was greater than 70.0% and its specificity greater than 94.1%. Two-way sensitivity analyses revealed that the pretest probability and test performance parameters (sensitivity and specificity) were the largest factors for identifying whether a particular strategy was cost-effective. Conclusion: The results of our model suggest that the cost-effectiveness of NGS to diagnose PJI depends primarily on the pretest probability of PJI and the performance characteristics of the NGS technology. Our results are consistent with the idea that NGS should be reserved for clinical contexts with a high pretest probability of PJI. Further study is required to determine the indications and subgroups for which NGS offers clinical benefit. © 2019 Elsevier Inc. All rights reserved.
Keywords: next-generation sequencing periprosthetic joint infection cost-effectiveness analysis total knee arthroplasty
Total knee arthroplasty (TKA) is one of the most successful surgical interventions in the modern medical era [1e4], and the number of TKA procedures is expected to increase in the coming decades [5,6]. Periprosthetic joint infection (PJI) is a rare but devastating
One or more of the authors of this paper have disclosed potential or pertinent conflicts of interest, which may include receipt of payment, either direct or indirect, institutional support, or association with an entity in the biomedical field which may be perceived to have potential conflict of interest with this work. For full disclosure statements refer to https://doi.org/10.1016/j.arth.2019.03.029. * Reprint requests: Michael T. Torchia, MD, MS, Department of Orthopaedics, 1 Medical Center Drive, Lebanon, NH 03756-0001. https://doi.org/10.1016/j.arth.2019.03.029 0883-5403/© 2019 Elsevier Inc. All rights reserved.
complication of TKA, and the numbers of PJIs are projected to increase in concert with the increased number of total knees being performed [7,8]. Despite the existence of clinical guidelines and specialty society definitions for PJI [9,10], diagnosis remains challenging [9,11e13]. Indeed, current literature shows that when using traditional culture-based techniques, a causative organism is not found in upwards of 20% of cases [14e18]. Molecular techniques, which use polymerase chain reaction to amplify and identify DNA in a sample, have been proposed as a solution to overcome several prior challenges of diagnosing PJI [19,20]. Next-generation sequencing (NGS) for diagnosing PJI permits the sequencing of all DNA present in a given sample [21]. Despite its
1334
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
increased sensitivity for identifying organisms compared to culture [20], tradeoffs exist in its clinical application. First, NGS is more expensive than cultures, even when taking into account the multiple cultures recommended for diagnosis of PJI [10]. Second, while NGS is more sensitive in identifying organisms compared to culture, it is also less specific, and therefore, concern exists about the increased rate of false-positive results [19,20]. Finally, NGS is a new technology, and there is still debate about how to interpret its results and in which clinical contexts it should be used. To date, there has been no cost-effectiveness analysis comparing NGS to culture for the diagnosis of periprosthetic infection and, specifically, no analysis that models the long-term consequences of false-positive and false-negative test results. Given the multiple cost and utility tradeoffs involved between culture and NGS testing strategies, the topic is particularly amenable to a cost-effectiveness analysis. The purpose of this study was to compare the costeffectiveness of NGS and culture for the diagnosis of PJI after total hip arthroplasty and TKA.
Model Structure A Markov, state-transition simulation model was created to determine the cost-effectiveness of culture compared to NGS for the diagnosis of knee PJI. The model projects lifetime costs and quality-adjusted life years (QALYs) and considers costs from a societal perspective. Costs were expressed in 2018 US dollars and were obtained from a combination of the published literature, company representatives, and internal hospital costs. Health state utilities and probabilities were obtained from the published literature, expert opinion, and US life tables [23]. Costs and utilities were discounted at 3% per year in accordance with current practices of cost-effectiveness analysis [24], and the willingness-to-pay threshold was set at 100,000/QALY [25]. The model was created using commercially available decision analysis software (TreeAge Pro 2017; TreeAge Software, Williamstown, MA). A diagram of the chance nodes before entry into the Markov nodes is shown in Figure 1, and a figure showing the treatment strategies, health states, and clinical events during each cycle is shown in Figure 2.
Methods Model Parameters Patient Population The population of interest for our analysis was patients with a primary TKA who were suspected of having a PJI. The age of the patient in our base case analysis was 65 years, determined from the mean age of revision TKA in the Medicare population [22], with a pretest probability of PJI of 50% to model clinical equipoise in the diagnosis of PJI.
Several assumptions were made regarding the model design. First, we assumed that an organism identified by NGS would be the same as that identified by culture based on the 96.1% concordance rate between NGS and culture reported by Shohat et al [19]. Second, we assumed that the performance characteristics (ie, sensitivity and specificity) of NGS were based on the situation in which it identifies a single organism representing the majority of bacteria
Fig. 1. Chance nodes modeling pretest probability, sensitivity, and specificity of each test before entry into Markov nodes. NGS, next-generation sequencing; PJI, periprosthetic joint infection; TJA, total joint arthroplasty.
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
1335
Fig. 2. Diagram showing treatment strategies, health states, and clinical events in the model. Patients enter the model with a primary TJA and concern for infection and undergo either culture or NGS to test for PJI. Test results are true positives, true negatives, false positives, or false negatives (see Fig. 1). Patients who have true-positive, false-positive, or false-negative results of testing undergo further revision surgery. Patients with true-negative test results do not undergo any further surgery and stay in a “post-TJA” state with a baseline annual risk of death. Patients who undergo further revision surgery either clear their infection, do not clear their infection, or die, with different probabilities depending on whether they started with a true-positive, false-positive, or false-negative test result. Patients can only undergo 3 failed revision surgeries before they are placed in the “uncured infection” state.
Table 1 Probabilities, Utilities, and Costs Used in the Model.
Probabilities Sensitivity of NGS Specificity of NGS Sensitivity of culture Specificity of culture Probability of clearing infection after revision Probability of perioperative death Probability of all-cause mortality Probability of death with continued infection Baseline probability of PJI in noninfected joint Utilities Utility of postprimary TJA Utility of successful revision arthroplasty Utility after second revision Utility of uncured infection Interval disutility of revision surgery Interval disutility of salvage operation Costs (in 2018 US dollars) Cost of NGS Cost of cultures (2) Cost of revision for PJI Cost of aseptic revision Cost of salvage procedure Cost yearly after TJA PJI, periprosthetic joint infection; TJA, total joint arthroplasty.
Base Case Value
Source
0.714 0.946 0.607 0.973 0.791 0.011 Variable 0.106 0.0092
Tarabichi et al [20] Tarabichi et al [20] Tarabichi et al [20] Tarabichi et al [20] Wu et al [27] Slover et al [28] 2014 US Life Tables [23] Zmistowski et al [29] Kurtz et al [8]
0.68 0.532 0.452 0.372 0.1 0.2 $250 $100 $31,158 $20,457 $15,154 $148
Slover et al [28] Slover et al [28] Expert opinion based on values from Slover et al [28] Expert opinion based on values from Slover et al [28] Assumption (expert opinion) Assumption (expert opinion) Company representative Hospital internal data Parvizi et al [30] Hospital internal data Hospital internal data Losina et al [1]
1336
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
Fig. 3. One-way sensitivity analysis evaluating the effect of the pretest probability of PJI on the cost-effectiveness of culture and NGS for the diagnosis of PJI. WTP, willingness to pay.
present in a sample as previously reported by Tarabichi et al [20], and not the situation where it detects any bacteria present. Third, we assumed that culture and NGS data would be the only decision point to proceed with revision surgery and that the type of revision surgery would be predicated upon culture and NGS results. Fourth, we assumed that patients with PJI underwent a 2-stage revision arthroplasty as this is currently the standard of care in the United States [26]. We did not explicitly model the risks and complications associated with prolonged intravenous antibiotic therapy. Finally, we modeled an “uncleared infection” state based on utilities combining knee fusion, above-knee amputation, and chronic suppression, given the diversity of choices available for this clinical situation. We performed extensive sensitivity analyses to test this assumption at all ranges of clinically relevant utilities and costs for this health state. Probabilities Transition probabilities used in our study were obtained from the literature and are presented in Table 1. We used differential base-rate annual mortality based on the most recent US life tables (2014) [23]. We assumed the base-rate annual mortality in patients who had cleared their infection to be equal to the base-rate annual mortality of age-matched men and women. The 1-year mortality of patients with an uncleared infection was 10.6% [29]. The perioperative mortality for a revision total joint arthroplasty was set at 1.1% based on Medicare data [28]. We assumed the probability of perioperative mortality from a second revision or salvage operation
would be equal to that of an initial revision arthroplasty. The values for sensitivity and specificity of both NGS and culture were obtained from the published literature [20]. The baseline probability of a PJI was determined to be 0.92% based on a study utilizing the Nationwide Inpatient Sample [8]. The model assumed that all patients with a PJI would undergo a two-stage revision arthroplasty, and the probability of clearing an infection after a two-stage reimplantation was 79.1% based on a 2014 systematic review [27]. Utilities Health state utilities were obtained from the published literature. Specifically, the utility of a postprimary TKA and the utility of the postoperative state after the first TKA revision were taken from a prior cost-effectiveness analysis of patients undergoing TKA [28]. The utilities for a continued infection after a second revision and the utility for an uncleared infection, which in our model was a combination of knee fusion, above-knee amputation, and chronic suppression, were not found in the literature. Thus, a consensus based on the expert opinion of the two senior-most authors of the manuscript was reached, and the utility for a continued infection after a second revision was assumed to be a 15% reduction from a successful revision, and the utility of an uncleared infection was assumed to be a 30% reduction from a successful revision. The utility for the "dead" state was set at zero. To account for the recovery period after surgery, an interval disutility was applied after surgical procedures. The disutility for revision surgery was valued at 0.1 as previously described in the total joint arthroplasty cost-
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
1337
Fig. 4. One-way sensitivity analysis evaluating the cost of NGS on the cost-effectiveness of culture and NGS for the diagnosis of PJI.
effectiveness literature [28], and the interval disutility for a salvage procedure was assumed to be double that of the revision surgery ( 0.2), given the increased complexity of salvage operations and the potential for limb shortening and presumed difficulty ambulating. All health state utilities and tolls are shown in Table 1. We found the utility values used in our analysis to be consistent with previously published values [27,31e33], but given the degree of uncertainty inherent in estimating utilities when reviewing studies of variable quality, multiple sensitivity analyses were performed. Costs Costs were derived from the published literature and internal hospital data and converted to 2018 US dollars using the consumer price index calculator from the United State Department of Labor [34]. Costs are shown in Table 1. No indirect cost considerations related to loss of employment from the effects of surgery were considered as we assumed only a small proportion of patients in the age range used in the model would still remain employed. The cost of revision for PJI was taken as the average cost of a revision for a TKA and total hip arthroplasty from a published analysis of costs using a nationally representative sample [35]. Costs for aseptic revisions, salvage procedures, and cultures were obtained from internal hospital data. The cost of a salvage procedure was assumed to be the average cost of a knee fusion, an above-above knee amputation, and chronic suppression with antibiotics. Consistent with the musculoskeletal infection society (MSIS) guidelines for the diagnosis of PJI [10], we assumed the cost of two cultures when
calculating culture costs for our base case scenario but varied the cost of cultures in multiple sensitivity analyses as typically more than 2 cultures are taken in the setting of revision arthroplasty for PJI. Finally, costs of NGS were obtained from a MicroGen sales representative (MicroGen Diagnostics, Lubbock, TX). Analysis The primary outcome of the study was the incremental costeffectiveness ratio, which is defined as the ratio of the difference in cost divided by the difference in effectiveness, as measured by QALYs, of the two strategies. The cycle length of the model was 1 year, and the model was run for a total of 30 cycles to simulate a clinically relevant time period for patients. Variation among model parameters was assessed with 1-way, 2-way, and 3-way deterministic sensitivity analyses. Probability and utility parameters were varied across all possible values, and cost values were varied up to a point exceeding the highest estimate cost for each parameter. Results Base Case Results Using our base case values, the total cost for the culture strategy was $28,828 with total accrued QALYs of 9.444, whereas the total cost for the NGS strategy was $26,399 with total accrued QALYs of 9.439. Based on these values, culture had an incremental cost-
1338
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
Fig. 5. Two-way sensitivity analysis evaluating pretest probability of PJI and the cost of NGS on the cost-effectiveness of culture and NGS for the diagnosis of PJI.
effectiveness ratio of $422,784 compared to NGS when used to diagnose PJI. Using the a priori willingness-to-pay threshold of $100,000, culture was therefore not found to be cost-effective when compared to NGS.
Sensitivity Analysis One-way sensitivity analysis showed that NGS was cost-effective above a pretest probability for PJI of 45.5% (Fig. 3). Additional 1-way sensitivity analyses showed that NGS was cost-effective if its sensitivity was greater than 70.0% and its specificity greater than 94.1%. Using NGS was also found to be cost-effective until its price reached $3915 (Fig. 4), if the probability of eradication of infection from revision arthroplasty was greater than 42.9%, if the utility of continued infection was less than 0.77, and if the utility of being infection free was above 0.43. Conversely, at base case values, culture was deemed to be cost-effective if the utility of continued infection was higher than 0.77 and if the utility after a second revision was higher than 0.98. At base case values, NGS was found to be always more cost-effective than culture at all costs of culture (analysis stopped at a culture cost of $5000), all costs of aseptic revision arthroplasty, all costs of septic revision arthroplasty, and salvage procedures (analysis stopped at costs for both procedures of $200,000), and at all utilities of the “well post-op” state. Two-way sensitivity analyses demonstrated that the pretest probability of PJI and the test performance parameters (sensitivity and specificity) of both culture and NGS were the prime drivers
behind whether a particular strategy was deemed cost-effective. A 2-way sensitivity analysis evaluating the pretest probability of PJI and the cost of NGS showed that for a pretest probability above 55%, NGS was cost-effective, regardless of its cost (Fig. 5); conversely, for pretest probabilities less than 45%, culture was cost-effective, regardless of its cost. This relationship was also observed in a two-way analysis of pretest probability and the cost of PJI revision: above a pretest probability of 50%, NGS was cost-effective, regardless of its cost. Furthermore, 2-way sensitivity analyses evaluating the specificity and sensitivity of NGS revealed that NGS is cost-effective within a relatively narrow range of specificity and sensitivity (Fig. 6), and this range corresponds closely to its published performance parameters (sensitivity of 71.4%, specificity of 94.6%) [20]. In addition, the same dynamic is observed in a 2-way sensitivity analysis of the sensitivity and specificity of culture: culture is cost-effective within a relatively narrow range of performance parameters that correspond closely to its previously published values (sensitivity of 60.7%, specificity of 97.3%) [20], and small deviations of these parameters determine which diagnostic strategy is favored (Fig. 7).
Discussion Diagnosis of PJI can be challenging after TKA. Both NGS and culture have multiple cost and utility tradeoffs, making their comparison amenable to a cost-effectiveness analysis. The results of this model demonstrate that based on the current published performance
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
1339
Fig. 6. Two-way sensitivity analysis evaluating sensitivity and specificity of NGS on the cost-effectiveness of culture and NGS for the diagnosis of PJI. Analysis shown is at a base case pretest probability for PJI of 50%.
parameters of NGS and culture, NGS is a cost-effective method to diagnose PJI, provided there is a sufficiently high pretest probability of infection (48.3%). Moreover, the pretest probability of PJI and the performance parameters of both culture and NGS are the primary drivers as to whether culture or NGS is deemed cost-effective. Varying the costs of NGS, culture, and revision surgeries, as well as the utilities of the various health states within clinically reasonable parameters did not change the cost-effectiveness of NGS, provided the pretest probability was more than 48.3%. The results of our study should be interpreted carefully. First, this study does not aim to determine whether NGS should be used in current clinical practice. As a new technology used in the context of PJI, further basic science and clinical studies are required to determine how its results should be interpreted and whether its performance characteristics vary in certain subgroups. However, if NGS will be used in the same way as culture to diagnose PJI in the future, our model does permit greater insights into the clinical contexts when NGS would be most cost-effective to use. From that perspective, our study implies that NGS is best suited to clinical scenarios where the pretest probability of PJI is high. This is consistent with current literature on the use of NGS to diagnose PJI. Indeed, the most recent paper on the application of NGS to the diagnosis of PJI stated, “…the pre-test probability determined by the clinical picture and other laboratory investigations should be closely examined when interpreting the results of next-generation sequencing.” [20] Moreover, the results of our model supporting the idea that NGS should be used as a confirmatory tool, rather than
a screening tool, is remarkably consistent with the conclusions reached by other authors regarding a different molecular test to diagnose PJI with similar performance characteristics as NGS [36]. Our model’s consistency with previously published literature on the topic lends a degree of face validity to the model. The finding in our model that NGS and culture are cost-effective within relatively narrow performance parameters (ie, sensitivity and specificity values), which correspond closely to their published parameters, has several implications. First, relatively small increases or decreases in the performance of either diagnostic test can lead to changes in which diagnostic strategy is favored in terms of cost-effectiveness. Second, our findings imply that further work should be done to determine particular clinical contexts where one test is clearly superior or inferior to the other. For example, alphadefensin, a highly sensitive and specific molecular test used to diagnose PJI, has been shown to have inferior performance characteristics in cases of adverse local tissue reaction secondary to metal-on-metal wear or head-neck junction corrosion [37]. Similarly, if there are other contexts where NGS or culture has clearly inferior performance characteristics, then an alternative diagnostic strategy may be favored given the narrow performance characteristics required for either culture or NGS to be cost-effective. The use of NGS and culture in situations such as severe metallosis in metalon-metal bearing surfaces, catastrophic polyethylene wear, implanted antibiotic spacers, and trunnionosis from junctional corrosion merit further study. Finally, given how new NGS technology is to the field of orthopedic surgery and the fact that its
1340
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
Fig. 7. Two-way sensitivity analysis evaluating sensitivity and specificity of culture on the cost-effectiveness of culture and NGS for the diagnosis of PJI. Analysis shown is at a base case pretest probability for PJI of 50%.
published performance characteristics were determined at one institution, our finding implies the need for future work to validate the performance characteristics of NGS in other settings. We acknowledge several limitations. First, model inputs were derived primarily from the published literature, and there was a degree of variation in the published values of different health states and costs in the literature. We are reassured, however, that extensive sensitivity analyses demonstrated our model to be robust to all clinically reasonable utility and cost values. It is also reassuring that while we did not explicitly model the cost of rehabilitation or home health which a proportion of patients inevitably require after surgery, the costs of surgery could be increased in sensitivity analysis to reflect hypothetical postsurgical costs, and doing so did not change which strategy was deemed cost-effective. Importantly, our fundamental conclusions did not change with multiple sensitivity analyses among feasible variations in the probability of PJI eradication, the utilities of any of the health states, and the cost of culture, NGS, or revision surgery (either aseptic, septic, or salvage), all of which may change in the future with the advent of new technology or different economic conditions. On a more fundamental level, all models are approximations of reality [38]. One of the main approximations, or assumptions, in our model is that culture and NGS data are the only decision point to proceed with surgery. At lower levels of pretest probability where there is ambiguity and conflicting results in other data points (such as the C-reactive protein, erythrocyte sedimentation rate, and serum white blood cell count), this may be a reasonable assumption.
However, at higher pretest probabilities, our model likely overestimates the value of culture and NGS (in effect, underestimates the sensitivity of the diagnostic test), given that clinicians are likely to act on other data points to proceed with a PJI revision surgery even in the face of culture- or NGS-negative results. Additionally, we assumed that NGS would identify the same organism as culturebased methods. Although there is a published 96.1% concordance rate between culture and NGS [19], this assumption does introduce a level of uncertainty into our model. Moreover, our model was based on the assumption of NGS being used to identify one organism that represented the majority of bacteria present in a sample while ignoring other, less prevalent organisms which often are not identified by culture. Further work will be necessary to elucidate whether these additional organisms represent pathologic entities, contaminants, or the natural microbiome of the joint, but the current clinical uncertainty in how to interpret these additional organisms precludes any useful modeling attempts. Future work should address this clinically relevant issue which will become more important with future use of NGS technology. Additionally, our model does not take into account the potential side effects of broadspectrum antibiotics that are often used in this situation, specifically the risk of Clostridium difficile infection in the setting of a falsepositive result. Future work will be required to more accurately model these clinical situations and in particular culture or NGSnegative results in the context of high pretest probability. A major strength of this study is that we model long-term consequences of an initial diagnosis of PJI under two different strategies of diagnosis.
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
This has not been previously evaluated in the literature, and a Markov model provides a unique opportunity to rigorously analyze this important area of joint arthroplasty. In addition, our model accounts for several different possible health states after PJI. Despite the limitations and assumptions of our model, our results have face validity and are consistent with the general understanding of how NGS should be used in current clinical practice.
[16]
[17]
[18]
Conclusions [19]
The results of our model suggest that the cost-effectiveness of NGS to diagnose PJI depends primarily on the pretest probability of PJI and the performance characteristics (sensitivity and specificity) of the NGS technology. Our results are consistent with the idea that NGS should be reserved for clinical contexts with a high pretest probability of PJI. Further study is required to determine the indications and subgroups for which NGS offers clinical benefit. References [1] Losina E, Walensky RP, Kessler CL, Emrani PS, Reichmann WM, Wright EA, et al. Cost-effectiveness of total knee arthroplasty in the United States: patient risk and hospital volume. Arch Intern Med 2009;169:1113e21. [2] March LM, Cross MJ, Lapsley H, Brnabic A, Tribe KL, Bachmeier C, et al. Outcomes after hip or knee replacement surgery for osteoarthritis. A prospective cohort study comparing patients' quality of life before and after surgery with age-related population norms. Med J Aust 1999;171:235e8. [3] Jones CA, Voaklander DC, Johnston DW, Suarez-Almazor ME. Health related quality of life outcomes after total hip and knee arthroplasties in a community based population. J Rheumatol 2000;27:1745e52. [4] Norman-Taylor FH, Palmer CR, Villar RN. Quality-of-life improvement compared after hip and knee replacement. J Bone Joint Surg Br 1996;78:74e7. [5] Kurtz S, Mowat F, Ong K, Chan N, Lau E, Halpern M. Prevalence of primary and revision total hip and knee arthroplasty in the United States from 1990 through 2002. J Bone Joint Surg Am 2005;87:1487e97. [6] Kurtz S, Ong K, Lau E, Mowat F, Halpern M. Projections of primary and revision hip and knee arthroplasty in the United States from 2005 to 2030. J Bone Joint Surg Am 2007;89:780e5. [7] Kurtz SM, Ong KL, Lau E, Bozic KJ, Berry D, Parvizi J. Prosthetic joint infection risk after TKA in the Medicare population. Clin Orthop Relat Res 2010;468:52e6. [8] Kurtz SM, Lau E, Schmier J, Ong KL, Zhao K, Parvizi J. Infection burden for hip and knee arthroplasty in the United States. J Arthroplasty 2008;23:984e91. [9] Della Valle C, Parvizi J, Bauer TW, DiCesare PE, Evans RP, Segreti J, et al. American Academy of Orthopaedic Surgeons clinical practice guideline on: the diagnosis of periprosthetic joint infections of the hip and knee. J Bone Joint Surg Am 2011;93:1355e7. [10] Parvizi J, Zmistowski B, Berbari EF, Bauer TW, Springer BD, Della Valle CJ, et al. New definition for periprosthetic joint infection: from the workgroup of the musculoskeletal infection society. Clin Orthop Relat Res 2011;469:2992e4. [11] Della Valle C, Parvizi J, Bauer TW, DiCesare PE, Evans RP, Segreti J, et al. Diagnosis of periprosthetic joint infections of the hip and knee. J Am Acad Orthop Surg 2010;18:760e70. [12] Parvizi J, Ghanem E, Menashe S, Barrack RL, Bauer TW. Periprosthetic infection: what are the diagnostic challenges? J Bone Joint Surg Am 2006;88(Suppl 4): 138e47. [13] Trampuz A, Piper KE, Jacobson MJ, Hanssen AD, Unni KK, Osmon DR, et al. Sonication of removed hip and knee prostheses for diagnosis of infection. N Engl J Med 2007;357:654e63. [14] Gallo J, Kolar M, Dendis M, Loveckova Y, Sauer P, Zapletalova J, et al. Culture and PCR analysis of joint fluid in the diagnosis of prosthetic joint infection. New Microbiol 2008;31:97e104. [15] Gomez E, Cazanave C, Cunningham SA, Greenwood-Quaintance KE, Steckelberg JM, Uhl JR, et al. Prosthetic joint infection diagnosis using
[20]
[21] [22]
[23] [24]
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34] [35] [36]
[37]
[38]
1341
broad-range PCR of biofilms dislodged from knee and hip arthroplasty surfaces using sonication. J Clin Microbiol 2012;50:3501e8. Spangehl MJ, Masri BA, O'connell JX, Duncan CP. Prospective analysis of preoperative and intraoperative investigations for the diagnosis of infection at the sites of two hundred and two revision total hip arthroplasties. J Bone Joint Surg Am 1999;81:672e83. Huang R, Hu C-C, Adeli B, Mortazavi J, Parvizi J. Culture-negative periprosthetic joint infection does not preclude infection control. Clin Orthop Relat Res 2012;470:2717e23. Malekzadeh D, Osmon DR, Lahr BD, Hanssen AD, Berbari EF. Prior use of antimicrobial therapy is a risk factor for culture-negative prosthetic joint infection. Clin Orthop Relat Res 2010;468:2039e45. Shohat N, Tarabichi M, Goswami K, Parvizi J. Can next generation sequencing play a role in detecting pathogens in synovial fluid? Bone Joint J 2018;100: 127e33. Tarabichi M, Shohat N, Goswami K, Alvand A, Silibovsky R, Belden K, et al. Diagnosis of periprosthetic joint infection: the potential of next-generation sequencing. J Bone Joint Surg Am 2018;100:147e54. Whitley R. The new age of molecular diagnostics for microbial agents. N Engl J Med 2008;358:988e9. Bozic KJ, Kurtz SM, Lau E, Ong K, Chiu V, Vail TP, et al. The epidemiology of revision total knee arthroplasty in the United States. Clin Orthop Relat Res 2010;468:45e51. US life tables. Hyattsville, MD: Centers for Disease Control and Prevention; 2017. Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG. Cost-effectiveness in health and medicine. Oxford, United Kingdom: Oxford University Press; 2016. Nwachukwu BU, Bozic KJ. Updating cost effectiveness analyses in orthopedic surgery: resilience of the $50,000 per QALY threshold. J Arthroplasty 2015;30: 1118e20. Sayeed Z, Anoushiravani A, El-Othmani M, Chambers M, Mihalko W, Jiranek W, et al. Two-stage revision total knee arthroplasty in the setting of periprosthetic knee infection. Instr Course Lect 2017;66:249e62. Wu CH, Gray CF, Lee G-C. Arthrodesis should be strongly considered after failed two-stage reimplantation TKA. Clin Orthop Relat Res 2014;472: 3295e304. Slover J, Espehaug B, Havelin LI, Engesaeter LB, Furnes O, Tomek I, et al. Costeffectiveness of unicompartmental and total knee arthroplasty in elderly lowdemand patients: a Markov decision analysis. J Bone Joint Surg Am 2006;88: 2348e55. Zmistowski B, Karam JA, Durinka JB, Casper DS, Parvizi J. Periprosthetic joint infection increases the risk of one-year mortality. J Bone Joint Surg Am 2013;95:2177e84. Parvizi J, Pawasarat IM, Azzam KA, Joshi A, Hansen EN, Bozic KJ. Periprosthetic joint infection: the economic impact of methicillin-resistant infections. J Arthroplasty 2010;25(6 Suppl):103e7. Jenkins P, Clement N, Hamilton D, Gaston P, Patton J, Howie C. Predicting the cost-effectiveness of total hip and knee replacement: a health economic analysis. Bone Joint J 2013;95:115e21. Ramkumar DB, Ramkumar N, Tapp SJ, Moschetti WE. Pharmacologic hemostatic agents in total joint arthroplastyda cost-effectiveness analysis. J Arthroplasty 2018;33:2092e9. Jacobson J, Schweitzer S, Kowalski C. Chemoprophylaxis of prosthetic joint patients during dental treatment: a decision-utility analysis. Oral Surg Oral Med Oral Pathol 1991;72:167e77. Calculator I. Bureau of Labor Statistics. 2010. Washington, DC, USA. http:// www.bls.gov/cpi/. [Accessed 22 May 2018]. Kurtz SM, Lau E, Watson H, Schmier JK, Parvizi J. Economic burden of periprosthetic joint infection in the United States. J Arthroplasty 2012;27:61e5. Renz N, Yermak K, Perka C, Trampuz A. Alpha defensin lateral flow test for diagnosis of periprosthetic joint infection: not a screening but a confirmatory test. J Bone Joint Surg Am 2018;100:742e50. Okroj KT, Calkins TE, Kayupov E, Kheir MM, Bingham JS, Beauchamp CP, et al. The alpha-defensin test for diagnosing periprosthetic joint infection in the setting of an adverse local tissue reaction secondary to a failed metal-onmetal bearing or corrosion at the head-neck junction. J Arthroplasty 2018;33:1896e8. Briggs A, Sculpher M. An introduction to Markov modelling for economic evaluation. Pharmacoeconomics 1998;13:397e409.
Contents lists available at ScienceDirect
The Journal of Arthroplasty journal homepage: www.arthroplastyjournal.org
Health Policy & Economics
Next-Generation Sequencing vs Culture-Based Methods for Diagnosing Periprosthetic Joint Infection After Total Knee Arthroplasty: A Cost-Effectiveness Analysis Michael T. Torchia, MD, MS a, *, Daniel C. Austin, MD, MS a, Samuel T. Kunkel, MD, MS a, Kevin W. Dwyer, MD, MS a, b, Wayne E. Moschetti, MD, MS a, b a b
Department of Orthopaedics, Dartmouth-Hitchcock Medical Center, Lebanon, NH Department of Orthopaedics, Geisel School of Medicine, Dartmouth College, Lebanon, NH
a r t i c l e i n f o
a b s t r a c t
Article history: Received 16 January 2019 Received in revised form 17 February 2019 Accepted 8 March 2019 Available online 19 March 2019
Background: Periprosthetic joint infection (PJI) after total knee arthroplasty is challenging to diagnose. Compared with culture-based techniques, next-generation sequencing (NGS) is more sensitive for identifying organisms but is also less specific and more expensive. To date, there has been no study comparing the cost-effectiveness of these two methods to diagnose PJI after total knee arthroplasty. Methods: A Markov, state-transition model projecting lifetime costs and quality-adjusted life years (QALYs) was constructed to determine the cost-effectiveness from a societal perspective. The primary outcome was incremental cost-effectiveness ratio, with a willingness-to-pay threshold of $100,000/QALY. Sensitivity analyses were performed to evaluate parameter assumptions. Results: At our base case values, culture was not determined to be cost-effective compared to NGS, with an incremental cost-effectiveness ratio of $422,784 per QALY. One-way sensitivity analyses found NGS to be the cost-effective choice above a pretest probability of 45.5% for PJI. In addition, NGS was costeffective if its sensitivity was greater than 70.0% and its specificity greater than 94.1%. Two-way sensitivity analyses revealed that the pretest probability and test performance parameters (sensitivity and specificity) were the largest factors for identifying whether a particular strategy was cost-effective. Conclusion: The results of our model suggest that the cost-effectiveness of NGS to diagnose PJI depends primarily on the pretest probability of PJI and the performance characteristics of the NGS technology. Our results are consistent with the idea that NGS should be reserved for clinical contexts with a high pretest probability of PJI. Further study is required to determine the indications and subgroups for which NGS offers clinical benefit. © 2019 Elsevier Inc. All rights reserved.
Keywords: next-generation sequencing periprosthetic joint infection cost-effectiveness analysis total knee arthroplasty
Total knee arthroplasty (TKA) is one of the most successful surgical interventions in the modern medical era [1e4], and the number of TKA procedures is expected to increase in the coming decades [5,6]. Periprosthetic joint infection (PJI) is a rare but devastating
One or more of the authors of this paper have disclosed potential or pertinent conflicts of interest, which may include receipt of payment, either direct or indirect, institutional support, or association with an entity in the biomedical field which may be perceived to have potential conflict of interest with this work. For full disclosure statements refer to https://doi.org/10.1016/j.arth.2019.03.029. * Reprint requests: Michael T. Torchia, MD, MS, Department of Orthopaedics, 1 Medical Center Drive, Lebanon, NH 03756-0001. https://doi.org/10.1016/j.arth.2019.03.029 0883-5403/© 2019 Elsevier Inc. All rights reserved.
complication of TKA, and the numbers of PJIs are projected to increase in concert with the increased number of total knees being performed [7,8]. Despite the existence of clinical guidelines and specialty society definitions for PJI [9,10], diagnosis remains challenging [9,11e13]. Indeed, current literature shows that when using traditional culture-based techniques, a causative organism is not found in upwards of 20% of cases [14e18]. Molecular techniques, which use polymerase chain reaction to amplify and identify DNA in a sample, have been proposed as a solution to overcome several prior challenges of diagnosing PJI [19,20]. Next-generation sequencing (NGS) for diagnosing PJI permits the sequencing of all DNA present in a given sample [21]. Despite its
1334
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
increased sensitivity for identifying organisms compared to culture [20], tradeoffs exist in its clinical application. First, NGS is more expensive than cultures, even when taking into account the multiple cultures recommended for diagnosis of PJI [10]. Second, while NGS is more sensitive in identifying organisms compared to culture, it is also less specific, and therefore, concern exists about the increased rate of false-positive results [19,20]. Finally, NGS is a new technology, and there is still debate about how to interpret its results and in which clinical contexts it should be used. To date, there has been no cost-effectiveness analysis comparing NGS to culture for the diagnosis of periprosthetic infection and, specifically, no analysis that models the long-term consequences of false-positive and false-negative test results. Given the multiple cost and utility tradeoffs involved between culture and NGS testing strategies, the topic is particularly amenable to a cost-effectiveness analysis. The purpose of this study was to compare the costeffectiveness of NGS and culture for the diagnosis of PJI after total hip arthroplasty and TKA.
Model Structure A Markov, state-transition simulation model was created to determine the cost-effectiveness of culture compared to NGS for the diagnosis of knee PJI. The model projects lifetime costs and quality-adjusted life years (QALYs) and considers costs from a societal perspective. Costs were expressed in 2018 US dollars and were obtained from a combination of the published literature, company representatives, and internal hospital costs. Health state utilities and probabilities were obtained from the published literature, expert opinion, and US life tables [23]. Costs and utilities were discounted at 3% per year in accordance with current practices of cost-effectiveness analysis [24], and the willingness-to-pay threshold was set at 100,000/QALY [25]. The model was created using commercially available decision analysis software (TreeAge Pro 2017; TreeAge Software, Williamstown, MA). A diagram of the chance nodes before entry into the Markov nodes is shown in Figure 1, and a figure showing the treatment strategies, health states, and clinical events during each cycle is shown in Figure 2.
Methods Model Parameters Patient Population The population of interest for our analysis was patients with a primary TKA who were suspected of having a PJI. The age of the patient in our base case analysis was 65 years, determined from the mean age of revision TKA in the Medicare population [22], with a pretest probability of PJI of 50% to model clinical equipoise in the diagnosis of PJI.
Several assumptions were made regarding the model design. First, we assumed that an organism identified by NGS would be the same as that identified by culture based on the 96.1% concordance rate between NGS and culture reported by Shohat et al [19]. Second, we assumed that the performance characteristics (ie, sensitivity and specificity) of NGS were based on the situation in which it identifies a single organism representing the majority of bacteria
Fig. 1. Chance nodes modeling pretest probability, sensitivity, and specificity of each test before entry into Markov nodes. NGS, next-generation sequencing; PJI, periprosthetic joint infection; TJA, total joint arthroplasty.
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
1335
Fig. 2. Diagram showing treatment strategies, health states, and clinical events in the model. Patients enter the model with a primary TJA and concern for infection and undergo either culture or NGS to test for PJI. Test results are true positives, true negatives, false positives, or false negatives (see Fig. 1). Patients who have true-positive, false-positive, or false-negative results of testing undergo further revision surgery. Patients with true-negative test results do not undergo any further surgery and stay in a “post-TJA” state with a baseline annual risk of death. Patients who undergo further revision surgery either clear their infection, do not clear their infection, or die, with different probabilities depending on whether they started with a true-positive, false-positive, or false-negative test result. Patients can only undergo 3 failed revision surgeries before they are placed in the “uncured infection” state.
Table 1 Probabilities, Utilities, and Costs Used in the Model.
Probabilities Sensitivity of NGS Specificity of NGS Sensitivity of culture Specificity of culture Probability of clearing infection after revision Probability of perioperative death Probability of all-cause mortality Probability of death with continued infection Baseline probability of PJI in noninfected joint Utilities Utility of postprimary TJA Utility of successful revision arthroplasty Utility after second revision Utility of uncured infection Interval disutility of revision surgery Interval disutility of salvage operation Costs (in 2018 US dollars) Cost of NGS Cost of cultures (2) Cost of revision for PJI Cost of aseptic revision Cost of salvage procedure Cost yearly after TJA PJI, periprosthetic joint infection; TJA, total joint arthroplasty.
Base Case Value
Source
0.714 0.946 0.607 0.973 0.791 0.011 Variable 0.106 0.0092
Tarabichi et al [20] Tarabichi et al [20] Tarabichi et al [20] Tarabichi et al [20] Wu et al [27] Slover et al [28] 2014 US Life Tables [23] Zmistowski et al [29] Kurtz et al [8]
0.68 0.532 0.452 0.372 0.1 0.2 $250 $100 $31,158 $20,457 $15,154 $148
Slover et al [28] Slover et al [28] Expert opinion based on values from Slover et al [28] Expert opinion based on values from Slover et al [28] Assumption (expert opinion) Assumption (expert opinion) Company representative Hospital internal data Parvizi et al [30] Hospital internal data Hospital internal data Losina et al [1]
1336
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
Fig. 3. One-way sensitivity analysis evaluating the effect of the pretest probability of PJI on the cost-effectiveness of culture and NGS for the diagnosis of PJI. WTP, willingness to pay.
present in a sample as previously reported by Tarabichi et al [20], and not the situation where it detects any bacteria present. Third, we assumed that culture and NGS data would be the only decision point to proceed with revision surgery and that the type of revision surgery would be predicated upon culture and NGS results. Fourth, we assumed that patients with PJI underwent a 2-stage revision arthroplasty as this is currently the standard of care in the United States [26]. We did not explicitly model the risks and complications associated with prolonged intravenous antibiotic therapy. Finally, we modeled an “uncleared infection” state based on utilities combining knee fusion, above-knee amputation, and chronic suppression, given the diversity of choices available for this clinical situation. We performed extensive sensitivity analyses to test this assumption at all ranges of clinically relevant utilities and costs for this health state. Probabilities Transition probabilities used in our study were obtained from the literature and are presented in Table 1. We used differential base-rate annual mortality based on the most recent US life tables (2014) [23]. We assumed the base-rate annual mortality in patients who had cleared their infection to be equal to the base-rate annual mortality of age-matched men and women. The 1-year mortality of patients with an uncleared infection was 10.6% [29]. The perioperative mortality for a revision total joint arthroplasty was set at 1.1% based on Medicare data [28]. We assumed the probability of perioperative mortality from a second revision or salvage operation
would be equal to that of an initial revision arthroplasty. The values for sensitivity and specificity of both NGS and culture were obtained from the published literature [20]. The baseline probability of a PJI was determined to be 0.92% based on a study utilizing the Nationwide Inpatient Sample [8]. The model assumed that all patients with a PJI would undergo a two-stage revision arthroplasty, and the probability of clearing an infection after a two-stage reimplantation was 79.1% based on a 2014 systematic review [27]. Utilities Health state utilities were obtained from the published literature. Specifically, the utility of a postprimary TKA and the utility of the postoperative state after the first TKA revision were taken from a prior cost-effectiveness analysis of patients undergoing TKA [28]. The utilities for a continued infection after a second revision and the utility for an uncleared infection, which in our model was a combination of knee fusion, above-knee amputation, and chronic suppression, were not found in the literature. Thus, a consensus based on the expert opinion of the two senior-most authors of the manuscript was reached, and the utility for a continued infection after a second revision was assumed to be a 15% reduction from a successful revision, and the utility of an uncleared infection was assumed to be a 30% reduction from a successful revision. The utility for the "dead" state was set at zero. To account for the recovery period after surgery, an interval disutility was applied after surgical procedures. The disutility for revision surgery was valued at 0.1 as previously described in the total joint arthroplasty cost-
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
1337
Fig. 4. One-way sensitivity analysis evaluating the cost of NGS on the cost-effectiveness of culture and NGS for the diagnosis of PJI.
effectiveness literature [28], and the interval disutility for a salvage procedure was assumed to be double that of the revision surgery ( 0.2), given the increased complexity of salvage operations and the potential for limb shortening and presumed difficulty ambulating. All health state utilities and tolls are shown in Table 1. We found the utility values used in our analysis to be consistent with previously published values [27,31e33], but given the degree of uncertainty inherent in estimating utilities when reviewing studies of variable quality, multiple sensitivity analyses were performed. Costs Costs were derived from the published literature and internal hospital data and converted to 2018 US dollars using the consumer price index calculator from the United State Department of Labor [34]. Costs are shown in Table 1. No indirect cost considerations related to loss of employment from the effects of surgery were considered as we assumed only a small proportion of patients in the age range used in the model would still remain employed. The cost of revision for PJI was taken as the average cost of a revision for a TKA and total hip arthroplasty from a published analysis of costs using a nationally representative sample [35]. Costs for aseptic revisions, salvage procedures, and cultures were obtained from internal hospital data. The cost of a salvage procedure was assumed to be the average cost of a knee fusion, an above-above knee amputation, and chronic suppression with antibiotics. Consistent with the musculoskeletal infection society (MSIS) guidelines for the diagnosis of PJI [10], we assumed the cost of two cultures when
calculating culture costs for our base case scenario but varied the cost of cultures in multiple sensitivity analyses as typically more than 2 cultures are taken in the setting of revision arthroplasty for PJI. Finally, costs of NGS were obtained from a MicroGen sales representative (MicroGen Diagnostics, Lubbock, TX). Analysis The primary outcome of the study was the incremental costeffectiveness ratio, which is defined as the ratio of the difference in cost divided by the difference in effectiveness, as measured by QALYs, of the two strategies. The cycle length of the model was 1 year, and the model was run for a total of 30 cycles to simulate a clinically relevant time period for patients. Variation among model parameters was assessed with 1-way, 2-way, and 3-way deterministic sensitivity analyses. Probability and utility parameters were varied across all possible values, and cost values were varied up to a point exceeding the highest estimate cost for each parameter. Results Base Case Results Using our base case values, the total cost for the culture strategy was $28,828 with total accrued QALYs of 9.444, whereas the total cost for the NGS strategy was $26,399 with total accrued QALYs of 9.439. Based on these values, culture had an incremental cost-
1338
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
Fig. 5. Two-way sensitivity analysis evaluating pretest probability of PJI and the cost of NGS on the cost-effectiveness of culture and NGS for the diagnosis of PJI.
effectiveness ratio of $422,784 compared to NGS when used to diagnose PJI. Using the a priori willingness-to-pay threshold of $100,000, culture was therefore not found to be cost-effective when compared to NGS.
Sensitivity Analysis One-way sensitivity analysis showed that NGS was cost-effective above a pretest probability for PJI of 45.5% (Fig. 3). Additional 1-way sensitivity analyses showed that NGS was cost-effective if its sensitivity was greater than 70.0% and its specificity greater than 94.1%. Using NGS was also found to be cost-effective until its price reached $3915 (Fig. 4), if the probability of eradication of infection from revision arthroplasty was greater than 42.9%, if the utility of continued infection was less than 0.77, and if the utility of being infection free was above 0.43. Conversely, at base case values, culture was deemed to be cost-effective if the utility of continued infection was higher than 0.77 and if the utility after a second revision was higher than 0.98. At base case values, NGS was found to be always more cost-effective than culture at all costs of culture (analysis stopped at a culture cost of $5000), all costs of aseptic revision arthroplasty, all costs of septic revision arthroplasty, and salvage procedures (analysis stopped at costs for both procedures of $200,000), and at all utilities of the “well post-op” state. Two-way sensitivity analyses demonstrated that the pretest probability of PJI and the test performance parameters (sensitivity and specificity) of both culture and NGS were the prime drivers
behind whether a particular strategy was deemed cost-effective. A 2-way sensitivity analysis evaluating the pretest probability of PJI and the cost of NGS showed that for a pretest probability above 55%, NGS was cost-effective, regardless of its cost (Fig. 5); conversely, for pretest probabilities less than 45%, culture was cost-effective, regardless of its cost. This relationship was also observed in a two-way analysis of pretest probability and the cost of PJI revision: above a pretest probability of 50%, NGS was cost-effective, regardless of its cost. Furthermore, 2-way sensitivity analyses evaluating the specificity and sensitivity of NGS revealed that NGS is cost-effective within a relatively narrow range of specificity and sensitivity (Fig. 6), and this range corresponds closely to its published performance parameters (sensitivity of 71.4%, specificity of 94.6%) [20]. In addition, the same dynamic is observed in a 2-way sensitivity analysis of the sensitivity and specificity of culture: culture is cost-effective within a relatively narrow range of performance parameters that correspond closely to its previously published values (sensitivity of 60.7%, specificity of 97.3%) [20], and small deviations of these parameters determine which diagnostic strategy is favored (Fig. 7).
Discussion Diagnosis of PJI can be challenging after TKA. Both NGS and culture have multiple cost and utility tradeoffs, making their comparison amenable to a cost-effectiveness analysis. The results of this model demonstrate that based on the current published performance
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
1339
Fig. 6. Two-way sensitivity analysis evaluating sensitivity and specificity of NGS on the cost-effectiveness of culture and NGS for the diagnosis of PJI. Analysis shown is at a base case pretest probability for PJI of 50%.
parameters of NGS and culture, NGS is a cost-effective method to diagnose PJI, provided there is a sufficiently high pretest probability of infection (48.3%). Moreover, the pretest probability of PJI and the performance parameters of both culture and NGS are the primary drivers as to whether culture or NGS is deemed cost-effective. Varying the costs of NGS, culture, and revision surgeries, as well as the utilities of the various health states within clinically reasonable parameters did not change the cost-effectiveness of NGS, provided the pretest probability was more than 48.3%. The results of our study should be interpreted carefully. First, this study does not aim to determine whether NGS should be used in current clinical practice. As a new technology used in the context of PJI, further basic science and clinical studies are required to determine how its results should be interpreted and whether its performance characteristics vary in certain subgroups. However, if NGS will be used in the same way as culture to diagnose PJI in the future, our model does permit greater insights into the clinical contexts when NGS would be most cost-effective to use. From that perspective, our study implies that NGS is best suited to clinical scenarios where the pretest probability of PJI is high. This is consistent with current literature on the use of NGS to diagnose PJI. Indeed, the most recent paper on the application of NGS to the diagnosis of PJI stated, “…the pre-test probability determined by the clinical picture and other laboratory investigations should be closely examined when interpreting the results of next-generation sequencing.” [20] Moreover, the results of our model supporting the idea that NGS should be used as a confirmatory tool, rather than
a screening tool, is remarkably consistent with the conclusions reached by other authors regarding a different molecular test to diagnose PJI with similar performance characteristics as NGS [36]. Our model’s consistency with previously published literature on the topic lends a degree of face validity to the model. The finding in our model that NGS and culture are cost-effective within relatively narrow performance parameters (ie, sensitivity and specificity values), which correspond closely to their published parameters, has several implications. First, relatively small increases or decreases in the performance of either diagnostic test can lead to changes in which diagnostic strategy is favored in terms of cost-effectiveness. Second, our findings imply that further work should be done to determine particular clinical contexts where one test is clearly superior or inferior to the other. For example, alphadefensin, a highly sensitive and specific molecular test used to diagnose PJI, has been shown to have inferior performance characteristics in cases of adverse local tissue reaction secondary to metal-on-metal wear or head-neck junction corrosion [37]. Similarly, if there are other contexts where NGS or culture has clearly inferior performance characteristics, then an alternative diagnostic strategy may be favored given the narrow performance characteristics required for either culture or NGS to be cost-effective. The use of NGS and culture in situations such as severe metallosis in metalon-metal bearing surfaces, catastrophic polyethylene wear, implanted antibiotic spacers, and trunnionosis from junctional corrosion merit further study. Finally, given how new NGS technology is to the field of orthopedic surgery and the fact that its
1340
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
Fig. 7. Two-way sensitivity analysis evaluating sensitivity and specificity of culture on the cost-effectiveness of culture and NGS for the diagnosis of PJI. Analysis shown is at a base case pretest probability for PJI of 50%.
published performance characteristics were determined at one institution, our finding implies the need for future work to validate the performance characteristics of NGS in other settings. We acknowledge several limitations. First, model inputs were derived primarily from the published literature, and there was a degree of variation in the published values of different health states and costs in the literature. We are reassured, however, that extensive sensitivity analyses demonstrated our model to be robust to all clinically reasonable utility and cost values. It is also reassuring that while we did not explicitly model the cost of rehabilitation or home health which a proportion of patients inevitably require after surgery, the costs of surgery could be increased in sensitivity analysis to reflect hypothetical postsurgical costs, and doing so did not change which strategy was deemed cost-effective. Importantly, our fundamental conclusions did not change with multiple sensitivity analyses among feasible variations in the probability of PJI eradication, the utilities of any of the health states, and the cost of culture, NGS, or revision surgery (either aseptic, septic, or salvage), all of which may change in the future with the advent of new technology or different economic conditions. On a more fundamental level, all models are approximations of reality [38]. One of the main approximations, or assumptions, in our model is that culture and NGS data are the only decision point to proceed with surgery. At lower levels of pretest probability where there is ambiguity and conflicting results in other data points (such as the C-reactive protein, erythrocyte sedimentation rate, and serum white blood cell count), this may be a reasonable assumption.
However, at higher pretest probabilities, our model likely overestimates the value of culture and NGS (in effect, underestimates the sensitivity of the diagnostic test), given that clinicians are likely to act on other data points to proceed with a PJI revision surgery even in the face of culture- or NGS-negative results. Additionally, we assumed that NGS would identify the same organism as culturebased methods. Although there is a published 96.1% concordance rate between culture and NGS [19], this assumption does introduce a level of uncertainty into our model. Moreover, our model was based on the assumption of NGS being used to identify one organism that represented the majority of bacteria present in a sample while ignoring other, less prevalent organisms which often are not identified by culture. Further work will be necessary to elucidate whether these additional organisms represent pathologic entities, contaminants, or the natural microbiome of the joint, but the current clinical uncertainty in how to interpret these additional organisms precludes any useful modeling attempts. Future work should address this clinically relevant issue which will become more important with future use of NGS technology. Additionally, our model does not take into account the potential side effects of broadspectrum antibiotics that are often used in this situation, specifically the risk of Clostridium difficile infection in the setting of a falsepositive result. Future work will be required to more accurately model these clinical situations and in particular culture or NGSnegative results in the context of high pretest probability. A major strength of this study is that we model long-term consequences of an initial diagnosis of PJI under two different strategies of diagnosis.
M.T. Torchia et al. / The Journal of Arthroplasty 34 (2019) 1333e1341
This has not been previously evaluated in the literature, and a Markov model provides a unique opportunity to rigorously analyze this important area of joint arthroplasty. In addition, our model accounts for several different possible health states after PJI. Despite the limitations and assumptions of our model, our results have face validity and are consistent with the general understanding of how NGS should be used in current clinical practice.
[16]
[17]
[18]
Conclusions [19]
The results of our model suggest that the cost-effectiveness of NGS to diagnose PJI depends primarily on the pretest probability of PJI and the performance characteristics (sensitivity and specificity) of the NGS technology. Our results are consistent with the idea that NGS should be reserved for clinical contexts with a high pretest probability of PJI. Further study is required to determine the indications and subgroups for which NGS offers clinical benefit. References [1] Losina E, Walensky RP, Kessler CL, Emrani PS, Reichmann WM, Wright EA, et al. Cost-effectiveness of total knee arthroplasty in the United States: patient risk and hospital volume. Arch Intern Med 2009;169:1113e21. [2] March LM, Cross MJ, Lapsley H, Brnabic A, Tribe KL, Bachmeier C, et al. Outcomes after hip or knee replacement surgery for osteoarthritis. A prospective cohort study comparing patients' quality of life before and after surgery with age-related population norms. Med J Aust 1999;171:235e8. [3] Jones CA, Voaklander DC, Johnston DW, Suarez-Almazor ME. Health related quality of life outcomes after total hip and knee arthroplasties in a community based population. J Rheumatol 2000;27:1745e52. [4] Norman-Taylor FH, Palmer CR, Villar RN. Quality-of-life improvement compared after hip and knee replacement. J Bone Joint Surg Br 1996;78:74e7. [5] Kurtz S, Mowat F, Ong K, Chan N, Lau E, Halpern M. Prevalence of primary and revision total hip and knee arthroplasty in the United States from 1990 through 2002. J Bone Joint Surg Am 2005;87:1487e97. [6] Kurtz S, Ong K, Lau E, Mowat F, Halpern M. Projections of primary and revision hip and knee arthroplasty in the United States from 2005 to 2030. J Bone Joint Surg Am 2007;89:780e5. [7] Kurtz SM, Ong KL, Lau E, Bozic KJ, Berry D, Parvizi J. Prosthetic joint infection risk after TKA in the Medicare population. Clin Orthop Relat Res 2010;468:52e6. [8] Kurtz SM, Lau E, Schmier J, Ong KL, Zhao K, Parvizi J. Infection burden for hip and knee arthroplasty in the United States. J Arthroplasty 2008;23:984e91. [9] Della Valle C, Parvizi J, Bauer TW, DiCesare PE, Evans RP, Segreti J, et al. American Academy of Orthopaedic Surgeons clinical practice guideline on: the diagnosis of periprosthetic joint infections of the hip and knee. J Bone Joint Surg Am 2011;93:1355e7. [10] Parvizi J, Zmistowski B, Berbari EF, Bauer TW, Springer BD, Della Valle CJ, et al. New definition for periprosthetic joint infection: from the workgroup of the musculoskeletal infection society. Clin Orthop Relat Res 2011;469:2992e4. [11] Della Valle C, Parvizi J, Bauer TW, DiCesare PE, Evans RP, Segreti J, et al. Diagnosis of periprosthetic joint infections of the hip and knee. J Am Acad Orthop Surg 2010;18:760e70. [12] Parvizi J, Ghanem E, Menashe S, Barrack RL, Bauer TW. Periprosthetic infection: what are the diagnostic challenges? J Bone Joint Surg Am 2006;88(Suppl 4): 138e47. [13] Trampuz A, Piper KE, Jacobson MJ, Hanssen AD, Unni KK, Osmon DR, et al. Sonication of removed hip and knee prostheses for diagnosis of infection. N Engl J Med 2007;357:654e63. [14] Gallo J, Kolar M, Dendis M, Loveckova Y, Sauer P, Zapletalova J, et al. Culture and PCR analysis of joint fluid in the diagnosis of prosthetic joint infection. New Microbiol 2008;31:97e104. [15] Gomez E, Cazanave C, Cunningham SA, Greenwood-Quaintance KE, Steckelberg JM, Uhl JR, et al. Prosthetic joint infection diagnosis using
[20]
[21] [22]
[23] [24]
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34] [35] [36]
[37]
[38]
1341
broad-range PCR of biofilms dislodged from knee and hip arthroplasty surfaces using sonication. J Clin Microbiol 2012;50:3501e8. Spangehl MJ, Masri BA, O'connell JX, Duncan CP. Prospective analysis of preoperative and intraoperative investigations for the diagnosis of infection at the sites of two hundred and two revision total hip arthroplasties. J Bone Joint Surg Am 1999;81:672e83. Huang R, Hu C-C, Adeli B, Mortazavi J, Parvizi J. Culture-negative periprosthetic joint infection does not preclude infection control. Clin Orthop Relat Res 2012;470:2717e23. Malekzadeh D, Osmon DR, Lahr BD, Hanssen AD, Berbari EF. Prior use of antimicrobial therapy is a risk factor for culture-negative prosthetic joint infection. Clin Orthop Relat Res 2010;468:2039e45. Shohat N, Tarabichi M, Goswami K, Parvizi J. Can next generation sequencing play a role in detecting pathogens in synovial fluid? Bone Joint J 2018;100: 127e33. Tarabichi M, Shohat N, Goswami K, Alvand A, Silibovsky R, Belden K, et al. Diagnosis of periprosthetic joint infection: the potential of next-generation sequencing. J Bone Joint Surg Am 2018;100:147e54. Whitley R. The new age of molecular diagnostics for microbial agents. N Engl J Med 2008;358:988e9. Bozic KJ, Kurtz SM, Lau E, Ong K, Chiu V, Vail TP, et al. The epidemiology of revision total knee arthroplasty in the United States. Clin Orthop Relat Res 2010;468:45e51. US life tables. Hyattsville, MD: Centers for Disease Control and Prevention; 2017. Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG. Cost-effectiveness in health and medicine. Oxford, United Kingdom: Oxford University Press; 2016. Nwachukwu BU, Bozic KJ. Updating cost effectiveness analyses in orthopedic surgery: resilience of the $50,000 per QALY threshold. J Arthroplasty 2015;30: 1118e20. Sayeed Z, Anoushiravani A, El-Othmani M, Chambers M, Mihalko W, Jiranek W, et al. Two-stage revision total knee arthroplasty in the setting of periprosthetic knee infection. Instr Course Lect 2017;66:249e62. Wu CH, Gray CF, Lee G-C. Arthrodesis should be strongly considered after failed two-stage reimplantation TKA. Clin Orthop Relat Res 2014;472: 3295e304. Slover J, Espehaug B, Havelin LI, Engesaeter LB, Furnes O, Tomek I, et al. Costeffectiveness of unicompartmental and total knee arthroplasty in elderly lowdemand patients: a Markov decision analysis. J Bone Joint Surg Am 2006;88: 2348e55. Zmistowski B, Karam JA, Durinka JB, Casper DS, Parvizi J. Periprosthetic joint infection increases the risk of one-year mortality. J Bone Joint Surg Am 2013;95:2177e84. Parvizi J, Pawasarat IM, Azzam KA, Joshi A, Hansen EN, Bozic KJ. Periprosthetic joint infection: the economic impact of methicillin-resistant infections. J Arthroplasty 2010;25(6 Suppl):103e7. Jenkins P, Clement N, Hamilton D, Gaston P, Patton J, Howie C. Predicting the cost-effectiveness of total hip and knee replacement: a health economic analysis. Bone Joint J 2013;95:115e21. Ramkumar DB, Ramkumar N, Tapp SJ, Moschetti WE. Pharmacologic hemostatic agents in total joint arthroplastyda cost-effectiveness analysis. J Arthroplasty 2018;33:2092e9. Jacobson J, Schweitzer S, Kowalski C. Chemoprophylaxis of prosthetic joint patients during dental treatment: a decision-utility analysis. Oral Surg Oral Med Oral Pathol 1991;72:167e77. Calculator I. Bureau of Labor Statistics. 2010. Washington, DC, USA. http:// www.bls.gov/cpi/. [Accessed 22 May 2018]. Kurtz SM, Lau E, Watson H, Schmier JK, Parvizi J. Economic burden of periprosthetic joint infection in the United States. J Arthroplasty 2012;27:61e5. Renz N, Yermak K, Perka C, Trampuz A. Alpha defensin lateral flow test for diagnosis of periprosthetic joint infection: not a screening but a confirmatory test. J Bone Joint Surg Am 2018;100:742e50. Okroj KT, Calkins TE, Kayupov E, Kheir MM, Bingham JS, Beauchamp CP, et al. The alpha-defensin test for diagnosing periprosthetic joint infection in the setting of an adverse local tissue reaction secondary to a failed metal-onmetal bearing or corrosion at the head-neck junction. J Arthroplasty 2018;33:1896e8. Briggs A, Sculpher M. An introduction to Markov modelling for economic evaluation. Pharmacoeconomics 1998;13:397e409.