- Email: [email protected]
Design of a Method for Prediction of Susceptibility to Antimicrobials
Proceedings of the 7th IFAC Symposium on Modelling and Control in Biomedical Systems Aalborg, Denmark, August 12-14, 2009
Design of a Method for Prediction of Susceptibility to Antimicrobials Steen Andreassen*, Alina Zalounina* Mical Paul**, Leonard Leibovici** *Center for Model-Based Medical Decision Support, Aalborg University, Aalborg Denmark (Tel:+45-9940-8812; e-mail: [email protected] or [email protected] ). **Rabin Medical Centre, Petah Tiqva, Israel (e-mail: [email protected] or [email protected]) Abstract: The results from the in vitro susceptibility tests are usually available only for a limited set of antibiotics. This paper provides a practical method for prediction of a posteriori probability for coverage for all relevant antibiotics. The method combines susceptibility results for a limited set of antibiotics with data on cross-resistance between antibiotics. The Brier distance was used to measure the accuracy of the predicted coverages. Across all pathogen/antibiotic combinations in the derivation database, the Brier distances for a priori coverages was 39%, reduced to 25% for predicted a posteriori coverages, indicating that there is a significant advantage to the method proposed (p<10 -99). Keywords: Bacterial infections; Antibiotic therapy; Susceptibility test; Cross-resistance; A posteriori probability. 1. INTRODUCTION The choice of appropriate antimicrobial treatment depends on many factors, one of the most important being accurate estimates of the in vitro susceptibility (or its negation, called resistance) of the available antibiotics to a given pathogen. At the onset of an infectious episode, the probability of susceptibility, in the following called the coverage, can be estimated from a database of microbial isolates containing the results of susceptibility tests for a range of antimicrobials. Over the course of the infectious episode, the clinical opinion on coverage will change from its a priori value at the beginning of the episode. One of such breaking points for reconsidering opinion on coverage is the stage where susceptibility results become available. However, there are several problems that remain after antibiotic susceptibilities are available: not all antibiotics can be tested; an antibiotic that was not tested might be needed because of: patient allergies, availability, to cover a polymicrobial infection using a single antibiotic, etc. For example, let us assume that Acinetobacter spp. had its susceptibility tested for meropenem (MER), but not for imipenem (IMI). The abbreviation MER will also represent a stochastic variable for coverage obtained by meropenem. Its two states, representing respectively that the pathogen is sensitive (MER=S) or resistant (MER=R) to meropenem, will be called mer and ¬mer. The same terminology is applied to the rest of antimicrobial drugs mentioned in the paper. From the isolate database (Table 1) it can be seen that Acinetobacter spp. is resistant to MER. How does the knowledge about resistance to MER affect the a posteriori probability for coverage by imipenem (IMI): P(imi | ¬mer)?
978-3-902661-49-4/09/$20.00 © 2009 IFAC
Previously (Zalounina et al. 2007), we have described how a coverage database, containing both resistances and crossresistances (the phenomenon of cross-resistance reflects the fact that some antibiotics share the mechanism by which they attack pathogens), can be calculated from the isolate database. From the conditional probabilities given in the coverage database (Table 2) it can be seen that P(imi | ¬mer) = 9 %. This can be compared to the a priori probability for coverage by imipenem P(imi) = 56% (Table 2). Often information on susceptibility becomes available not on a single antimicrobial but simultaneously on a number of antimicrobials. This happens in 10-30% of the patients in whom the pathogen is successfully isolated, and subsequently tested for susceptibility to a set of antimicrobials. For economic and practical reasons, this set is restricted, typically to 8-25 different antimicrobials. For example, assuming that the isolated pathogen Acinetobacter spp. from episode 1 had its susceptibility tested for ofloxacin (OFL), meropenem, and amikacin (AMK) with the results given in Table 1, we could ask for an a posteriori estimate of coverage for imipenem: P(imi | ¬ofl, ¬mer, amk) A tabulation of these conditional probabilities, derived from the isolate database is not a practical solution. In the isolate database used in this paper, the average number of antimicrobials tested for susceptibility is 19. This would not only make the resulting 20 dimensional conditional probability tables very large, but it would also make it difficult to derive reliable estimates for the conditional probabilities because of the limited size of the isolate database. To make things even more unwieldy the 19 antibiotics tested on average is not a fixed set, but a choice out of 31 antibiotics tested on a regular basis, which means that a single large 20 dimensional table is not sufficient. A
313
10.3182/20090812-3-DK-2006.0064
7th IFAC MCBMS (MCBMS'09) Aalborg, Denmark, August 12-14, 2009
Table 2. The coverage database for Acinetobacter spp.
number of tables covering different combinations of tested antimicrobials is required, making this approach impractical.
a)
The purpose of this paper is to develop a practical method for calculation of a posteriori coverages for all relevant antibiotics, given susceptibility results for a restricted set of antibiotics.
Susceptibility test antibiotic Treatment
2. METHODS AND MATERIAL IMI
The methods section will start by defining the isolate and coverage database. Subsequently the problem of calculating a posteriori coverages will be formally defined and some approximate solutions will be proposed.
Episode Pathogen OFL MER AMK 1 Acinetobacter spp. R R S 2 E. coli R S R 3 Proteus spp. S NA S S = susceptible R = resistant NA = the susceptibility is not assessed
N IMI,imi
N IMI, ¬imi
88
0
1
84
74
0
0.96
0.09
N MER,imi N MER, ¬imi N MER,mer N MER, ¬mer
152 P(mer)
N IMI,mer N IMI, ¬mer
P(imi|imi) P(imi|¬imi) P(imi|mer) P(imi|¬mer )
88
70
152
0
P(mer|imi) P(mer|¬imi) P(mer|mer) P(imi|¬mer)
0.57
The isolate database includes 3347 clinically significant blood isolates collected between 2002-2004 at Rabin Medical Center in Israel. The isolate database specifies the name of a pathogen and its in vitro susceptibility to a total of 31 antibiotics. A segment of the database, showing susceptibility to three antibiotics - ofloxacin, meropenem and amikacin, is given in Table 1.
Table 1. The isolate database
P(imi)
MER a posteriori
NIMI 88
NMER
2.1 The isolate and coverage databases
Based on the isolate database the counts of coverage and probabilities of coverage for each pathogen were calculated and stored in the coverage database. A segment of the coverage database for Acinetobacter spp. is shown in Table 2. Table 2a contains the a priori/a posteriori coverages and a priori/a posteriori counts these probabilities were based on. The a posteriori data for the combination imipenem/meropenem (see bold rectangle in Table 2a) are expanded in Table 2b. The first three rows following the header row in Table 2b show a posteriori counts. E.g., among all isolates tested against imipenem, 74 were resistant to meropenem (NIMI, ¬mer = 74). Out of these 74 isolates, 7 were susceptible to imipenem (Nimi, ¬mer=7) and 67 resistant to imipenem (N¬imi, ¬mer=67). Conditional probabilities are shown in the next two rows. E.g., P(imi | ¬mer) = Nimi, ¬mer / NIMI, ¬mer = 7 / 74 = 0.09. The following row demonstrates, that knowledge that Acinetobacter is sensitive to meropenem increases the odds for imipenem=S by odds ratio (ORimi, mer) of 21.5. Resistance to meropenem reduces the odds by an odds ratio of 0.08 (ORimi, ¬mer). The p values chosen to decide which odds ratios are significantly different from 1, were calculated using the Fisher’s exact test and are shown in the last row.
a priori
0.56
MER
IMI
0.92
0.04
1
0
b) MER
IMI
N IMI,mer 84 Nimi,mer 81 N ¬imi,mer 3 P(imi | mer) 0.96 P(¬imi | mer) 0.04 ORimi| mer 21.5
N IMI, ¬mer 74 N imi, ¬mer 7 N ¬imi, ¬mer 67 P(imi | ¬mer) 0.09 P(¬imi | ¬mer) 0.91 ORimi| ¬mer 0.08
p< 10-11
p< 10-11
Table (b) is an expansion of data shown in the bold rectangle in Table (a)
2.2 Calculation of a posteriori coverage given susceptibility results for a set of antibiotics Formally, the problem to be considered is as follows. We assume, that the susceptibility of a given pathogen to a set of T antimicrobials: AT = {A1, … At, … AT} has been tested, and each At has been assigned a value at (susceptible) or ¬at (resistant). The a priori coverage, i.e. probability of susceptibility to an antimicrobial B, which is not a member of the set AT, is P(b). Estimate the a posteriori coverage for B, given AT, P(b|AT). In the introduction it was already concluded, that compiling estimates for P(b|AT) from the isolate database is not a viable solution. A detailed knowledge of the mutations that confer resistance to the given pathogen might make it possible to
314
7th IFAC MCBMS (MCBMS'09) Aalborg, Denmark, August 12-14, 2009
make a causal model of the coverage, but in general such knowledge is not available. Alternatively stochastic models, such as regression or logistic regression or factor analysis, with the At’s as independent variables might be a solution, but suffer from the fact that the set AT varies from isolate to isolate. A step towards a simple approximation can be taken by using Bayes theorem to write: P(b|AT) = P(AT|b) P(b) / P(AT) (1) If we make the “naïve Bayes” assumption of conditional independence within the set AT, given b, then P(AT|b) = Пt P(At|b)
(2)
A Bayes inversion of P(At|b) gives: P(At|b) = P(b|At) P(At) / P(b) Insertion of (2) and (3) in (1) gives:
(3)
As an intermediate solution between taking all significant odds ratios into account and only taking the maximum and minimum odds ratios into account we can explore a MinMax2 strategy. In this strategy the 2 largest significant odds ratios and the 2 smallest significant odds ratios are used in the calculation of Пt ORb|At. 2.4 Assessment of quality of a posteriori probability
P(b|AT) = Пt P(At|b) P(b) / P(AT) = Пt [P(b|At) P(At) / P(b)] P(b) / P(AT)
can be improved by reducing the number of factors in (8), simply by ignoring some of the susceptibility results. This alternative strategy called MinMax, where only the largest factor in ORb|at and the smallest factor in ORb|¬at are used, will be considered. Odds ratios greater than 1 represent reasons for susceptibility and odds ratios smaller than 1 represent reasons for resistance. The rationale for the MinMax strategies is that the underlying genetic reasons for susceptibility are probably independent from the reasons for resistance.
(4)
To quantify the potential improvement in estimates of coverage, the Brier distance (Brier 1950) can be used across all antibiotics for which a susceptibility result is available:
ODb = P(b) / P(¬b) and the aposteriori odds ODb|AT for P(b|AT) as:
(5)
BrierDist =
ODb|AT = P(b|AT) / P(¬b|AT)
(6)
= Пt [P(b|At) / P(b)] Пt P(At) P(b) / P(AT)
We now define the a priori odds ODb for P(b) as:
2 ∑ P(b t | A T-t ) + ∑ (1 - P(b t | A T-t )) 2 A ˛ Re s A t˛ Sus t NBrier
Inserting (4) into (6) gives:
NBrier = Nsus + Nres,
ODb|AT = (Пt [P(b|At) / P(b)] P(b)) / (Пt [P(¬b|At) / P(¬b)] P(¬b)) = (Пt [P(b|At) / P(¬b|At) / ODb] ODb)
(7)
Equation (7) can also be written as: ODb|AT = Пt ORb|At ODb
where Sus is the set of antibiotics to which the isolate is susceptible and Res is the set of antibiotics to which the isolate is resistant; Nsus and Nres are the number of members in the sets Sus and Res, respectively.
(8)
where: ORb|At = P(b|At) / P(¬b|At) / ODb
(11)
3. RESULTS
(9)
3.1 Illustration of the method for a single pathogen Finally the a posteriori probability can be calculated from: P(b|AT) = ODb|AT / (1 + ODb|AT)
(10)
using that P(¬b|AT) = 1- P(b|AT). The proposed method will explore the use of (8) for the calculation of the a posteriori odds for coverage.
A posteriori determinations performed as given by eqs. 8 and 10 will be tested on the isolate database. The testing will be done by using eq. 8 to predict coverage for each of the antibiotics for which a susceptibility results is available, using the rest of the susceptibility results. 3.1.1 Result for a single antibiotic
2.3 Strategies for a posteriori determination Equation (8) was derived using the naïve Bayes assumption of conditional independence. The accuracy of the calculation therefore depends on how realistic the assumption of conditional independence is. The first strategy chosen in this paper is to use all available significant odds ratios ORb|At, assuming complete independence between susceptibility results. In the result section it will be explored if the accuracy
Let us for example use eq. 8 to predict the coverage for imipenem. The result from the susceptibility testing shows that the isolate, Acinetobacter spp., actually is not tested against imipenem (Table 3, column 4). The a priori coverage of imipenem is 56% (Table 3, column 3), and it is therefore a priori far from given that treating Acinetobacter spp. with imipenem is appropriate. To use equation 8 to calculate a posteriori coverage for imipenem, given all the other
315
7th IFAC MCBMS (MCBMS'09) Aalborg, Denmark, August 12-14, 2009
susceptibility results we need ORimi|At for each At ˛ AT , where AT = Sus ¨ Res. The odds ratios ORimi|At are given in Table 3, columns 5 and 6. Only odds ratios which are significantly different from 1 (p ≤ 0.1) are listed – for drugs amikacin, minocycline (MIN) og meropenem.
Apriori coverage %
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
imipenem 56 penbritin 3 piperacillin 11 keflin 2 cefuroxim 4 ceftazidim 17 ceftriaxone 3 gentamycin 25 tobramycin 34 amikacin 30 septrin 14 ofloxacin 15 axtreonam 3 colistin 98 tetracyclin 10 augmentin 5 ciproflocacin 15 minocycline 52 tazocin 14 cefepime 18 meropenem 57 At = susceptibility result NA = not assessed
At
NA R R R R R R R R S R R R S R R R R R R S
ORimi|at
ORimi|¬at
p ≤ 0.1
p ≤ 0.1
3.1.2 Results for all antibiotics The a posteriori “all odds ratios” coverage for imipenem is given in Table 4, along with the a posteriori coverages for other antibiotics (column 5). Table 4 also shows the susceptibility results for 21 antibiotics (including known result for imipenem) (column 4).
Table 3. Susceptibility test results and odds ratios for Acinetobacter spp. being susceptible to imipenem given knowledge about susceptibility to other antibiotics Acinetobac ter spp.
column 4. This would make imipenem an excellent choice for treating this infection.
Table 4. A posteriori coverages Acinetobac ter spp.
BrierDist 0.27 0.27 1 imipenem 56 S 99 2 penbritin 3 R 0 3 piperacillin 11 R 0 4 keflin 2 R 5 cefuroxim 4 R 0 6 ceftazidim 17 R 0 7 ceftriaxone 3 R 0 8 gentamycin 25 R 0 9 tobramycin 34 R 8 10 amikacin 30 S 3 11 septrin 14 R 0 12 ofloxacin 15 R 0 13 axtreonam 3 R 0 14 colistin 98 S 15 tetracyclin 10 R 0 16 augmentin 5 R 0 17 ciproflocacin 15 R 0 18 minocycline 52 R 74 19 tazocin 14 R 0 20 cefepime 18 R 0 21 meropenem 57 S 93 At = susceptibility result NA = not assessed
5.8
0.47
21.5
Forming the product over the sets Sus and Res, respectively, including only values of ORimi|At which are significantly different from 1 for the antibiotic imipenem gives: Пt ORimi|At = ORimi | amk * ORimi | ¬min * ORimi | mer = 5.83 * 0.47 * 21.5 = 58.9
Apriori At All odds MinMax MinMax2 coverage coverage coverage ratios % % % coverage 0,229 93
0,231 99
0 3
0 0
0 3 0 2 31 29 7 0 0
0 1 0 1 23 29 1 0 0
3 0 0 66 9 8 83
1 0 0 74 1 3 93
The Brier distance for the a posteriori coverage can then be compared to the Brier distance for the a priori coverage. In row 2 of Table 4 the Brier distances calculated across all antibiotics tested against Acinetobacter spp. are shown.
Inserting this in eq. 8 gives: ODimi|AT = Пt ORimi | At * ODimi = 58.9 *0.56 / (1-0.56) = 75.0 Eq. 10 then gives: P(imi | amk, ¬min, mer) = ODimi | AT / (1 + ODimi | AT) = 98.7% which is our estimate of the a posteriori coverage of imipenem, given the susceptibility results from Table 3,
The Brier distance for the a posteriori coverage is 0,27, which is the same as the Brier distance for the apriori coverage, indicating that for this isolate there is no net advantage of the amendment shown. For two antibiotics, amikacin and minocycline, the a posteriori coverages (3% and 74%,
316
7th IFAC MCBMS (MCBMS'09) Aalborg, Denmark, August 12-14, 2009
the
actual
A posteriori coverages resulting from the MinMax strategy, where only the largest factor in ORb|at and the smallest factor in ORb|¬at are used, are shown in Table 4 column 6. It can be seen that this MinMax strategy reduces the error and overall there is a reduction in the Brier distance to 0.229. The resulting a posteriori coverages from the MinMax2 strategy, where the 2 largest significant odds ratios and the 2 smallest significant odds ratios are used in the calculation of Пt ORb|At, are shown in Table 4 column 7 with a resulting Brier distance of 0.231, slightly higher than for the MinMax strategy.
Fig. 2 shows that the MinMax strategy is better than the other two strategies, although the difference between the MinMax and the MinMax2 strategies is quite small. All strategies are however considerably better than just keeping the a priori probabilities of coverage. For the a priori coverages, the Brier distance is 0.39. It is also apparent that a p value of 0.1 is not necessarily optimal. The Brier distance for the MinMax strategy decreases with the p value, levelling off around 0.254 for a p value of 0.1 to 0.15 and reaching 0.252 for a p value of 0.5. Comparison of strategies 0,31
Brier distance
respectively) deviate considerably from susceptibility results (S and R, respectively).
3.2. Comparison of strategies To determine if it applies in general that the MinMax strategy is better than the MinMax2 strategy and the all-odds-ratios strategy, these strategies were applied to all 3347 isolates in the database. Results will depend on the p value chosen to decide which odds ratios are significantly different from 1. Therefore the a posteriori determinations were carried out with different p values, ranging from 0.02 to 0.5. In the database a total of 31 different antibiotics were used to test the susceptibility of the isolated pathogens. Each isolate was tested with a subset of the 31 antibiotics. On average each isolate was tested with 19 antibiotics or 61% of the antibiotics, giving a total of 63590 determinations of susceptibility in the database. The results were obtained by deleting a single result of a susceptibility test for a single isolated pathogen, and then attempting to estimate the a posteriori coverage P(bt|AT\t) from the remaining susceptibility test results for that isolated pathogen. This was repeated for all 63590 susceptibility results. Fig. 1 shows that for most of those 63590 susceptibility determinations, a posteriori coverages can be estimated from the other determinations. With a p value of 0.02 for significance of odds ratios, 74.7% of those can be estimated, and with a p value of 0.5, 91.6% can be estimated.
0,30 All odds ratios MinMax odds ratios MinMax2 odds ratios
0,29 0,28 0,27 0,26 0,25 0,00
0,10
0,20
0,30
0,40
0,50
p value
Fig. 2. Brier distances for the 3 different strategies where all 63590 susceptibility determinations have been estimated, either by calculating a posteriori coverage when possible or by keeping the a priori coverage. To achieve a better understanding of the appropriate choice of p value, the Brier distance for the MinMax strategy has been plotted again vs. the p value in Fig. 3. In addition the Brier distance for the subset of susceptibilities where a posteriori determinations were possible was plotted. The Brier distance for the a priori coverages is the remaining subset of susceptibility determinations. The results show that the Brier distances for the a posteriori coverages is fairly constant about 0.27, irrespective of the p value. The Brier distance of the remaining, a priori susceptibilities is substantially smaller. This is mainly because the remaining, a priori susceptibility determinations tend to have extreme values of a priori coverage, either close to 0 or close to 1. Brier distances for a posteriori and a priori coverages
Percent amendments
0,30
Brier distance
100% 90% 80%
0,20
0,10
70% 0,00
0,10
0,20
0,30
0,40
0,00 0,00
0,50
p value
Brier for all Brier for a posteriori coverages Brier for a priori coverages
0,10
0,20
0,30
0,40
0,50
p value
Fig. 1. Percent of susceptibilities which can be determined a posteriori, based on the other determinations.
Fig. 3. Brier distances for a posteriori coverages, for the remaining a priori coverages and for the union of those two sets.
317
7th IFAC MCBMS (MCBMS'09) Aalborg, Denmark, August 12-14, 2009
The final choice of p value is a compromise. A p value higher than 0.1 reduces the Brier distance slightly, but introduces additional a posteriori determinations, which are based on odds ratios with dubious significance. A choice of p value of 0.1 therefore seems a reasonable compromise. Fig. 4 shows the number of new a posteriori determinations in percent of the number of susceptibility determinations. For a p value of 0.1, 7.4% new a posteriori determinations could be made, corresponding to about 1.4 new a posteriori determinations on average per isolate. This is a relatively low number, but should be seen in the light that with an average of 19 susceptibility determinations per pathogen, there are not too many relevant antibiotics left without a determination of susceptibility.
15%
To conclude, the method described allows estimation of a posteriori coverages for all relevant antibiotics, given susceptibility test results for a restricted set of antibiotics. This technique provides a significant improvement in estimates of coverage, compared to a priori estimates, with the MinMax strategy being slightly better than the MinMax2 strategy.
10% 5%
0,10
0,20
0,30
One of the drawbacks of the model is in its simplified approach to understanding and imitation of the mechanisms of cross-resistance. A causal model reflecting precise genetic mechanisms behind cross-resistance is one of the solutions of the problem. Another approach would be to apply advanced statistical methods like factor analysis. In future, we plan to extend the model to cover estimation of a posteriori coverages for combinations of antibiotics (or actual antibiotic treatments).
Percent new coverages
0% 0,00
The database used to derive the a posteriori probabilities provided rather a high number of antibiotics tested - on average each isolate was tested with 19 antibiotics. In the case of a sparse database, one could probably expect different output from the model. The validation of the model was performed looking at the Brier distances calculated across pathogen/antibiotic combinations in the derivation database. A fair comparison needs an external validation database.
0,40
0,50
p value
Fig. 4. Number of a posteriori determinations for pathogen/antibiotic combinations for which susceptibility has not been determined, expressed in percent of total number of susceptibility determinations. 3.3. Results across all pathogen/antibiotic combinations Across all pathogen/antibiotic combinations in the derivation database, the Brier distances for a priori coverages was 39%, reduced to 25% for a posteriori coverages (MinMax strategy), indicating that there is a significant advantage to the amendment (p<10 -99). 4. DISCUSSION
REFERENCES Brier G.W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Rev, 78, pp 1-3. Kofoed K., Zalounina A., Andersen O., Lisby G., Paul M., Leibovici L., Andreassen S., (2008). Performance of the TREAT decision support system in an environment with low prevalence of resistant pathogens. Artif Intell Med, 40(1), pp 57-63. Zalounina A., Paul M., Leibovici L., Andreassen S., (2007). A stochastic model of coverage – the effects of coresistance between antibiotic treatments. Artif Intell Med, 40(1), pp 57-63.
The model proposed for calculation of a posteriori coverages for a broader range of antibiotics can have several practical implications. The a posteriori coverages can be used by clinicians and infectious diseases specialists for prediction of pathogens’ susceptibility to antibiotics and prescription of antibiotics not tested for in vitro susceptibility. The model can also be applied to a process of planning susceptibility tests at a microbiological laboratory by specifying that drugs can be added or omitted from a test, allowing limited test sets. Finally, the computerized algorithm for a posteriori coverage will be incorporated into the TREAT system. TREAT is a computerized decision support system for antibiotic treatment The system can reduce inappropriate treatments both in countries with intermediate levels of microbial resistance, from 43% to 30%, and in countries with low levels of microbial resistance, from 34% to 14% (Kofoed et al. 2008).
318
Design of a Method for Prediction of Susceptibility to Antimicrobials Steen Andreassen*, Alina Zalounina* Mical Paul**, Leonard Leibovici** *Center for Model-Based Medical Decision Support, Aalborg University, Aalborg Denmark (Tel:+45-9940-8812; e-mail: [email protected] or [email protected] ). **Rabin Medical Centre, Petah Tiqva, Israel (e-mail: [email protected] or [email protected]) Abstract: The results from the in vitro susceptibility tests are usually available only for a limited set of antibiotics. This paper provides a practical method for prediction of a posteriori probability for coverage for all relevant antibiotics. The method combines susceptibility results for a limited set of antibiotics with data on cross-resistance between antibiotics. The Brier distance was used to measure the accuracy of the predicted coverages. Across all pathogen/antibiotic combinations in the derivation database, the Brier distances for a priori coverages was 39%, reduced to 25% for predicted a posteriori coverages, indicating that there is a significant advantage to the method proposed (p<10 -99). Keywords: Bacterial infections; Antibiotic therapy; Susceptibility test; Cross-resistance; A posteriori probability. 1. INTRODUCTION The choice of appropriate antimicrobial treatment depends on many factors, one of the most important being accurate estimates of the in vitro susceptibility (or its negation, called resistance) of the available antibiotics to a given pathogen. At the onset of an infectious episode, the probability of susceptibility, in the following called the coverage, can be estimated from a database of microbial isolates containing the results of susceptibility tests for a range of antimicrobials. Over the course of the infectious episode, the clinical opinion on coverage will change from its a priori value at the beginning of the episode. One of such breaking points for reconsidering opinion on coverage is the stage where susceptibility results become available. However, there are several problems that remain after antibiotic susceptibilities are available: not all antibiotics can be tested; an antibiotic that was not tested might be needed because of: patient allergies, availability, to cover a polymicrobial infection using a single antibiotic, etc. For example, let us assume that Acinetobacter spp. had its susceptibility tested for meropenem (MER), but not for imipenem (IMI). The abbreviation MER will also represent a stochastic variable for coverage obtained by meropenem. Its two states, representing respectively that the pathogen is sensitive (MER=S) or resistant (MER=R) to meropenem, will be called mer and ¬mer. The same terminology is applied to the rest of antimicrobial drugs mentioned in the paper. From the isolate database (Table 1) it can be seen that Acinetobacter spp. is resistant to MER. How does the knowledge about resistance to MER affect the a posteriori probability for coverage by imipenem (IMI): P(imi | ¬mer)?
978-3-902661-49-4/09/$20.00 © 2009 IFAC
Previously (Zalounina et al. 2007), we have described how a coverage database, containing both resistances and crossresistances (the phenomenon of cross-resistance reflects the fact that some antibiotics share the mechanism by which they attack pathogens), can be calculated from the isolate database. From the conditional probabilities given in the coverage database (Table 2) it can be seen that P(imi | ¬mer) = 9 %. This can be compared to the a priori probability for coverage by imipenem P(imi) = 56% (Table 2). Often information on susceptibility becomes available not on a single antimicrobial but simultaneously on a number of antimicrobials. This happens in 10-30% of the patients in whom the pathogen is successfully isolated, and subsequently tested for susceptibility to a set of antimicrobials. For economic and practical reasons, this set is restricted, typically to 8-25 different antimicrobials. For example, assuming that the isolated pathogen Acinetobacter spp. from episode 1 had its susceptibility tested for ofloxacin (OFL), meropenem, and amikacin (AMK) with the results given in Table 1, we could ask for an a posteriori estimate of coverage for imipenem: P(imi | ¬ofl, ¬mer, amk) A tabulation of these conditional probabilities, derived from the isolate database is not a practical solution. In the isolate database used in this paper, the average number of antimicrobials tested for susceptibility is 19. This would not only make the resulting 20 dimensional conditional probability tables very large, but it would also make it difficult to derive reliable estimates for the conditional probabilities because of the limited size of the isolate database. To make things even more unwieldy the 19 antibiotics tested on average is not a fixed set, but a choice out of 31 antibiotics tested on a regular basis, which means that a single large 20 dimensional table is not sufficient. A
313
10.3182/20090812-3-DK-2006.0064
7th IFAC MCBMS (MCBMS'09) Aalborg, Denmark, August 12-14, 2009
Table 2. The coverage database for Acinetobacter spp.
number of tables covering different combinations of tested antimicrobials is required, making this approach impractical.
a)
The purpose of this paper is to develop a practical method for calculation of a posteriori coverages for all relevant antibiotics, given susceptibility results for a restricted set of antibiotics.
Susceptibility test antibiotic Treatment
2. METHODS AND MATERIAL IMI
The methods section will start by defining the isolate and coverage database. Subsequently the problem of calculating a posteriori coverages will be formally defined and some approximate solutions will be proposed.
Episode Pathogen OFL MER AMK 1 Acinetobacter spp. R R S 2 E. coli R S R 3 Proteus spp. S NA S S = susceptible R = resistant NA = the susceptibility is not assessed
N IMI,imi
N IMI, ¬imi
88
0
1
84
74
0
0.96
0.09
N MER,imi N MER, ¬imi N MER,mer N MER, ¬mer
152 P(mer)
N IMI,mer N IMI, ¬mer
P(imi|imi) P(imi|¬imi) P(imi|mer) P(imi|¬mer )
88
70
152
0
P(mer|imi) P(mer|¬imi) P(mer|mer) P(imi|¬mer)
0.57
The isolate database includes 3347 clinically significant blood isolates collected between 2002-2004 at Rabin Medical Center in Israel. The isolate database specifies the name of a pathogen and its in vitro susceptibility to a total of 31 antibiotics. A segment of the database, showing susceptibility to three antibiotics - ofloxacin, meropenem and amikacin, is given in Table 1.
Table 1. The isolate database
P(imi)
MER a posteriori
NIMI 88
NMER
2.1 The isolate and coverage databases
Based on the isolate database the counts of coverage and probabilities of coverage for each pathogen were calculated and stored in the coverage database. A segment of the coverage database for Acinetobacter spp. is shown in Table 2. Table 2a contains the a priori/a posteriori coverages and a priori/a posteriori counts these probabilities were based on. The a posteriori data for the combination imipenem/meropenem (see bold rectangle in Table 2a) are expanded in Table 2b. The first three rows following the header row in Table 2b show a posteriori counts. E.g., among all isolates tested against imipenem, 74 were resistant to meropenem (NIMI, ¬mer = 74). Out of these 74 isolates, 7 were susceptible to imipenem (Nimi, ¬mer=7) and 67 resistant to imipenem (N¬imi, ¬mer=67). Conditional probabilities are shown in the next two rows. E.g., P(imi | ¬mer) = Nimi, ¬mer / NIMI, ¬mer = 7 / 74 = 0.09. The following row demonstrates, that knowledge that Acinetobacter is sensitive to meropenem increases the odds for imipenem=S by odds ratio (ORimi, mer) of 21.5. Resistance to meropenem reduces the odds by an odds ratio of 0.08 (ORimi, ¬mer). The p values chosen to decide which odds ratios are significantly different from 1, were calculated using the Fisher’s exact test and are shown in the last row.
a priori
0.56
MER
IMI
0.92
0.04
1
0
b) MER
IMI
N IMI,mer 84 Nimi,mer 81 N ¬imi,mer 3 P(imi | mer) 0.96 P(¬imi | mer) 0.04 ORimi| mer 21.5
N IMI, ¬mer 74 N imi, ¬mer 7 N ¬imi, ¬mer 67 P(imi | ¬mer) 0.09 P(¬imi | ¬mer) 0.91 ORimi| ¬mer 0.08
p< 10-11
p< 10-11
Table (b) is an expansion of data shown in the bold rectangle in Table (a)
2.2 Calculation of a posteriori coverage given susceptibility results for a set of antibiotics Formally, the problem to be considered is as follows. We assume, that the susceptibility of a given pathogen to a set of T antimicrobials: AT = {A1, … At, … AT} has been tested, and each At has been assigned a value at (susceptible) or ¬at (resistant). The a priori coverage, i.e. probability of susceptibility to an antimicrobial B, which is not a member of the set AT, is P(b). Estimate the a posteriori coverage for B, given AT, P(b|AT). In the introduction it was already concluded, that compiling estimates for P(b|AT) from the isolate database is not a viable solution. A detailed knowledge of the mutations that confer resistance to the given pathogen might make it possible to
314
7th IFAC MCBMS (MCBMS'09) Aalborg, Denmark, August 12-14, 2009
make a causal model of the coverage, but in general such knowledge is not available. Alternatively stochastic models, such as regression or logistic regression or factor analysis, with the At’s as independent variables might be a solution, but suffer from the fact that the set AT varies from isolate to isolate. A step towards a simple approximation can be taken by using Bayes theorem to write: P(b|AT) = P(AT|b) P(b) / P(AT) (1) If we make the “naïve Bayes” assumption of conditional independence within the set AT, given b, then P(AT|b) = Пt P(At|b)
(2)
A Bayes inversion of P(At|b) gives: P(At|b) = P(b|At) P(At) / P(b) Insertion of (2) and (3) in (1) gives:
(3)
As an intermediate solution between taking all significant odds ratios into account and only taking the maximum and minimum odds ratios into account we can explore a MinMax2 strategy. In this strategy the 2 largest significant odds ratios and the 2 smallest significant odds ratios are used in the calculation of Пt ORb|At. 2.4 Assessment of quality of a posteriori probability
P(b|AT) = Пt P(At|b) P(b) / P(AT) = Пt [P(b|At) P(At) / P(b)] P(b) / P(AT)
can be improved by reducing the number of factors in (8), simply by ignoring some of the susceptibility results. This alternative strategy called MinMax, where only the largest factor in ORb|at and the smallest factor in ORb|¬at are used, will be considered. Odds ratios greater than 1 represent reasons for susceptibility and odds ratios smaller than 1 represent reasons for resistance. The rationale for the MinMax strategies is that the underlying genetic reasons for susceptibility are probably independent from the reasons for resistance.
(4)
To quantify the potential improvement in estimates of coverage, the Brier distance (Brier 1950) can be used across all antibiotics for which a susceptibility result is available:
ODb = P(b) / P(¬b) and the aposteriori odds ODb|AT for P(b|AT) as:
(5)
BrierDist =
ODb|AT = P(b|AT) / P(¬b|AT)
(6)
= Пt [P(b|At) / P(b)] Пt P(At) P(b) / P(AT)
We now define the a priori odds ODb for P(b) as:
2 ∑ P(b t | A T-t ) + ∑ (1 - P(b t | A T-t )) 2 A ˛ Re s A t˛ Sus t NBrier
Inserting (4) into (6) gives:
NBrier = Nsus + Nres,
ODb|AT = (Пt [P(b|At) / P(b)] P(b)) / (Пt [P(¬b|At) / P(¬b)] P(¬b)) = (Пt [P(b|At) / P(¬b|At) / ODb] ODb)
(7)
Equation (7) can also be written as: ODb|AT = Пt ORb|At ODb
where Sus is the set of antibiotics to which the isolate is susceptible and Res is the set of antibiotics to which the isolate is resistant; Nsus and Nres are the number of members in the sets Sus and Res, respectively.
(8)
where: ORb|At = P(b|At) / P(¬b|At) / ODb
(11)
3. RESULTS
(9)
3.1 Illustration of the method for a single pathogen Finally the a posteriori probability can be calculated from: P(b|AT) = ODb|AT / (1 + ODb|AT)
(10)
using that P(¬b|AT) = 1- P(b|AT). The proposed method will explore the use of (8) for the calculation of the a posteriori odds for coverage.
A posteriori determinations performed as given by eqs. 8 and 10 will be tested on the isolate database. The testing will be done by using eq. 8 to predict coverage for each of the antibiotics for which a susceptibility results is available, using the rest of the susceptibility results. 3.1.1 Result for a single antibiotic
2.3 Strategies for a posteriori determination Equation (8) was derived using the naïve Bayes assumption of conditional independence. The accuracy of the calculation therefore depends on how realistic the assumption of conditional independence is. The first strategy chosen in this paper is to use all available significant odds ratios ORb|At, assuming complete independence between susceptibility results. In the result section it will be explored if the accuracy
Let us for example use eq. 8 to predict the coverage for imipenem. The result from the susceptibility testing shows that the isolate, Acinetobacter spp., actually is not tested against imipenem (Table 3, column 4). The a priori coverage of imipenem is 56% (Table 3, column 3), and it is therefore a priori far from given that treating Acinetobacter spp. with imipenem is appropriate. To use equation 8 to calculate a posteriori coverage for imipenem, given all the other
315
7th IFAC MCBMS (MCBMS'09) Aalborg, Denmark, August 12-14, 2009
susceptibility results we need ORimi|At for each At ˛ AT , where AT = Sus ¨ Res. The odds ratios ORimi|At are given in Table 3, columns 5 and 6. Only odds ratios which are significantly different from 1 (p ≤ 0.1) are listed – for drugs amikacin, minocycline (MIN) og meropenem.
Apriori coverage %
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
imipenem 56 penbritin 3 piperacillin 11 keflin 2 cefuroxim 4 ceftazidim 17 ceftriaxone 3 gentamycin 25 tobramycin 34 amikacin 30 septrin 14 ofloxacin 15 axtreonam 3 colistin 98 tetracyclin 10 augmentin 5 ciproflocacin 15 minocycline 52 tazocin 14 cefepime 18 meropenem 57 At = susceptibility result NA = not assessed
At
NA R R R R R R R R S R R R S R R R R R R S
ORimi|at
ORimi|¬at
p ≤ 0.1
p ≤ 0.1
3.1.2 Results for all antibiotics The a posteriori “all odds ratios” coverage for imipenem is given in Table 4, along with the a posteriori coverages for other antibiotics (column 5). Table 4 also shows the susceptibility results for 21 antibiotics (including known result for imipenem) (column 4).
Table 3. Susceptibility test results and odds ratios for Acinetobacter spp. being susceptible to imipenem given knowledge about susceptibility to other antibiotics Acinetobac ter spp.
column 4. This would make imipenem an excellent choice for treating this infection.
Table 4. A posteriori coverages Acinetobac ter spp.
BrierDist 0.27 0.27 1 imipenem 56 S 99 2 penbritin 3 R 0 3 piperacillin 11 R 0 4 keflin 2 R 5 cefuroxim 4 R 0 6 ceftazidim 17 R 0 7 ceftriaxone 3 R 0 8 gentamycin 25 R 0 9 tobramycin 34 R 8 10 amikacin 30 S 3 11 septrin 14 R 0 12 ofloxacin 15 R 0 13 axtreonam 3 R 0 14 colistin 98 S 15 tetracyclin 10 R 0 16 augmentin 5 R 0 17 ciproflocacin 15 R 0 18 minocycline 52 R 74 19 tazocin 14 R 0 20 cefepime 18 R 0 21 meropenem 57 S 93 At = susceptibility result NA = not assessed
5.8
0.47
21.5
Forming the product over the sets Sus and Res, respectively, including only values of ORimi|At which are significantly different from 1 for the antibiotic imipenem gives: Пt ORimi|At = ORimi | amk * ORimi | ¬min * ORimi | mer = 5.83 * 0.47 * 21.5 = 58.9
Apriori At All odds MinMax MinMax2 coverage coverage coverage ratios % % % coverage 0,229 93
0,231 99
0 3
0 0
0 3 0 2 31 29 7 0 0
0 1 0 1 23 29 1 0 0
3 0 0 66 9 8 83
1 0 0 74 1 3 93
The Brier distance for the a posteriori coverage can then be compared to the Brier distance for the a priori coverage. In row 2 of Table 4 the Brier distances calculated across all antibiotics tested against Acinetobacter spp. are shown.
Inserting this in eq. 8 gives: ODimi|AT = Пt ORimi | At * ODimi = 58.9 *0.56 / (1-0.56) = 75.0 Eq. 10 then gives: P(imi | amk, ¬min, mer) = ODimi | AT / (1 + ODimi | AT) = 98.7% which is our estimate of the a posteriori coverage of imipenem, given the susceptibility results from Table 3,
The Brier distance for the a posteriori coverage is 0,27, which is the same as the Brier distance for the apriori coverage, indicating that for this isolate there is no net advantage of the amendment shown. For two antibiotics, amikacin and minocycline, the a posteriori coverages (3% and 74%,
316
7th IFAC MCBMS (MCBMS'09) Aalborg, Denmark, August 12-14, 2009
the
actual
A posteriori coverages resulting from the MinMax strategy, where only the largest factor in ORb|at and the smallest factor in ORb|¬at are used, are shown in Table 4 column 6. It can be seen that this MinMax strategy reduces the error and overall there is a reduction in the Brier distance to 0.229. The resulting a posteriori coverages from the MinMax2 strategy, where the 2 largest significant odds ratios and the 2 smallest significant odds ratios are used in the calculation of Пt ORb|At, are shown in Table 4 column 7 with a resulting Brier distance of 0.231, slightly higher than for the MinMax strategy.
Fig. 2 shows that the MinMax strategy is better than the other two strategies, although the difference between the MinMax and the MinMax2 strategies is quite small. All strategies are however considerably better than just keeping the a priori probabilities of coverage. For the a priori coverages, the Brier distance is 0.39. It is also apparent that a p value of 0.1 is not necessarily optimal. The Brier distance for the MinMax strategy decreases with the p value, levelling off around 0.254 for a p value of 0.1 to 0.15 and reaching 0.252 for a p value of 0.5. Comparison of strategies 0,31
Brier distance
respectively) deviate considerably from susceptibility results (S and R, respectively).
3.2. Comparison of strategies To determine if it applies in general that the MinMax strategy is better than the MinMax2 strategy and the all-odds-ratios strategy, these strategies were applied to all 3347 isolates in the database. Results will depend on the p value chosen to decide which odds ratios are significantly different from 1. Therefore the a posteriori determinations were carried out with different p values, ranging from 0.02 to 0.5. In the database a total of 31 different antibiotics were used to test the susceptibility of the isolated pathogens. Each isolate was tested with a subset of the 31 antibiotics. On average each isolate was tested with 19 antibiotics or 61% of the antibiotics, giving a total of 63590 determinations of susceptibility in the database. The results were obtained by deleting a single result of a susceptibility test for a single isolated pathogen, and then attempting to estimate the a posteriori coverage P(bt|AT\t) from the remaining susceptibility test results for that isolated pathogen. This was repeated for all 63590 susceptibility results. Fig. 1 shows that for most of those 63590 susceptibility determinations, a posteriori coverages can be estimated from the other determinations. With a p value of 0.02 for significance of odds ratios, 74.7% of those can be estimated, and with a p value of 0.5, 91.6% can be estimated.
0,30 All odds ratios MinMax odds ratios MinMax2 odds ratios
0,29 0,28 0,27 0,26 0,25 0,00
0,10
0,20
0,30
0,40
0,50
p value
Fig. 2. Brier distances for the 3 different strategies where all 63590 susceptibility determinations have been estimated, either by calculating a posteriori coverage when possible or by keeping the a priori coverage. To achieve a better understanding of the appropriate choice of p value, the Brier distance for the MinMax strategy has been plotted again vs. the p value in Fig. 3. In addition the Brier distance for the subset of susceptibilities where a posteriori determinations were possible was plotted. The Brier distance for the a priori coverages is the remaining subset of susceptibility determinations. The results show that the Brier distances for the a posteriori coverages is fairly constant about 0.27, irrespective of the p value. The Brier distance of the remaining, a priori susceptibilities is substantially smaller. This is mainly because the remaining, a priori susceptibility determinations tend to have extreme values of a priori coverage, either close to 0 or close to 1. Brier distances for a posteriori and a priori coverages
Percent amendments
0,30
Brier distance
100% 90% 80%
0,20
0,10
70% 0,00
0,10
0,20
0,30
0,40
0,00 0,00
0,50
p value
Brier for all Brier for a posteriori coverages Brier for a priori coverages
0,10
0,20
0,30
0,40
0,50
p value
Fig. 1. Percent of susceptibilities which can be determined a posteriori, based on the other determinations.
Fig. 3. Brier distances for a posteriori coverages, for the remaining a priori coverages and for the union of those two sets.
317
7th IFAC MCBMS (MCBMS'09) Aalborg, Denmark, August 12-14, 2009
The final choice of p value is a compromise. A p value higher than 0.1 reduces the Brier distance slightly, but introduces additional a posteriori determinations, which are based on odds ratios with dubious significance. A choice of p value of 0.1 therefore seems a reasonable compromise. Fig. 4 shows the number of new a posteriori determinations in percent of the number of susceptibility determinations. For a p value of 0.1, 7.4% new a posteriori determinations could be made, corresponding to about 1.4 new a posteriori determinations on average per isolate. This is a relatively low number, but should be seen in the light that with an average of 19 susceptibility determinations per pathogen, there are not too many relevant antibiotics left without a determination of susceptibility.
15%
To conclude, the method described allows estimation of a posteriori coverages for all relevant antibiotics, given susceptibility test results for a restricted set of antibiotics. This technique provides a significant improvement in estimates of coverage, compared to a priori estimates, with the MinMax strategy being slightly better than the MinMax2 strategy.
10% 5%
0,10
0,20
0,30
One of the drawbacks of the model is in its simplified approach to understanding and imitation of the mechanisms of cross-resistance. A causal model reflecting precise genetic mechanisms behind cross-resistance is one of the solutions of the problem. Another approach would be to apply advanced statistical methods like factor analysis. In future, we plan to extend the model to cover estimation of a posteriori coverages for combinations of antibiotics (or actual antibiotic treatments).
Percent new coverages
0% 0,00
The database used to derive the a posteriori probabilities provided rather a high number of antibiotics tested - on average each isolate was tested with 19 antibiotics. In the case of a sparse database, one could probably expect different output from the model. The validation of the model was performed looking at the Brier distances calculated across pathogen/antibiotic combinations in the derivation database. A fair comparison needs an external validation database.
0,40
0,50
p value
Fig. 4. Number of a posteriori determinations for pathogen/antibiotic combinations for which susceptibility has not been determined, expressed in percent of total number of susceptibility determinations. 3.3. Results across all pathogen/antibiotic combinations Across all pathogen/antibiotic combinations in the derivation database, the Brier distances for a priori coverages was 39%, reduced to 25% for a posteriori coverages (MinMax strategy), indicating that there is a significant advantage to the amendment (p<10 -99). 4. DISCUSSION
REFERENCES Brier G.W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Rev, 78, pp 1-3. Kofoed K., Zalounina A., Andersen O., Lisby G., Paul M., Leibovici L., Andreassen S., (2008). Performance of the TREAT decision support system in an environment with low prevalence of resistant pathogens. Artif Intell Med, 40(1), pp 57-63. Zalounina A., Paul M., Leibovici L., Andreassen S., (2007). A stochastic model of coverage – the effects of coresistance between antibiotic treatments. Artif Intell Med, 40(1), pp 57-63.
The model proposed for calculation of a posteriori coverages for a broader range of antibiotics can have several practical implications. The a posteriori coverages can be used by clinicians and infectious diseases specialists for prediction of pathogens’ susceptibility to antibiotics and prescription of antibiotics not tested for in vitro susceptibility. The model can also be applied to a process of planning susceptibility tests at a microbiological laboratory by specifying that drugs can be added or omitted from a test, allowing limited test sets. Finally, the computerized algorithm for a posteriori coverage will be incorporated into the TREAT system. TREAT is a computerized decision support system for antibiotic treatment The system can reduce inappropriate treatments both in countries with intermediate levels of microbial resistance, from 43% to 30%, and in countries with low levels of microbial resistance, from 34% to 14% (Kofoed et al. 2008).
318