- Email: [email protected]
A model for diagnosis of pulmonary infections in solid-organ transplant recipients
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
journal homepage: www.intl.elsevierhealth.com/journals/cmpb
A model for diagnosis of pulmonary infections in solid-organ transplant recipients Galia Kariv a,b,∗ , Vered Shani a , Elad Goldberg a,b , Leonard Leibovici a,b , Mical Paul a,b a b
Rabin Medical Center, Petach-Tikva, Israel Sackler Faculty of Medicine, Tel-Aviv University, Ramat-Aviv, Israel
a r t i c l e
i n f o
a b s t r a c t
Article history:
Background: Opportunistic pulmonary infections are a major cause of morbidity and mortal-
Received 14 December 2009
ity among solid organ transplant recipients. The diagnosis of these infections is challenging
Received in revised form
because of the broad spectrum of bacteria, fungi and viruses affecting these patients and the
13 April 2010
lack of specific signs and symptoms. Treatment directed at the offending organism started
Accepted 28 June 2010
as soon as possible improves survival. Objective: To develop a decision support system for the diagnosis of pulmonary infections
Keywords:
in solid-organ transplant recipients. The model’s goal is to improve the accuracy of the
Transplant
diagnosis and thus the appropriateness of empirical treatment.
Solid organ
Design: The model is built using a Bayesian network (also known as causal probabilistic
Infection
network). The network is based on pathogen segments which are the main building blocks
DSS
of the model. Segments share common risk factors, such as time after transplantation,
CPN
latent infections of donor/recipient and organ transplanted. The segments are linked at
Bayesian network
symptoms, signs and diagnostic tests common to all pathogens. The outputs of the model are predicted probabilities of infectious pathogens. To populate the model with data we have mainly abstracted data from the literature, using a systematic approach. The structure of the model and its adaptation for decision support will be presented. Evaluation: The first evaluation phase assessed the model’s diagnosis in a series of 20 representative cases of opportunistic infections. A match between the case’s diagnosis and the model’s prediction was achieved in 17/20 of cases. The next evaluation phase will consist of a prospective observational study comparing the accuracy of the model’s diagnosis vs. that of the physician within 24 h of episode onset, as compared with a gold-standard diagnosis ascribed to the patients at the end of the infectious episode by two independent experts. Data for this phase are currently collected prospectively. © 2010 Elsevier Ireland Ltd. All rights reserved.
1.
Introduction
The number of living solid organ transplant recipients has increased dramatically in recent years. At the end of 2005 there were about 163,000 people living in the United States with a
functioning transplanted organ, 2.1% increase from 2004 and more than 160% rise since 1997 [1]. Infections are a major problem following transplantation, incurring severe morbidity and mortality. Bacterial infections occur in 33–68% of liver transplant recipients, 21–30% of heart, 47% of kidney, and 54% of lung transplant recipients [2].
∗ Corresponding author at: Internal Department E, Rabin Medical Center, Beilinson Hospital, 39 Jabotinski St., Petah Tikva 49100, Israel. Tel.: +972 3 9377358. E-mail address: [email protected] (G. Kariv). 0169-2607/$ – see front matter © 2010 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.cmpb.2010.06.018
136
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
Infections were responsible for 21% of deaths among heart transplant recipients, 59% of deaths in liver and kidney and 63% of deaths in lung transplant recipients [3]. Other than the direct consequences of infection, infections have several indirect untoward effects in solid-organ transplant recipients. These include further immune suppression, development of malignancy and allograft rejection [4,5]. Infections are caused by several mechanisms including latent infections (present in a dormant state and reactivated during immune suppression), community acquired or hospital-acquired infections. The spectrum of infectious agents includes bacteria, mycobacteria, viruses (especially Herpes viruses), fungi and parasites. Infections pose a complex diagnostic challenge to clinicians, mainly due to this broad spectrum of pathogens. The incidence and etiology of infection episodes is affected by many and diverse risk factors. Common to most of them is time after transplantation, type of organ transplanted, and prophylaxis given. The type of organ transplanted influences the site of infection and the range of infecting pathogens. For example, the incidence of Cytomegalovirus pneumonia is higher in heart/lung transplant recipients than other organ recipients [6]. Antibiotic, antiviral and antifungal prophylaxis commonly given to these patients further complicates the diagnostic matrix. Another diagnostic challenge is the fact that rejection of the graft may mimic infections, or may present concurrently with an infection, thus complicating the differential diagnosis of transplant recipient patients presenting with fever. Performing a complete evaluation, using all available diagnostic tests is not practical. Some diagnostic tests are invasive, thus might harm the patient. Treatment is, therefore, often given empirically, prior to or without microbiological documentation. Appropriate empirical antibiotic treatment improves patients’ survival [7–9]. Unruly empirical treatment is associated with induction of resistance and adverse events. Optimal therapy should consist of the narrowest-spectrum treatment targeting the suspected pathogen/s given as early as possible. The matrix leading to an accurate diagnosis, in order to select an appropriate empirical treatment is complex and data are often sparse. Computerized decision support systems have been shown to improve clinician performance when tackling complex matrices [10–16]. The burden of infections following transplantation combined with the complexity of accurate diagnosis calls for a decision support system for infections among solid organ transplant recipients. We set to develop a computerized decision support system to assist clinicians in the diagnosis of infectious complications following solidorgan transplantation. To our knowledge no such system has been previously developed.
2.
of the project, the system’s predictions will be compared to a “gold standard” diagnosis provided for each episode and to a physician’s diagnosis of the same episode. We aim to improve the accuracy of the diagnosis for these patients, and show an improvement of 15%, from 60% to 75% correct diagnoses with the proposed model.
3.
Design
3.1.
Decision support system platform
The model was built using a Bayesian network (also known as causal probabilistic network – CPN – or belief network) [17]. Bayesian networks are directed acyclic graphs, consisting of nodes and links. Nodes represent variables, with a finite number of states (or have the option of continuous variable, currently not used by this model). The links between nodes represent causal relations. For example, a disease, represented as a variable/node causes a symptom (also represented as a variable/node). The causal relation between a disease and symptom is represented by a link. Each node in a CPN represents a stochastic variable. An arc, the relation between a node and its parent nodes, is provided by conditional probabilities (e.g. the probability of the symptom given the existence or non-existence of the disease or the probability of a disease given the existence of risk factors).
3.2.
Model construction
In order to construct the network, data were extracted from literature and local databases, as previously described for similar networks [18]. We used a hierarchical approach to data extraction from the literature, where data are preferentially extracted from systematic reviews with meta-analysis; if not available, we searched for prospective clinical studies, then retrospective and finally case reports and expert opinion. Two main data and information aspects were extracted: • Information on pathogens causing infections. The main risk factors for each pathogen were defined and incidence rates according to the various risk factors obtained. • Symptoms, signs and diagnostic tests for the different infections. Mainly sensitivity and specificity data for the variables included in the model were obtained. The interdependence between the variables was identified and modelled correctly to avoid repeating effects that lie along the same pathogenesis pathway. Optimally, such inter-dependence should be derived from a database. Such database does not exist, and inter-dependence was modelled via understanding of the epidemiology and pathogenesis of the infection.
Objectives
The main objective is to build and test a model for a decision support system for the diagnosis of pulmonary infections in solid organ transplant recipients. Data are entered into the model, including patient’s background and current status. The model’s output includes the likely pathogens and the probabilities of their causing an infection. During the testing phase
A brief example is given for data extraction on the incidence of Mycobacterium tuberculosis. Data were extracted from several studies [19–34] on the incidence of M. Tuberculosis in solid organ transplant recipients and from the WHO global TB database [35] for country endemicity level. We extracted from each study the number of transplanted patients with infection out of a cohort of transplanted patients, per type of organ
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
137
Fig. 1 – Conditional probabilities table of tuberculosis infection.
and timing of occurrence post-transplantation. For this type of infection, we determined that time after transplantation did not affect the incidence of infection, while the local level of endemicity did. Therefore, we derived daily incidence values per type of organ and level of endemicity, by dividing the total incidence by the number of follow-up days in the study. Data from all available studies were pooled. Calculated daily incidence values were entered into the relevant probability tables in the CPN, using the two risk factors: endemicity level and type of organ transplanted, represented by two parent nodes, respectively (Fig. 1). Similarly data were extracted and pooled regarding symptoms, signs and diagnostic testing for M. tuberculosis. A similar process was repeated for each net pathogen. A full description of the literature search is beyond the scope of this article. However, all data sources, data extracted and values used for the CPN were documented and will be available with the final version of the system.
The model was built to handle major viruses, fungi and bacteria responsible for most opportunistic pulmonary infections in these patients. The model currently supports infections caused by Cytomegalovirus, Nocardia spp., Pneumocystis jiroveci, Aspergillus spp., M. tuberculosis and other non-tuberculous mycobacteria. Additional pathogens will be added to the model, including community acquired respiratory viruses, Ebstein-Barr virus and other Herpes viruses, Mucor sp. and other mold fungi. Before using the model as a decision support system, incorporating support for non-opportunistic pathogens is needed. Bacterial pathogens are the most frequent infections during the first month after transplantation. The support for diagnosis of these pathogens will be handled by interaction, after calibration, with the TREAT system, a decision support system for antibiotic treatment in inpatients with common bacterial infections [36].
3.3.
A pathogen segment includes a centric node, providing the probability of the infection by this pathogen, expressed as daily incidence as extracted from literature (Figs. 1 and 3). The parent nodes of this infection incidence node are its risk factors. Some risk factors are common to most pathogen segments, including the time elapsed after transplantation, organ transplanted and whether prophylaxis against this pathogen is given. The time elapsed since transplantation is fragmented into periods representing the different stages of immune suppression and pathogen exposure following transplantation. During the first month after transplantation, most infectious episodes are related to surgical complications and to the fact the patient is hospitalized. Most opportunistic infections are encountered during the period of the second to sixth month following transplantation. The second half-year posttransplantation is characterized mostly by infections common
Model structure overview
The network is based on pathogen segments, which are the main building blocks of the model (Fig. 2). Each pathogen segment describes its pathogenesis, including risk factors, laboratory tests, clinical signs and histology findings. All pathogen segments are linked to shared segments, which present common signs and symptoms, such as sepsis signs (which include fever, erythrocyte sedimentation rate, albumin and other signs), lung histology, respiratory signs, skin lesions and others. Most clinical symptoms relevant to the respiratory system are not directly linked to pathogen segments, but rather connected to common histology abnormalities segments. The reasoning behind this design is the attempt to achieve the correct weight of these symptoms, and prevent an exaggerated contribution of each individual sign, symptom or radiological finding.
3.4.
Pathogen segment structure
Fig. 2 – Network overview schema.
138
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
Fig. 3 – Pathogen segment overview.
to general population, although opportunistic infections may still occur [2,37,38]. Because of the fact that more data in the literature are available referencing these time windows, we chose to use them as the main risk factor and not epidemiologic exposure or immunosuppression therapy which might parallel and overlap the time frames concept. The information on an infection during the previous month is also a risk factor for many pathogens, and is taken into account in the daily incidence node for the specific infection. Each site holds the specific symptoms and laboratory tests relevant to it. Laboratory tests include cultures, molecular based essays, and antigen/antibody tests. Their nodes hold the sensitivity and specificity for each test, preferably as documented for solid organ transplant recipients. Common symptoms and signs are shared between different pathogen segments.
3.5.
Software and tools
The model was built using Hugin Expert, a graphical user interface for building and running Bayesian networks (http://www.hugin.com). A web based application was developed in-house to enable the collection of patient and episode data, in order to run different evaluation phases of the developed model. The application was built with Oracle Application Express, a rapid web application development tool, on top of Oracle database 10 g Express Edition (http://www.oracle.com). When turning the model into a decision support system, an API (Application Programming Interface) will be provided to integrate between existing computerized medical record systems within the implementing hospital site and the decision support model. A Java package for the execution of the model was developed in-house, written with Oracle Jdeveloper and Hugin Developer Java API. This package enables loading a Bayesian network developed in Hugin Expert into a repository stored in the database; map database tables (storing episode data)
to nodes and states within the network; execute the episodes and store the output probabilities of the network. An option of running episodes on the network in batch mode is also provided.
4.
Evaluation
4.1.
Proof of concept and face validity test
Twenty solid organ transplant recipients’ episodes were included in the first evaluation phase. These cases are representative of the major syndromes seen among solid organ transplant recipients and the expected diagnosis was ascribed by an expert in infectious diseases. Data describing the episodes, the expected diagnosis and all model’s predictions whose probabilities were above 5% are shown in Table 1. An agreement between the expected diagnosis and model’s diagnosis was observed in 17 of 20 episodes. The probabilities of the expected diagnosis predicted by the model for the 17 agreements ranged between 9% and 99%. Three disagreements were observed: in episodes #4 and #7 the predicted probabilities for Aspergillus were too high and in #13 the probability might have been too low. We believe that these disagreements are due to incorrect probabilities of sepsis with Aspergillus spp. infections and will be calibrated.
4.2.
Observational testing
A model for a decision support system needs to be evaluated on real data prior to its interventional clinical testing. The data used by the model are extensive. We also need to document physician’s estimated diagnosis and treatment decisions at the different time points in the patient’s encounter (early, initial assessment and consequent encounters with clinical improvement/deterioration or when diagnostic tests become available). As a database containing all needed information is not available, a database is currently prospectively gathered.
139
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
Table 1 – Patient and episode characteristic tested in proof of concept evaluation phase. # 1
Episode description
Expected diagnosis
Model’s diagnosis
Heart recipient, 1–6 months post-transplantation Prolonged high fever Low albumin, elevated creatinine Necrotic skin lesions Cavitary lung lesions #1 without necrotic skin lesions
Aspergillus or other fungi
Aspergillus 99%
Aspergillus/Nocardia/TB
Kidney recipient, 1–6 months post-transplantation. Fever 38◦ , dyspnea, tachypnea, hypoxia, bilateral interstitial infiltrates, CMV D+/R+, on CMV Prophylaxis #3 with PCP Prophylaxis
PCP
TB 28% PCP 26% Nocardia 26% Aspergillus 19% PCP 97%
5 6
#3 without CMV Prophylaxis #3 with CMV antigenemia positive (more than 10 cells)
PCP CMV
7
Liver recipient, first month post-transplantation. Fever, low albumin, low leucocytes, lymphopernia, therapy for acute rejection, diffuse interstitial and lobar infiltrate. CMV, PCP and Aspergillus prophylaxis #7, 1–6 months post-transplantation Kidney recipient, 3 months post-transplantation, 38◦ fever, elevated creatinine, low albumin, PCP, Aspergillus and CMV prophylaxis, CMV serostatus D+/R−, normal chest X-ray Lung recipient, 3 years post-transplantation, no prophylaxis, bilateral lung nodules, no fever, no hypoxia, no other respiratory signs #10 with positive culture of Aspergillus from sputum #10 with positive culture of Aspergillus from BAL #10 with positive galactomannan on serum #10 with positive galactomannan and positive culture of Aspergillus from BAL #10 with negative galactomannan on serum, positive culture of Aspergillus and Nocardia from BAL #15 with positive Aspergillus PCR from BAL #15 with positive galactomannan in serum Lung recipient, 1 year post-transplantation, on PCP prophylaxis, diarrhea, vomiting, pleural effusion, negative CMV antigenemia,
Bacteremia, candidemia and suspected PCP infection
2
3
4
8 9
10
11 12 13 14
15
16 17 18
CMV
Rise in PCP and CMV in relation to #7 CMV GI
Aspergillus 31% CMV 2%a PCP 97% CMV 50% (CMV pneumonia 41%, CMV GI 36%) PCP 19% Aspergillus 10% Aspergillus 40% PCP 31% TB 7% CMV 6%a
PCP 47% TB 10% Aspergillus 9% CMV 9% CMV 79% (CMV GI 55%)
Mostly non-infectious
Nocardia 1% Aspergillus 0.2%b
Mostly non-infectious
Likely Aspergillus
Nocardia 1.1% Aspergillus 0.7%b Aspergillus 5% Nocardia 1% Aspergillus 2.4% Nocardia 1.1% Aspergillus 34%
Nocardia
Nocardia 80%
Nocardia
Nocardia 80%
Nocardia and Aspergillus
Nocardia 73% Aspergillus 9% Null (all below 0.1%)
Possible Aspergillus Possible Aspergillus
Non-infectious or non-specific viral infection
140
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
– Table 1 (Continued) #
Episode description
19
Kidney recipient, 6 years post-transplantation, CMV R+, donor serostatus unknown, diarrhea 14 days, fever 38◦ Heart and Lung recipient, 2 years post-transplantations. Hospitalized for 2 weeks, receiving antibiotics for suspected pneumonia, (chest X-ray – small\non-specific infiltrate, left lung), Diarrhea with mucous
20
Expected diagnosis
Model’s diagnosis
Drug related
Null (all below 0.1%)
Clostridium difficile
Clostridium difficile PMC 62% Antibiotic associated diarrhea 8%
Model’s diagnosis showing pathogens with >5% probability. PCP – Pneumocystis jiroveci, TB – Mycobacteria Tuberculosis, CMV – Cytomegalovirus, D – donor, and R – recipient. a Probably due to high Aspergillus sepsis rates. Needs calibration. b Top 2 results presented by the model.
We are in the process of developing a computerized decision support system for the diagnosis of infections among solid organ transplant recipients. The model was developed using a Bayesian network logical platform, which serves well the model’s domain:
design enables a more accurate contribution of each assay to the probability of a Cytomegalovirus infection. While populating the network with probabilities, values can be derived from both literature and/or databases. The Bayesian network platform allows for natural use of sensitivity and specificity values of laboratory tests and imaging procedures. For example, studies have shown that a galactomannan test of BAL fluid in lung transplant recipients at index cutoff value of ≥0.5 has a sensitivity of 60% and specificity of 95% for the diagnosis of invasive pulmonary Aspergillosis [39]. These values are populated as conditional probabilities to the CPN. The post-test probability in the CPN is naturally and appropriately dependent on the pretest probability of the infection. Clinicians commonly do not have the knowledge of the diagnostic characteristics of a test and cannot apply known or estimated values to the myriad tests involved in the diagnostic process. The network allows for missing data, which is common in these highly complex situations. It uses the baseline probabilities populated to the network, derived from literature and/or local databases. For example, in many cases the donor’s Cytomegalovirus serology status is unknown. The CPN estimates the serology status using the Cytomegalovirus endemicity level of the donor’s country. The model is built with an explicit definition of universal vs. local factors. Universal factors need not be modified locally because they represent biological behavior or standard test characteristics (e.g., the association of sepsis with fever). Local factors need calibration (e.g. diagnostic performance of an in-house PCR assay).
• The platform allows the constructor of the model to build it in a structural manner, following the complexity of the domain. The network can be built using multiple layers/levels of variables, links and causations, following disease pathogenesis. An accurate graphical structure accounts correctly for the independence or the relationships between the variables in the model. For example, there are multiple laboratory assays available for the diagnosis of Cytomegalovirus (antigenemia, PCR, shell vial, etc.). While constructing the pathogen segment of Cytomegalovirus, a multi-layer design was used (Fig. 4). This
The preliminary assessment of the model on a series of “representative” cases showed that the model accurately predicted the assumed etiology in 85% of the cases. This assessment has served to identify major problems of the model. The limitation of the “representative” case assessment is the lack of a gold standard diagnosis. The next testing phase will consist of an observational study using “real-life” data collected prospectively from solid-organ transplant recipients with a defined gold standard diagnosis. We are aware of several limitations of the model. There is a difficulty in representing
• •
Fig. 4 – Cytomegalovirus laboratory assays schema.
This cohort will be used to compare the percentage of physicians’ accurate diagnosis vs. that of the model. Both diagnoses will be compared to a “gold standard” diagnosis, determined by a committee of two independent physicians, based on clinical, microbiological and radiological data with the outcome of the patient. We are aiming to collect 300 patients to show an improvement over physicians’ diagnosis of 15% (from 60% to 75%) with a power of 0.8 among 95 assessable episodes (McNemar test).
5.
•
Comments •
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
the time dimension in CPNs. The CPN looks at one specific point in time; the next encounter with the patient has to base on the previous encounter and handle new information. This is technically complex to achieve with our current model. There is a difficulty in representing co-infections because the different pathogens in the CPNs compete to explain the same signs and symptoms. Another limitation is the fact that some data used in the CPN are imprecise or unavailable and thus estimated; the ultimate test of this will be the performance of the DSS. Randomized controlled trials are the gold standard evaluation method of interventions in medicine. Decision support systems are no different from other interventions and should be evaluated as rigorously as any other intervention aimed at improving patient’s care. Ultimately this CPN will have to be tested in a randomized control trial.
[10]
[11]
[12]
[13]
[14]
Funding This work was supported in part by a grant from the Israel Science Foundation No.1193/07.
[15]
Conflict of interest statement Authors have no conflict of interest.
[16]
references [17] [1] Annual Report, 2007 Annual Report of the U.S. Organ Procurement and Transplantation Network and the Scientific Registry of Transplant Recipients: Transplant Data 1997–2006. Health Resources and Services Administration, Healthcare Systems Bureau, Division of Transplantation, Rockville, MD. [2] R. Patel, C.V. Paya, Infections in solid-organ transplant recipients, Clin. Microbiol. Rev. 10 (1997) 86–124. [3] B. Sanroman Budino, E. Vazquez Martul, S. Pertega Diaz, A. Veiga Barreiro, E. Carro Rey, J. Mosquera Reboredo, Autopsy-determined causes of death in solid organ transplant recipients, Transplant. Proc. 36 (2004) 787–789. [4] R.H. Rubin, A. Schaffner, R. Speich, Introduction to the Immunocompromised Host Society consensus conference on epidemiology, prevention, diagnosis, and management of infections in solid-organ transplant patients, Clin. Infect. Dis. 33 (Suppl. 1) (2001) S1–S4. [5] F. Cainelli, S. Vento, Infections and solid organ transplant rejection: a cause-and-effect relationship? Lancet Infect. Dis. 2 (2002) 539–549. [6] M.G. Ison, J.A. Fishman, Cytomegalovirus pneumonia in transplant recipients, Clin. Chest Med. 26 (December (4)) (2005), 691–705, viii. [7] L. Leibovici, I. Shraga, M. Drucker, H. Konigsberger, Z. Samra, S.D. Pitlik, The benefit of appropriate empirical antibiotic treatment in patients with bloodstream infection, J. Intern. Med. 244 (1998) 379–386. [8] S. Harbarth, J. Garbino, J. Pugin, J.A. Romand, D. Lew, D. Pittet, Inappropriate initial antimicrobial therapy and its effect on survival in a clinical trial of immunomodulating therapy for severe sepsis, Am. J. Med. 115 (2003) 529–535. [9] J. Garnacho-Montero, J.L. Garcia-Garmendia, A. Barrero-Almodovar, F.J. Jimenez-Jimenez, C. Perez-Paredes, C. Ortiz-Leyba, Impact of adequate empirical antibiotic
[18]
[19]
[20]
[21] [22] [23]
[24]
[25]
[26]
[27]
[28]
141
therapy on the outcome of patients admitted to the intensive care unit with sepsis, Crit. Care Med. 31 (2003) 2742–2751. V.L. Yu, L.M. Fagan, S.M. Wraith, et al., Antimicrobial selection by a computer. A blinded evaluation by infectious diseases experts, JAMA 242 (1979) 1279–1282. R.S. Evans, D.C. Classen, S.L. Pestotnik, H.P. Lundsgaarde, J.P. Burke, Improving empiric antibiotic selection using computer decision support, Arch. Intern. Med. 154 (1994) 878–884. L. Leibovici, V. Gitelman, Y. Yehezkelli, et al., Improving empirical antibiotic treatment: prospective, nonintervention testing of a decision support system, J. Intern. Med. 242 (1997) 395–400. R.S. Evans, S.L. Pestotnik, D.C. Classen, et al., A computer-assisted management program for antibiotics and other antiinfective agents, N. Engl. J. Med. 338 (1998) 232–238. M. Paul, S. Andreassen, E. Tacconelli, A.D. Nielsen, N. Almanasreh, U. Frank, R. Cauda, L. Leibovici, TREAT Study Group, Improving empirical antibiotic treatment using TREAT, a computerized decision support system: cluster randomized trial, J. Antimicrob. Chemother. 58 (December (6)) (2006) 1238–1245 (Epub September 23, 2006). M. Paul, A.D. Nielsen, E. Goldberg, S. Andreassen, E. Tacconelli, N. Almanasreh, U. Frank, R. Cauda, L. Leibovici, TREAT Study Group, Prediction of specific pathogens in patients with sepsis: evaluation of TREAT, a computerized decision support system, J. Antimicrob. Chemother. 59 (June (6)) (2007) 1204–1207 (Epub April 21, 2007). K. Kofoed, A. Zalounina, O. Andersen, G. Lisby, M. Paul, L. Leibovici, S. Andreassen, Performance of the TREAT decision support system in an environment with a low prevalence of resistant pathogens, J. Antimicrob. Chemother. 63 (February (2)) (2009) 400–404 (Epub December 17, 2008). T.A. Stephenson, An introduction to Bayesian network theory and usage, in: IDIAP Research Report 00-03, 2000. L. Leibovici, et al., A causal probabilistic network for optimal treatment of bacterial infections, IEEE Trans. Knowl. Data Eng. 12 (4) (2000) 517–528. R. Shreeniwas, et al., Opportunistic bronchopulmonary infections after lung transplantation: clinical and radiographic findings, Radiology 200 (2) (1996) 349–356. V. Jha, et al., Successful management of pulmonary tuberculosis in renal allograft recipients in a single center, Kidney Int. 56 (1999) 1944–1950. S. Kesten, C. Chaparro, Mycobacterial infections in lung transplant recipients, Chest 115 (3) (1999) 741–745. R. Lenner, et al., Pulmonary complications in cardiac transplant recipients, Chest 120 (2) (2001) 508–513. H.P. McAdams, J.J. Erasmus, S.M. Palmer, Complications (excluding hyperinflation) involving the native lung after single-lung transplantation: incidence, radiologic features, and clinical importance, Radiology 218 (1) (2001) 233–241. C. Cavusoglu, et al., Mycobacterium tuberculosis infection and laboratory diagnosis in solid-organ transplant recipients, Clin. Transplant. 16 (4) (2002) 257–261. J.A. Queipo, et al., Mycobacterial infection in a series of 1261 renal transplant recipients, Clin. Microbiol. Infect. 9 (6) (2003) 518–525. A. Vandermarliere, et al., Mycobacterial infection after renal transplantation in a Western population, Transplant. Infect. Dis. 5 (1) (2003) 9–15. G.C. Chang, et al., The diagnosis of pneumonia in renal transplant recipients using invasive and noninvasive procedures, Chest 125 (2) (2004) 541–547. R.B. Hsu, et al., Infectious complications after heart transplantation in Chinese recipients, Am. J. Transplant. 5 (8) (2005) 2011–2016.
142
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
[29] T. Jie, et al., Mycobacterial infections after kidney transplant, Transplant. Proc. 37 (2) (2005) 937–939. [30] P. Morales, et al., Pulmonary tuberculosis in lung and heart-lung transplantation: fifteen years of experience in a single center in Spain, Transplant. Proc. 37 (9) (2005) 4050–4055. [31] R. Naqvi, et al., Efficacy of isoniazid prophylaxis in renal allograft recipients, Transplant. Proc. 38 (7) (2006) 2057–2058. [32] A.C. Chan, et al., Implications for management of Mycobacterium tuberculosis infection in adult-to-adult live donor liver transplantation, Liver Int. 27 (1) (2007) 81–85. [33] A.Y. Peleg, et al., Opportunistic infections in 547 organ transplant recipients receiving alemtuzumab, a humanized monoclonal CD-52 antibody, Clin. Infect. Dis. 44 (2) (2007) 204–212. [34] J. Torre-Cisneros, et al., Tuberculosis after solid-organ transplant: incidence, risk factors, and clinical characteristics in the RESITRA (Spanish Network of Infection in Transplantation) cohort, Clin. Infect. Dis. 48 (12) (2009) 1657–1665.
[35] World Health Organization, Global Tuberculosis Database, http://www.who.int/tb/country/global tb database/en/index.html (accessed 12/09). [36] L. Leibovici, M. Paul, A.D. Nielsen, E. Tacconelli, S. Andreassen, The TREAT project: decision support and prediction using causal probabilistic networks, Int. J. Antimicrob. Agents 30 (November (Suppl. 1)) (2007) S93–S102 (Epub September 24, 2007). [37] J.A. Fishman, R.H. Rubin, Infection in organ-transplant recipients, N. Engl. J. Med. 338 (June (24)) (1998) 1741–1751. [38] J.A. Fishman, Infection in solid-organ transplant recipients, N. Engl. J. Med. 357 (December (25)) (2007) 2601–2614. [39] S. Husain, D.L. Paterson, S.M. Studer, M. Crespo, J. Pilewski, M. Durkin, J.L. Wheat, B. Johnson, L. McLaughlin, C. Bentsen, K.R. McCurry, N. Singh, Aspergillus galactomannan antigen in the bronchoalveolar lavage fluid for the diagnosis of invasive aspergillosis in lung transplant recipients, Transplantation 83 (May (10)) (2007) 1330–1336.
journal homepage: www.intl.elsevierhealth.com/journals/cmpb
A model for diagnosis of pulmonary infections in solid-organ transplant recipients Galia Kariv a,b,∗ , Vered Shani a , Elad Goldberg a,b , Leonard Leibovici a,b , Mical Paul a,b a b
Rabin Medical Center, Petach-Tikva, Israel Sackler Faculty of Medicine, Tel-Aviv University, Ramat-Aviv, Israel
a r t i c l e
i n f o
a b s t r a c t
Article history:
Background: Opportunistic pulmonary infections are a major cause of morbidity and mortal-
Received 14 December 2009
ity among solid organ transplant recipients. The diagnosis of these infections is challenging
Received in revised form
because of the broad spectrum of bacteria, fungi and viruses affecting these patients and the
13 April 2010
lack of specific signs and symptoms. Treatment directed at the offending organism started
Accepted 28 June 2010
as soon as possible improves survival. Objective: To develop a decision support system for the diagnosis of pulmonary infections
Keywords:
in solid-organ transplant recipients. The model’s goal is to improve the accuracy of the
Transplant
diagnosis and thus the appropriateness of empirical treatment.
Solid organ
Design: The model is built using a Bayesian network (also known as causal probabilistic
Infection
network). The network is based on pathogen segments which are the main building blocks
DSS
of the model. Segments share common risk factors, such as time after transplantation,
CPN
latent infections of donor/recipient and organ transplanted. The segments are linked at
Bayesian network
symptoms, signs and diagnostic tests common to all pathogens. The outputs of the model are predicted probabilities of infectious pathogens. To populate the model with data we have mainly abstracted data from the literature, using a systematic approach. The structure of the model and its adaptation for decision support will be presented. Evaluation: The first evaluation phase assessed the model’s diagnosis in a series of 20 representative cases of opportunistic infections. A match between the case’s diagnosis and the model’s prediction was achieved in 17/20 of cases. The next evaluation phase will consist of a prospective observational study comparing the accuracy of the model’s diagnosis vs. that of the physician within 24 h of episode onset, as compared with a gold-standard diagnosis ascribed to the patients at the end of the infectious episode by two independent experts. Data for this phase are currently collected prospectively. © 2010 Elsevier Ireland Ltd. All rights reserved.
1.
Introduction
The number of living solid organ transplant recipients has increased dramatically in recent years. At the end of 2005 there were about 163,000 people living in the United States with a
functioning transplanted organ, 2.1% increase from 2004 and more than 160% rise since 1997 [1]. Infections are a major problem following transplantation, incurring severe morbidity and mortality. Bacterial infections occur in 33–68% of liver transplant recipients, 21–30% of heart, 47% of kidney, and 54% of lung transplant recipients [2].
∗ Corresponding author at: Internal Department E, Rabin Medical Center, Beilinson Hospital, 39 Jabotinski St., Petah Tikva 49100, Israel. Tel.: +972 3 9377358. E-mail address: [email protected] (G. Kariv). 0169-2607/$ – see front matter © 2010 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.cmpb.2010.06.018
136
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
Infections were responsible for 21% of deaths among heart transplant recipients, 59% of deaths in liver and kidney and 63% of deaths in lung transplant recipients [3]. Other than the direct consequences of infection, infections have several indirect untoward effects in solid-organ transplant recipients. These include further immune suppression, development of malignancy and allograft rejection [4,5]. Infections are caused by several mechanisms including latent infections (present in a dormant state and reactivated during immune suppression), community acquired or hospital-acquired infections. The spectrum of infectious agents includes bacteria, mycobacteria, viruses (especially Herpes viruses), fungi and parasites. Infections pose a complex diagnostic challenge to clinicians, mainly due to this broad spectrum of pathogens. The incidence and etiology of infection episodes is affected by many and diverse risk factors. Common to most of them is time after transplantation, type of organ transplanted, and prophylaxis given. The type of organ transplanted influences the site of infection and the range of infecting pathogens. For example, the incidence of Cytomegalovirus pneumonia is higher in heart/lung transplant recipients than other organ recipients [6]. Antibiotic, antiviral and antifungal prophylaxis commonly given to these patients further complicates the diagnostic matrix. Another diagnostic challenge is the fact that rejection of the graft may mimic infections, or may present concurrently with an infection, thus complicating the differential diagnosis of transplant recipient patients presenting with fever. Performing a complete evaluation, using all available diagnostic tests is not practical. Some diagnostic tests are invasive, thus might harm the patient. Treatment is, therefore, often given empirically, prior to or without microbiological documentation. Appropriate empirical antibiotic treatment improves patients’ survival [7–9]. Unruly empirical treatment is associated with induction of resistance and adverse events. Optimal therapy should consist of the narrowest-spectrum treatment targeting the suspected pathogen/s given as early as possible. The matrix leading to an accurate diagnosis, in order to select an appropriate empirical treatment is complex and data are often sparse. Computerized decision support systems have been shown to improve clinician performance when tackling complex matrices [10–16]. The burden of infections following transplantation combined with the complexity of accurate diagnosis calls for a decision support system for infections among solid organ transplant recipients. We set to develop a computerized decision support system to assist clinicians in the diagnosis of infectious complications following solidorgan transplantation. To our knowledge no such system has been previously developed.
2.
of the project, the system’s predictions will be compared to a “gold standard” diagnosis provided for each episode and to a physician’s diagnosis of the same episode. We aim to improve the accuracy of the diagnosis for these patients, and show an improvement of 15%, from 60% to 75% correct diagnoses with the proposed model.
3.
Design
3.1.
Decision support system platform
The model was built using a Bayesian network (also known as causal probabilistic network – CPN – or belief network) [17]. Bayesian networks are directed acyclic graphs, consisting of nodes and links. Nodes represent variables, with a finite number of states (or have the option of continuous variable, currently not used by this model). The links between nodes represent causal relations. For example, a disease, represented as a variable/node causes a symptom (also represented as a variable/node). The causal relation between a disease and symptom is represented by a link. Each node in a CPN represents a stochastic variable. An arc, the relation between a node and its parent nodes, is provided by conditional probabilities (e.g. the probability of the symptom given the existence or non-existence of the disease or the probability of a disease given the existence of risk factors).
3.2.
Model construction
In order to construct the network, data were extracted from literature and local databases, as previously described for similar networks [18]. We used a hierarchical approach to data extraction from the literature, where data are preferentially extracted from systematic reviews with meta-analysis; if not available, we searched for prospective clinical studies, then retrospective and finally case reports and expert opinion. Two main data and information aspects were extracted: • Information on pathogens causing infections. The main risk factors for each pathogen were defined and incidence rates according to the various risk factors obtained. • Symptoms, signs and diagnostic tests for the different infections. Mainly sensitivity and specificity data for the variables included in the model were obtained. The interdependence between the variables was identified and modelled correctly to avoid repeating effects that lie along the same pathogenesis pathway. Optimally, such inter-dependence should be derived from a database. Such database does not exist, and inter-dependence was modelled via understanding of the epidemiology and pathogenesis of the infection.
Objectives
The main objective is to build and test a model for a decision support system for the diagnosis of pulmonary infections in solid organ transplant recipients. Data are entered into the model, including patient’s background and current status. The model’s output includes the likely pathogens and the probabilities of their causing an infection. During the testing phase
A brief example is given for data extraction on the incidence of Mycobacterium tuberculosis. Data were extracted from several studies [19–34] on the incidence of M. Tuberculosis in solid organ transplant recipients and from the WHO global TB database [35] for country endemicity level. We extracted from each study the number of transplanted patients with infection out of a cohort of transplanted patients, per type of organ
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
137
Fig. 1 – Conditional probabilities table of tuberculosis infection.
and timing of occurrence post-transplantation. For this type of infection, we determined that time after transplantation did not affect the incidence of infection, while the local level of endemicity did. Therefore, we derived daily incidence values per type of organ and level of endemicity, by dividing the total incidence by the number of follow-up days in the study. Data from all available studies were pooled. Calculated daily incidence values were entered into the relevant probability tables in the CPN, using the two risk factors: endemicity level and type of organ transplanted, represented by two parent nodes, respectively (Fig. 1). Similarly data were extracted and pooled regarding symptoms, signs and diagnostic testing for M. tuberculosis. A similar process was repeated for each net pathogen. A full description of the literature search is beyond the scope of this article. However, all data sources, data extracted and values used for the CPN were documented and will be available with the final version of the system.
The model was built to handle major viruses, fungi and bacteria responsible for most opportunistic pulmonary infections in these patients. The model currently supports infections caused by Cytomegalovirus, Nocardia spp., Pneumocystis jiroveci, Aspergillus spp., M. tuberculosis and other non-tuberculous mycobacteria. Additional pathogens will be added to the model, including community acquired respiratory viruses, Ebstein-Barr virus and other Herpes viruses, Mucor sp. and other mold fungi. Before using the model as a decision support system, incorporating support for non-opportunistic pathogens is needed. Bacterial pathogens are the most frequent infections during the first month after transplantation. The support for diagnosis of these pathogens will be handled by interaction, after calibration, with the TREAT system, a decision support system for antibiotic treatment in inpatients with common bacterial infections [36].
3.3.
A pathogen segment includes a centric node, providing the probability of the infection by this pathogen, expressed as daily incidence as extracted from literature (Figs. 1 and 3). The parent nodes of this infection incidence node are its risk factors. Some risk factors are common to most pathogen segments, including the time elapsed after transplantation, organ transplanted and whether prophylaxis against this pathogen is given. The time elapsed since transplantation is fragmented into periods representing the different stages of immune suppression and pathogen exposure following transplantation. During the first month after transplantation, most infectious episodes are related to surgical complications and to the fact the patient is hospitalized. Most opportunistic infections are encountered during the period of the second to sixth month following transplantation. The second half-year posttransplantation is characterized mostly by infections common
Model structure overview
The network is based on pathogen segments, which are the main building blocks of the model (Fig. 2). Each pathogen segment describes its pathogenesis, including risk factors, laboratory tests, clinical signs and histology findings. All pathogen segments are linked to shared segments, which present common signs and symptoms, such as sepsis signs (which include fever, erythrocyte sedimentation rate, albumin and other signs), lung histology, respiratory signs, skin lesions and others. Most clinical symptoms relevant to the respiratory system are not directly linked to pathogen segments, but rather connected to common histology abnormalities segments. The reasoning behind this design is the attempt to achieve the correct weight of these symptoms, and prevent an exaggerated contribution of each individual sign, symptom or radiological finding.
3.4.
Pathogen segment structure
Fig. 2 – Network overview schema.
138
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
Fig. 3 – Pathogen segment overview.
to general population, although opportunistic infections may still occur [2,37,38]. Because of the fact that more data in the literature are available referencing these time windows, we chose to use them as the main risk factor and not epidemiologic exposure or immunosuppression therapy which might parallel and overlap the time frames concept. The information on an infection during the previous month is also a risk factor for many pathogens, and is taken into account in the daily incidence node for the specific infection. Each site holds the specific symptoms and laboratory tests relevant to it. Laboratory tests include cultures, molecular based essays, and antigen/antibody tests. Their nodes hold the sensitivity and specificity for each test, preferably as documented for solid organ transplant recipients. Common symptoms and signs are shared between different pathogen segments.
3.5.
Software and tools
The model was built using Hugin Expert, a graphical user interface for building and running Bayesian networks (http://www.hugin.com). A web based application was developed in-house to enable the collection of patient and episode data, in order to run different evaluation phases of the developed model. The application was built with Oracle Application Express, a rapid web application development tool, on top of Oracle database 10 g Express Edition (http://www.oracle.com). When turning the model into a decision support system, an API (Application Programming Interface) will be provided to integrate between existing computerized medical record systems within the implementing hospital site and the decision support model. A Java package for the execution of the model was developed in-house, written with Oracle Jdeveloper and Hugin Developer Java API. This package enables loading a Bayesian network developed in Hugin Expert into a repository stored in the database; map database tables (storing episode data)
to nodes and states within the network; execute the episodes and store the output probabilities of the network. An option of running episodes on the network in batch mode is also provided.
4.
Evaluation
4.1.
Proof of concept and face validity test
Twenty solid organ transplant recipients’ episodes were included in the first evaluation phase. These cases are representative of the major syndromes seen among solid organ transplant recipients and the expected diagnosis was ascribed by an expert in infectious diseases. Data describing the episodes, the expected diagnosis and all model’s predictions whose probabilities were above 5% are shown in Table 1. An agreement between the expected diagnosis and model’s diagnosis was observed in 17 of 20 episodes. The probabilities of the expected diagnosis predicted by the model for the 17 agreements ranged between 9% and 99%. Three disagreements were observed: in episodes #4 and #7 the predicted probabilities for Aspergillus were too high and in #13 the probability might have been too low. We believe that these disagreements are due to incorrect probabilities of sepsis with Aspergillus spp. infections and will be calibrated.
4.2.
Observational testing
A model for a decision support system needs to be evaluated on real data prior to its interventional clinical testing. The data used by the model are extensive. We also need to document physician’s estimated diagnosis and treatment decisions at the different time points in the patient’s encounter (early, initial assessment and consequent encounters with clinical improvement/deterioration or when diagnostic tests become available). As a database containing all needed information is not available, a database is currently prospectively gathered.
139
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
Table 1 – Patient and episode characteristic tested in proof of concept evaluation phase. # 1
Episode description
Expected diagnosis
Model’s diagnosis
Heart recipient, 1–6 months post-transplantation Prolonged high fever Low albumin, elevated creatinine Necrotic skin lesions Cavitary lung lesions #1 without necrotic skin lesions
Aspergillus or other fungi
Aspergillus 99%
Aspergillus/Nocardia/TB
Kidney recipient, 1–6 months post-transplantation. Fever 38◦ , dyspnea, tachypnea, hypoxia, bilateral interstitial infiltrates, CMV D+/R+, on CMV Prophylaxis #3 with PCP Prophylaxis
PCP
TB 28% PCP 26% Nocardia 26% Aspergillus 19% PCP 97%
5 6
#3 without CMV Prophylaxis #3 with CMV antigenemia positive (more than 10 cells)
PCP CMV
7
Liver recipient, first month post-transplantation. Fever, low albumin, low leucocytes, lymphopernia, therapy for acute rejection, diffuse interstitial and lobar infiltrate. CMV, PCP and Aspergillus prophylaxis #7, 1–6 months post-transplantation Kidney recipient, 3 months post-transplantation, 38◦ fever, elevated creatinine, low albumin, PCP, Aspergillus and CMV prophylaxis, CMV serostatus D+/R−, normal chest X-ray Lung recipient, 3 years post-transplantation, no prophylaxis, bilateral lung nodules, no fever, no hypoxia, no other respiratory signs #10 with positive culture of Aspergillus from sputum #10 with positive culture of Aspergillus from BAL #10 with positive galactomannan on serum #10 with positive galactomannan and positive culture of Aspergillus from BAL #10 with negative galactomannan on serum, positive culture of Aspergillus and Nocardia from BAL #15 with positive Aspergillus PCR from BAL #15 with positive galactomannan in serum Lung recipient, 1 year post-transplantation, on PCP prophylaxis, diarrhea, vomiting, pleural effusion, negative CMV antigenemia,
Bacteremia, candidemia and suspected PCP infection
2
3
4
8 9
10
11 12 13 14
15
16 17 18
CMV
Rise in PCP and CMV in relation to #7 CMV GI
Aspergillus 31% CMV 2%a PCP 97% CMV 50% (CMV pneumonia 41%, CMV GI 36%) PCP 19% Aspergillus 10% Aspergillus 40% PCP 31% TB 7% CMV 6%a
PCP 47% TB 10% Aspergillus 9% CMV 9% CMV 79% (CMV GI 55%)
Mostly non-infectious
Nocardia 1% Aspergillus 0.2%b
Mostly non-infectious
Likely Aspergillus
Nocardia 1.1% Aspergillus 0.7%b Aspergillus 5% Nocardia 1% Aspergillus 2.4% Nocardia 1.1% Aspergillus 34%
Nocardia
Nocardia 80%
Nocardia
Nocardia 80%
Nocardia and Aspergillus
Nocardia 73% Aspergillus 9% Null (all below 0.1%)
Possible Aspergillus Possible Aspergillus
Non-infectious or non-specific viral infection
140
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
– Table 1 (Continued) #
Episode description
19
Kidney recipient, 6 years post-transplantation, CMV R+, donor serostatus unknown, diarrhea 14 days, fever 38◦ Heart and Lung recipient, 2 years post-transplantations. Hospitalized for 2 weeks, receiving antibiotics for suspected pneumonia, (chest X-ray – small\non-specific infiltrate, left lung), Diarrhea with mucous
20
Expected diagnosis
Model’s diagnosis
Drug related
Null (all below 0.1%)
Clostridium difficile
Clostridium difficile PMC 62% Antibiotic associated diarrhea 8%
Model’s diagnosis showing pathogens with >5% probability. PCP – Pneumocystis jiroveci, TB – Mycobacteria Tuberculosis, CMV – Cytomegalovirus, D – donor, and R – recipient. a Probably due to high Aspergillus sepsis rates. Needs calibration. b Top 2 results presented by the model.
We are in the process of developing a computerized decision support system for the diagnosis of infections among solid organ transplant recipients. The model was developed using a Bayesian network logical platform, which serves well the model’s domain:
design enables a more accurate contribution of each assay to the probability of a Cytomegalovirus infection. While populating the network with probabilities, values can be derived from both literature and/or databases. The Bayesian network platform allows for natural use of sensitivity and specificity values of laboratory tests and imaging procedures. For example, studies have shown that a galactomannan test of BAL fluid in lung transplant recipients at index cutoff value of ≥0.5 has a sensitivity of 60% and specificity of 95% for the diagnosis of invasive pulmonary Aspergillosis [39]. These values are populated as conditional probabilities to the CPN. The post-test probability in the CPN is naturally and appropriately dependent on the pretest probability of the infection. Clinicians commonly do not have the knowledge of the diagnostic characteristics of a test and cannot apply known or estimated values to the myriad tests involved in the diagnostic process. The network allows for missing data, which is common in these highly complex situations. It uses the baseline probabilities populated to the network, derived from literature and/or local databases. For example, in many cases the donor’s Cytomegalovirus serology status is unknown. The CPN estimates the serology status using the Cytomegalovirus endemicity level of the donor’s country. The model is built with an explicit definition of universal vs. local factors. Universal factors need not be modified locally because they represent biological behavior or standard test characteristics (e.g., the association of sepsis with fever). Local factors need calibration (e.g. diagnostic performance of an in-house PCR assay).
• The platform allows the constructor of the model to build it in a structural manner, following the complexity of the domain. The network can be built using multiple layers/levels of variables, links and causations, following disease pathogenesis. An accurate graphical structure accounts correctly for the independence or the relationships between the variables in the model. For example, there are multiple laboratory assays available for the diagnosis of Cytomegalovirus (antigenemia, PCR, shell vial, etc.). While constructing the pathogen segment of Cytomegalovirus, a multi-layer design was used (Fig. 4). This
The preliminary assessment of the model on a series of “representative” cases showed that the model accurately predicted the assumed etiology in 85% of the cases. This assessment has served to identify major problems of the model. The limitation of the “representative” case assessment is the lack of a gold standard diagnosis. The next testing phase will consist of an observational study using “real-life” data collected prospectively from solid-organ transplant recipients with a defined gold standard diagnosis. We are aware of several limitations of the model. There is a difficulty in representing
• •
Fig. 4 – Cytomegalovirus laboratory assays schema.
This cohort will be used to compare the percentage of physicians’ accurate diagnosis vs. that of the model. Both diagnoses will be compared to a “gold standard” diagnosis, determined by a committee of two independent physicians, based on clinical, microbiological and radiological data with the outcome of the patient. We are aiming to collect 300 patients to show an improvement over physicians’ diagnosis of 15% (from 60% to 75%) with a power of 0.8 among 95 assessable episodes (McNemar test).
5.
•
Comments •
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
the time dimension in CPNs. The CPN looks at one specific point in time; the next encounter with the patient has to base on the previous encounter and handle new information. This is technically complex to achieve with our current model. There is a difficulty in representing co-infections because the different pathogens in the CPNs compete to explain the same signs and symptoms. Another limitation is the fact that some data used in the CPN are imprecise or unavailable and thus estimated; the ultimate test of this will be the performance of the DSS. Randomized controlled trials are the gold standard evaluation method of interventions in medicine. Decision support systems are no different from other interventions and should be evaluated as rigorously as any other intervention aimed at improving patient’s care. Ultimately this CPN will have to be tested in a randomized control trial.
[10]
[11]
[12]
[13]
[14]
Funding This work was supported in part by a grant from the Israel Science Foundation No.1193/07.
[15]
Conflict of interest statement Authors have no conflict of interest.
[16]
references [17] [1] Annual Report, 2007 Annual Report of the U.S. Organ Procurement and Transplantation Network and the Scientific Registry of Transplant Recipients: Transplant Data 1997–2006. Health Resources and Services Administration, Healthcare Systems Bureau, Division of Transplantation, Rockville, MD. [2] R. Patel, C.V. Paya, Infections in solid-organ transplant recipients, Clin. Microbiol. Rev. 10 (1997) 86–124. [3] B. Sanroman Budino, E. Vazquez Martul, S. Pertega Diaz, A. Veiga Barreiro, E. Carro Rey, J. Mosquera Reboredo, Autopsy-determined causes of death in solid organ transplant recipients, Transplant. Proc. 36 (2004) 787–789. [4] R.H. Rubin, A. Schaffner, R. Speich, Introduction to the Immunocompromised Host Society consensus conference on epidemiology, prevention, diagnosis, and management of infections in solid-organ transplant patients, Clin. Infect. Dis. 33 (Suppl. 1) (2001) S1–S4. [5] F. Cainelli, S. Vento, Infections and solid organ transplant rejection: a cause-and-effect relationship? Lancet Infect. Dis. 2 (2002) 539–549. [6] M.G. Ison, J.A. Fishman, Cytomegalovirus pneumonia in transplant recipients, Clin. Chest Med. 26 (December (4)) (2005), 691–705, viii. [7] L. Leibovici, I. Shraga, M. Drucker, H. Konigsberger, Z. Samra, S.D. Pitlik, The benefit of appropriate empirical antibiotic treatment in patients with bloodstream infection, J. Intern. Med. 244 (1998) 379–386. [8] S. Harbarth, J. Garbino, J. Pugin, J.A. Romand, D. Lew, D. Pittet, Inappropriate initial antimicrobial therapy and its effect on survival in a clinical trial of immunomodulating therapy for severe sepsis, Am. J. Med. 115 (2003) 529–535. [9] J. Garnacho-Montero, J.L. Garcia-Garmendia, A. Barrero-Almodovar, F.J. Jimenez-Jimenez, C. Perez-Paredes, C. Ortiz-Leyba, Impact of adequate empirical antibiotic
[18]
[19]
[20]
[21] [22] [23]
[24]
[25]
[26]
[27]
[28]
141
therapy on the outcome of patients admitted to the intensive care unit with sepsis, Crit. Care Med. 31 (2003) 2742–2751. V.L. Yu, L.M. Fagan, S.M. Wraith, et al., Antimicrobial selection by a computer. A blinded evaluation by infectious diseases experts, JAMA 242 (1979) 1279–1282. R.S. Evans, D.C. Classen, S.L. Pestotnik, H.P. Lundsgaarde, J.P. Burke, Improving empiric antibiotic selection using computer decision support, Arch. Intern. Med. 154 (1994) 878–884. L. Leibovici, V. Gitelman, Y. Yehezkelli, et al., Improving empirical antibiotic treatment: prospective, nonintervention testing of a decision support system, J. Intern. Med. 242 (1997) 395–400. R.S. Evans, S.L. Pestotnik, D.C. Classen, et al., A computer-assisted management program for antibiotics and other antiinfective agents, N. Engl. J. Med. 338 (1998) 232–238. M. Paul, S. Andreassen, E. Tacconelli, A.D. Nielsen, N. Almanasreh, U. Frank, R. Cauda, L. Leibovici, TREAT Study Group, Improving empirical antibiotic treatment using TREAT, a computerized decision support system: cluster randomized trial, J. Antimicrob. Chemother. 58 (December (6)) (2006) 1238–1245 (Epub September 23, 2006). M. Paul, A.D. Nielsen, E. Goldberg, S. Andreassen, E. Tacconelli, N. Almanasreh, U. Frank, R. Cauda, L. Leibovici, TREAT Study Group, Prediction of specific pathogens in patients with sepsis: evaluation of TREAT, a computerized decision support system, J. Antimicrob. Chemother. 59 (June (6)) (2007) 1204–1207 (Epub April 21, 2007). K. Kofoed, A. Zalounina, O. Andersen, G. Lisby, M. Paul, L. Leibovici, S. Andreassen, Performance of the TREAT decision support system in an environment with a low prevalence of resistant pathogens, J. Antimicrob. Chemother. 63 (February (2)) (2009) 400–404 (Epub December 17, 2008). T.A. Stephenson, An introduction to Bayesian network theory and usage, in: IDIAP Research Report 00-03, 2000. L. Leibovici, et al., A causal probabilistic network for optimal treatment of bacterial infections, IEEE Trans. Knowl. Data Eng. 12 (4) (2000) 517–528. R. Shreeniwas, et al., Opportunistic bronchopulmonary infections after lung transplantation: clinical and radiographic findings, Radiology 200 (2) (1996) 349–356. V. Jha, et al., Successful management of pulmonary tuberculosis in renal allograft recipients in a single center, Kidney Int. 56 (1999) 1944–1950. S. Kesten, C. Chaparro, Mycobacterial infections in lung transplant recipients, Chest 115 (3) (1999) 741–745. R. Lenner, et al., Pulmonary complications in cardiac transplant recipients, Chest 120 (2) (2001) 508–513. H.P. McAdams, J.J. Erasmus, S.M. Palmer, Complications (excluding hyperinflation) involving the native lung after single-lung transplantation: incidence, radiologic features, and clinical importance, Radiology 218 (1) (2001) 233–241. C. Cavusoglu, et al., Mycobacterium tuberculosis infection and laboratory diagnosis in solid-organ transplant recipients, Clin. Transplant. 16 (4) (2002) 257–261. J.A. Queipo, et al., Mycobacterial infection in a series of 1261 renal transplant recipients, Clin. Microbiol. Infect. 9 (6) (2003) 518–525. A. Vandermarliere, et al., Mycobacterial infection after renal transplantation in a Western population, Transplant. Infect. Dis. 5 (1) (2003) 9–15. G.C. Chang, et al., The diagnosis of pneumonia in renal transplant recipients using invasive and noninvasive procedures, Chest 125 (2) (2004) 541–547. R.B. Hsu, et al., Infectious complications after heart transplantation in Chinese recipients, Am. J. Transplant. 5 (8) (2005) 2011–2016.
142
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 0 4 ( 2 0 1 1 ) 135–142
[29] T. Jie, et al., Mycobacterial infections after kidney transplant, Transplant. Proc. 37 (2) (2005) 937–939. [30] P. Morales, et al., Pulmonary tuberculosis in lung and heart-lung transplantation: fifteen years of experience in a single center in Spain, Transplant. Proc. 37 (9) (2005) 4050–4055. [31] R. Naqvi, et al., Efficacy of isoniazid prophylaxis in renal allograft recipients, Transplant. Proc. 38 (7) (2006) 2057–2058. [32] A.C. Chan, et al., Implications for management of Mycobacterium tuberculosis infection in adult-to-adult live donor liver transplantation, Liver Int. 27 (1) (2007) 81–85. [33] A.Y. Peleg, et al., Opportunistic infections in 547 organ transplant recipients receiving alemtuzumab, a humanized monoclonal CD-52 antibody, Clin. Infect. Dis. 44 (2) (2007) 204–212. [34] J. Torre-Cisneros, et al., Tuberculosis after solid-organ transplant: incidence, risk factors, and clinical characteristics in the RESITRA (Spanish Network of Infection in Transplantation) cohort, Clin. Infect. Dis. 48 (12) (2009) 1657–1665.
[35] World Health Organization, Global Tuberculosis Database, http://www.who.int/tb/country/global tb database/en/index.html (accessed 12/09). [36] L. Leibovici, M. Paul, A.D. Nielsen, E. Tacconelli, S. Andreassen, The TREAT project: decision support and prediction using causal probabilistic networks, Int. J. Antimicrob. Agents 30 (November (Suppl. 1)) (2007) S93–S102 (Epub September 24, 2007). [37] J.A. Fishman, R.H. Rubin, Infection in organ-transplant recipients, N. Engl. J. Med. 338 (June (24)) (1998) 1741–1751. [38] J.A. Fishman, Infection in solid-organ transplant recipients, N. Engl. J. Med. 357 (December (25)) (2007) 2601–2614. [39] S. Husain, D.L. Paterson, S.M. Studer, M. Crespo, J. Pilewski, M. Durkin, J.L. Wheat, B. Johnson, L. McLaughlin, C. Bentsen, K.R. McCurry, N. Singh, Aspergillus galactomannan antigen in the bronchoalveolar lavage fluid for the diagnosis of invasive aspergillosis in lung transplant recipients, Transplantation 83 (May (10)) (2007) 1330–1336.