Computers in Biology and Medicine 102 (2018) 191–199
Contents lists available at ScienceDirect
Computers in Biology and Medicine journal homepage: www.elsevier.com/locate/compbiomed
Early-warning of ARDS using novelty detection and data fusion Aline Taoum a b
a,b,∗
a
, Farah Mourad-chehade , Hassan Amoud
T
b
Laboratoire de Modélisation et Sûreté des Systèmes, Institut Charles Delaunay, Université de Technologies de Troyes, Troyes, France Laboratory of Technology and Instrumentation for Health, Azm Platform for Research in Biotechnology and Its Applications, EDST, Lebanese University, Tripoli, Lebanon
A R T I C LE I N FO
A B S T R A C T
Keywords: Acute respiratory distress syndrome Data fusion Early-warning Novelty detection
Acute respiratory distress syndrome (ARDS) is a critical condition that disturbs the respiratory system and may lead to death. Early identification of this syndrome is crucial for the implementation of preventive measures. The present paper focuses on the prediction of the onset of this syndrome using physiological records of patients. Heart rate, respiratory rate, peripheral arterial oxygen saturation and mean airway blood pressure were considered. The method proposed in this paper uses first distance-based novelty detection that allows detecting deviations from normal states for each signal. Then, linear and nonlinear kernel-based data fusion algorithms are introduced to combine the individual signal decisions. The proposed method is evaluated using the MIMIC II physiological database. As a result, ARDS is detected in the early phases of occurrence with sensitivity and specificity of 65% and 100% respectively for the combination of all the signals in study. Moreover, the proposed method outperforms current state-of-the-art methods in real-time surveillance of ARDS using only physiological data with an average prediction before 39 h of onset.
1. Introduction Acute Respiratory Distress Syndrome (ARDS) is a sudden failure of the respiratory system with a mortality rate going up to 40% in most patients and reaching 80% in patients having undergone cardiac surgery [1,2]. ARDS is a syndrome of lung injury caused by an intense pulmonary inflammation. It results in pulmonary edema and an increase in permeability of the alveolar-capillary membrane, leading to difficulties of gas exchange with the blood. The term “respiratory distress syndrome” was introduced by Ashbaugh et al. in 1967 [3], then a formal definition of ARDS was developed during the American-European Consensus Conference (AECC) in 1994 [4]. The Berlin Definition of ARDS was published afterwards in 2012 to address the limitations of the AECC definition [5]. According to the Berlin Definition, three mutually exclusive subgroups of ARDS are considered using the ratio of the partial pressure of oxygen in the arterial blood PaO2 over the fraction of oxygen in the inspired air FiO2, which is expressed in mmHg: Mild (200 < PaO2/FiO≤2300), Moderate (100 < PaO2/FiO≤2200), and Severe ARDS (PaO2/FiO≤2100); All three have positive end expiratory pressure (PEEP) higher than 5 cmH2O. Epidemiological studies conducted on ARDS have shown that mortality increases with age from approximately 30% for young patients to 68% for elderly patients (> 85 years) [6,7]. ARDS is a hypoxemia that develops within one week of a severe pulmonary insult such as
∗
pneumonia, aspiration of gastric contents or sepsis [8,9] or other secondary risk factors like obesity, diabetes, high tidal volume, and others [7,10–12]. Some people who have survived ARDS recover completely, while others may have lasting lung damage and other health problems. Thus, it is important to predict the occurrence of ARDS because of its severity and profound economic implications. Predictive algorithms of ARDS were developed using clinical data of intensive care unit patients, as in Ref. [13]. Clinical variables which were collected irregularly, such as plateau pressure, mean airway pressure, oxygen saturation and heart rate, were used to establish single and combined rules. These rules consist of comparing the variables to adjusted thresholds. They reached sensitivity and specificity of 60% and 82% respectively on a single rule basis, and 50% and 90% for combined rules [13]. Other researchers have worked on mortality prediction related to ARDS [14,15] or respiratory complications [16]. In a different scenario, an approach for computing alert thresholds for pulmonary disease patients was developed. Univariate and multivariate methodologies based on novelty detection are used to analyze daily symptom scores, heart rate and oxygen saturation [17]. To our knowledge, all the work in the literature has used either physiological signals collected irregularly or discrete clinical data that require the presence of patients in hospital units for the acquisition procedure. There is no work that proposes real-time surveillance model to predict the onset of ARDS using continuous physiological signals only.
Corresponding author. Laboratoire de Modélisation et Sûreté des Systèmes, Institut Charles Delaunay, Université de Technologies de Troyes, Troyes, France. E-mail addresses:
[email protected] (A. Taoum),
[email protected] (F. Mourad-chehade),
[email protected] (H. Amoud).
https://doi.org/10.1016/j.compbiomed.2018.09.030 Received 23 May 2018; Received in revised form 10 September 2018; Accepted 30 September 2018 0010-4825/ © 2018 Elsevier Ltd. All rights reserved.
Computers in Biology and Medicine 102 (2018) 191–199
A. Taoum et al.
considering either moderate or severe definitions. One needs then to explore the clinical database to extract P/F ratios of the patients and compare it to 200, to identify whether they develop ARDS or not, during their ICU stay. In this study, the patient selection procedure is taken from Refs. [10,13], that is, the selection of mechanically ventilated subjects for more than 48 h. Therefore, a comparison can be made between methods on the same dataset. The patient selection procedure consists of different phases. First, the clinical database of MIMIC II is considered to extract patients that have undergone mechanical ventilation for more than 48 h as in Refs. [10,13]. These patients are divided into two groups according to the Berlin definition. All subjects must have PaO2/FiO2 ratios higher than 200 mmHg at the beginning of recordings to ensure a homogeneous stable state for all subjects at the beginning of the analysis. Hence, the unstable group is formed by patients developing moderate or severe ARDS, thus starting with a PaO2/FiO2 ratio (P/F) higher than 200 mmHg, then having worsening P/F values; while the stable group consists of subjects that maintain a P/F higher than 200 mmHg all over the signals recordings. After that, numeric time series of these subjects are extracted from the waveform database, that are HR, RESP, SpO2 and ABPMean which is equal to one-third of the systolic added to two-thirds of the diastolic blood pressure. These time series have a sampling frequency of one sample/minute. For the processing step, the first 6 h of the signals are removed to rule out the effect of the ventilator onset and to ensure the stable state at the beginning of the analysis, because of the random variability in initial ventilator settings. For stable subjects, data is considered for the entire period of ventilation, whereas for unstable subjects, data is extracted until a respiratory instability is detected, that is P/ F ≤ 200 mmHg. In order to validate the proposed approach, only subjects having high duration signals are considered, thus signals longer than 1440 min (24 h). Finally, 14 stable and 26 unstable subjects are obtained. Information related to patients’ demographics are given in Table 1.
This paper describes a novel approach, based on real-time analysis for early prediction of ARDS onset using only continuous physiological signals. The proposed approach is subject-based, performing novelty detection on heart rate, respiratory rate, peripheral arterial oxygen saturation and mean airway blood pressure. First, outliers are identified in each signal and compared to outliers of an initial stable segment to detect divergence. This leads to real-time individual decisions that are combined afterwards using different linear data fusion techniques based on error rates or performances. A nonlinear fusion technique based on kernel ridge regression is proposed as well. In this paper, the MIMIC II database is considered to extract the signals and evaluate the proposed method. The rest of this paper is organized as follows. Section 2 presents the proposed approach, including materials, novelty detection algorithm and data fusion. Section 3 illustrates the results of this work. Section 4 presents a discussion that interprets the obtained results. Finally, Section 5 summarizes the paper and proposes possible perspectives. 2. Real-time prediction approach The objective of the method is to develop an early warning model for both moderate and severe ARDS using time series data. In the following section, the approach is first described then the materials are briefly introduced. A detailed description of the novelty detection algorithm is given afterwards followed by the data fusion phase. 2.1. Description of the approach The aim of the model is to predict at time k the onset of ARDS for a subject s based on real-time recordings. Four types of physiological time series are examined that are the heart rate (HR), the respiratory rate (RESP), the peripheral arterial oxygen saturation (SpO2) and the mean airway blood pressure (ABPMean). Let xs,i(ℓ), i = 1, …,4, denote respectively these time series for the subject s recorded at an instant ℓ, ℓ∈{1, …,k} in minutes, and let xs,i (a:b)=(xs,i(a), …,xs,i(b)) denote the recorded data between time a and time b. Consider ys is the label of subject s, that is ys = +1 if the subject develops ARDS and ys = −1 otherwise. Then, the prediction model takes the four recorded time series xs,i (1:k) as inputs and yields yˆs (k ) as output with yˆs (k ) = −1 if ARDS is not yet predicted, and yˆs (k ) = +1 if ARDS is predicted. A warning is then generated for the subject s after a positive yˆs (k ) . This prediction model proposes a novelty detection algorithm. It considers an initial segment of each signal set as a normal state for each subject, then the remaining segments of the signals are analyzed to detect whether they deviate from normal. A deviation in a signal means that the ARDS is predicted according to this signal. Individual signals decisions are then combined using data fusion algorithms, leading to a more accurate global one. In the following, the subjects that develop ARDS, with ys = +1, are called unstable, whereas non-ARDS subjects, with ys = −1, are called stable. The proposed method is summarized in the block diagram of Fig. 1.
2.3. Novelty detection algorithm Novelty detection or one-class classification is a challenging issue in machine learning. It consists of constructing a model of normality by deriving a classification boundary that separates this model from possible outliers [21]. It is applied in different fields such as fault detection [22], medical applications [23,24], process control [25,26] and others. Wide reviews on novelty detection are presented in Refs. [27,28]. This paper proposes a subject-based approach that uses novelty detection. It identifies for each subject a normal state according to a first segment of its recorded signals and then detects in real-time the divergence from this normal state in the following recorded data. The study is conducted for each signal i for every monitored subject s. Let k′ < k be the considered duration of the first segment selected from each signal. k′ could be taken equal to 6, 12 or even 24 h, that is, 360, 720, or 1440 min respectively, or it can be determined by the medical team. An optimization approach is proposed in the following to set its value. Hence, for the signal i of a subject s, xs,i (1:k′)=(xs,i (1), …,xs,i (k′)), called first segment, is used to define its normal state; then a sliding window of the same length k′ is considered for each segment of new recorded data to detect its divergence from the first segment. In realtime, xs,i (k−k′+1:k), called latter segment, is then used for divergence study. A first solution for defining the normal state consists of finding the distribution of xs,i (1:k′), and then checking whether xs,i (k−k′+1:k) follows the same distribution or not. However, this solution is worthless due to its overfitting and thus oversensitivity. Alternatively, we propose a distance-based algorithm that consists of comparing the number of outliers of the latter segment to the one of the first segment. The recorded data is called an outlier if it lies far away from the mean of the normal data. The outliers are identified according to the two following definitions.
2.2. Materials The datasets considered in the study are selected from the Physio bank's Multi-parameter Intelligent Monitoring of Intensive Care-II (MIMIC II) database [18–20]. These datasets are needed in the fusion phase of the algorithm and also for evaluating the proposed approach. The MIMIC II directory contains two types of databases, namely, the clinical and the waveform databases. Waveform database contains physiological measurements recorded continuously using bedside monitors with numeric time series. However, clinical database includes detailed clinical information collected irregularly such as laboratory results, ventilator settings, medications, etc. According to the Berlin definition, ARDS is detected if the PaO2/ FiO2 ratio, also called P/F ratio, decreases to less than 200, by 192
Computers in Biology and Medicine 102 (2018) 191–199
A. Taoum et al.
Fig. 1. The general block diagram of the real-time prediction model for a subject s using their signals.
signal i if the number of the counted outliers is higher than 1% of the length k′ of the latter segment, and an individual decision us,i(k) = +1 is released; otherwise us,i(k) = −1. This algorithm works well when data is normally distributed. However, in most real applications, it is unknown whether data follows a predefined distribution. Therefore, a more general definition for the distance-based model is proposed in the following.
Table 1 Demographics of the population in study. Data are presented as mean ± standard deviation (range). Records length are in minutes.
Sex M/F Age Records length
Stable subjects (n = 14)
ARDS subjects (n = 26)
8/6 64.42 ± 14.7 (44–90) 4289.28 ± 1988
16/10 72 ± 13.32 (44–90) 5099.23 ± 3781
Definition 2. Considering a distance D, each point lying beyond D from the normal data is called a D-outlier. Let f(.,.) be the function that computes the distance between any data point to a set of data. By applying this to our problem, a point xs,i(ℓ) is a D-outlier if its distance f (xs,i(ℓ),xs,i(1:k′)) to the set xs,i(1:k′) is higher than D. Details on the definition of D are given later on. In this paper, the purpose is to detect the deviation of segment xs,i(k−k′+1:k) from xs,i(1:k′). The Normalized Euclidean distance is used since it is the best distance function, among others, suited to this application [30]. It is defined as follows
Definition 1. For normally distributed data, the proportion of data lying three or more standard deviations from their mean is equal to 1%. These data are called outliers. For each signal i, the mean μs,i and the standard deviation σs,i of its first segment are computed as follows,
μs, i =
1 k′
ℓ = k′
∑ ℓ=1
xs, i (ℓ), σs, i =
1 k′
ℓ = k′
∑ ℓ=1
(xs, i (ℓ) − μs, i )2 .
(1)
Then the outliers are detected in the latter segment by counting the number of points lying beyond 3σs,i from μs,i, as presented in our previous work [29]. A subject s is considered unstable according to the
f (xs, i (ℓ), xs, i (1: k ′)) = 193
|xs, i (ℓ) − μs, i | σs, i
,
(2)
Computers in Biology and Medicine 102 (2018) 191–199
A. Taoum et al.
where μs,i and σs,i are the mean and the standard deviation of the first segment respectively, computed as shown in (1). Let (xs, i (ℓ)) be an indicator yielding 1 if xs,i(ℓ) is a D-outlier and 0 otherwise, that is,
1, if f (xs, i (ℓ), xs, i (1: k ′)) ≥ D ; (xs, i (ℓ)) = ⎧ ⎨ ⎩ 0, otherwise.
Spi =
Nb of correctly identified non−ARDS subjects for signal i . Total number of non − ARDS subjects
These performance indexes are then used to determine a weight noted ai, i = 1, …,4, that will be used jointly with the decision of each signal. Hence, the function ψ(us(k)) is defined by the following equation,
(3) ′
Hence, the D-outliers within the latter segment xs,i (k−k +1:k) are counted and their ratio rs,i(k) over the length of the latter segment is computed,
ψ (us (k )) =
rs, i (k ) =
k′
.
Several weighting functions based on performance indexes have been proposed in the literature. To apply such methods, increasing functions of sensitivity and specificity are proposed to compute the weights ai, such as in Refs. [32,33]. The simplest method is applied in this work, which is defined as follows:
(4)
Let ps,i be the proportion of D-outliers within the first segment of ′
∑ ℓ = k (xs, i (ℓ))
signal i for subject s. In other words, ps, i = ℓ = 1 k′ . Then, rs,i(k) is compared to ps,i and an individual decision is generated,
+ 1, if rs, i (k ) > ps, i ; us, i (k ) = ⎧ ⎨ ⎩− 1, otherwise.
(7)
i
ℓ=k
∑ ℓ = k − k′+ 1 (xs, i (ℓ))
∑ ai us,i (k ).
Sei , if us, i (k ) > 0, ai = ⎧ Sp ⎨ ⎩ i , otherwise.
(8)
Then, the final decision yˆs (k ) is obtained according to (6).
(5)
2.4.2. Linear fusion based on error rates Instead of taking the performance indexes, we propose in this paragraph to use the error rate of the individual decisions related to each signal. Let τi,+ and τi,− be, respectively, the error rates of positive and negative estimations. τi,+ is defined as the percentage of subjects identified to develop ARDS with us,i(ℓ) = +1 at any time ℓ but actually they are not; whereas τi,− is the percentage of the subjects identified as stable but who will develop ARDS. These quantities are inversely proportional to the reliability of an individual decision. The smaller the error rate of a decision, the higher its confidence or weight should be. The weights ai are then computed from the error rates, using different decreasing weighting functions h (.). The goal is to give more influence to signals performing better than others in the classification step. Let τi be a variable representing the error rates as follows:
A positive us,i(k) means that the considered subject will develop ARDS in the near future, according to its signal i.
2.4. Data fusion In order to improve the performance of the classification, it is important to combine the classifiers obtained above. Indeed, data fusion can be considered as a promising research path in machine learning and pattern recognition, since it aggregates decisions from several models in order to obtain a more accurate classification [31]. Before applying fusion algorithms, the correlation between the four signals is tested to prevent duplication of information. Indeed, if two signals are correlated then these signals provide similar information. Therefore, one of them can be excluded. Let ψ(⋅) denote the fusion function that takes as input the vector of individual decisions us(k)=(us,1(k), …,us,4(k)) of a subject s, and gives as output a value as close as possible to the true state of the subject ys that is +1 if the subject develops ARDS and −1 if it remains in a stable state. The final decision yˆs (k ) consists then of the sign of the response of ψ, as follows:
τi, +, if us, i > 0, τi = ⎧ ⎨ ⎩ τi, −, otherwise. Then the weights are defined as ai = h (τi). First, a simple function h (τi) is proposed using the error rates, as follows:
h (τi ) = 1 − τi. + 1, if ψ (us (k )) > 0, yˆs (k ) = ⎧ ⎨ otherwise. ⎩− 1,
Another weighting function is proposed based on the weighted majority voting or WMV, presented by Benediktsson [32]. It is modified to the error rate approach. Here, h (τi) performs a normalization of the error rates among all signals.
(6)
In this work, several decision fusion functions are proposed; linear functions based on performance and on error rates and a nonlinear fusion function that is based on kernel functions. These fusion functions are based on weighting rules to compute the measures of confidence or weights. In this paper, the weights are computed conditionally on each state, thus the reliability of the decision provided is taken into consideration. For all proposed fusion algorithms, a database of signal records of stable and unstable subjects is needed to compute the fusion parameters.
h (τi ) =
(1 − τi ) − τmin . τmax − τmin
(10)
τmin and τmax are the minimum and maximum error rates among positive and negative error rates for all signals i, τmin = min(min(τi, +, τi, −)) i
and τmax = max(max(τi, +, τi, −)) . Then, the fusion is performed using equation (7).
2.4.1. Linear fusion based on performance indexes Each signal of the database goes through a preliminary classification using the proposed novelty detection algorithm to test its performance for detecting the pathology. An ARDS-subject is supposed to be correctly identified according to a signal i if it has us,i(ℓ) = +1 at any time ℓ of the signal; whereas a non-ARDS subject is correctly identified if us,i(ℓ) = −1 for all times ℓ. The performance indexes consist then of the sensitivity Se and the specificity Sp indexes that are computed for each signal i separately as follows:
Sei =
(9)
i
2.4.3. Nonlinear fusion using kernel ridge regression In this section, we propose a fusion function based on the kernel ridge regression algorithm [34,35]. It consists of a non-parametric technique defining the function ψ(⋅) that assigns to each vector us(k)= (us,1(k), …,us,4(k)) a value as close as possible to the correct label ys of the subject s. The constructed database of stable and unstable subjects’ signals is used to do this. Indeed, the proposed novelty detection algorithm is run for all the signals for all the subjects and individual decisions vectors are obtained at regular time steps for each subject. For unstable subjects, the last decision vectors are selected since they are the closest to the ARDS onset; whereas for stable subjects the first decision vectors are considered since they represent at best their stability.
Number of correctly identified ARDS subjects for signal i , Total number of ARDS subjects 194
Computers in Biology and Medicine 102 (2018) 191–199
A. Taoum et al.
2.6. Selection of signals
More than one decision vector could also be selected for each subject. A learning database of size N including N decision vectors un with their corresponding labels yn, n∈{1, …,N} is then obtained. The function ψ(⋅) is estimated using the learning database by minimizing the mean quadratic error between the models outputs ψ(us(k)) and the desired outputs yn:
min ψ∈H
1 N
In the previous subsection, different data fusion functions were proposed to combine the set of the extracted time series. The aim of the fusion was to aggregate the individual decisions obtained from these time series in order to enhance the prediction's performance. In many cases, some signals may be missing for the reason of a sudden movement, the displacement of electrodes, or other logistic reasons. Moreover, it is possible that one or more signals are less discriminative than others and might compromise the prediction of ARDS. Therefore, we are interested in the fusion of different sub-combinations of the signals used in this study. All of the possible 2- and 3- signals subsets of HR, RESP, SpO2 and ABPMean are considered, thus all the possibilities of missing signals are covered. For each of these subsets, fusion functions are performed, then sensitivity and specificity are computed to indicate the capacity of each subset fusion to correctly identify ARDS from non- ARDS patients. Afterwards, the subset of signals giving higher performance is identified, thus the combination of these signals is more relevant to discriminate ARDS patients than other combinations.
N 2 , ∑ (yn − ψ (us (k )))2 + η‖ψ‖H n=1
(11)
η is a smoothness parameter that controls the tradeoff between the error and the complexity of the solution, ℋ is the functional Hilbert space and ∥⋅∥ℋ is the norm in the Hilbert space. Hence, the optimal function can be written according to the representer theorem [36] as follows: N
ψ (⋅) =
∑ αn κ (un, ⋅),
(12)
n=1
where κ is a reproducing kernel and αn are parameters to be estimated. By replacing ψ in (11) by its expression in (12), the solution α=(α1, …,αN) is then given by,
α = (K + ηNIN )−1Y ,
3. Results
(13)
where IN is the N-by-N identity matrix, K is the Gram N-by-N matrix having K (n,m) = κ(un,um) as its (n,m)-th element. Y is the vector of length N of the labels yn. Several kernel functions κ exist in the literature. In this paper, we consider the Gaussian kernel defined by:
κ (un, um) = exp ⎛ ⎝
⎜
−‖un − um ‖2 ⎞ , 2σ 2 ⎠
The proposed approach is validated using the MIMIC II database as shown in Section 2.2, leading to 40 subjects, of which 26 are unstable and 14 are stable. Using these subjects, a LOOCV procedure is considered for evaluation. In other words, 39 are selected for the training database and only one remains for evaluation. This selection is repeated for all the subjects, leading to 40 sets of data and thus an evaluation of the approach 40 times.
⎟
(14)
where σ is the kernel bandwidth controlling with the parameter η the noise tolerance and the degree of smoothness. The estimation of the values of σ and η is presented in the following subsection.
3.1. Illustration of the proposed approach First of all, the correlation between the four signals of the subjects is tested. To do this, the correlation coefficients between each pair of signals are computed and the results are presented as a matrix of correlation coefficients in Table 2. These coefficients take values in the interval [−1,1]. The closer the coefficient is to −1 or 1, the stronger the correlation between the signals is. Table 2 shows that there is a weak linear relationship between signals. Therefore, the four signals are used in this work. In order to evaluate the performance of the proposed approach, we first start with an optimization of the parameters as described in Section 2.5. Therefore, we sweep all the possible combinations of values of k′ and D and the performances Se and Sp of the method are computed for each couple of values. This procedure is done separately for each fusion technique. Table 3 presents the optimal windows length k ′o and distance Do, that correspond to the highest Youden Index in the training phase for each fusion technique. It also presents the mean and standard deviations of the performances in the training phase through the cross validation procedure as well as the performance of the test subjects. Note that KRR indicates the kernel ridge regression fusion. It is shown that the training performances barely fluctuate in the performance-, simple error- and KRR-based fusion techniques. In addition, the results show that the kernel ridge regression fusion performs better than the linear techniques over both training and testing subjects, and that the
2.5. Parameters estimation 2.5.1. Optimization of D and k′ As mentioned previously, the distance D and the length of the first segment k′ could be fixed by the user. However, in this paper, we propose to optimize their values. Consider the signals database with the 14 stable and 26 unstable subjects is divided into a training set and a testing set. Then, the training set is used to perform optimization. Let S be the total number of subjects covered by this database. To overcome the limited number of subjects and avoid overfitting, a Leave-One-Out Cross Validation (LOOCV) procedure is proposed for optimization. k′ and D are varied respectively in {60,120, …,1440} and {0.1,0.2, …,0.9}. For each couple of values of k′ and D, the signals of (S−1) subjects are used with the proposed approach to compute the performance indexes and the error rates per signal, and the remaining subject is used with these indexes for validation. This LOOCV procedure is repeated for all S subjects of the training database, leading to S successive computations. Hence, the sensitivity Se and the specificity Sp of the whole approach are estimated for each couple of values of k′ and D. The optimal k ′o and Do are obtained by maximizing the Youden index = Se + Sp−100.
Table 2 Matrix of correlation coefficients between signals.
2.5.2. Kernel data fusion parameters In order to perform the aforementioned kernel data fusion technique, it is necessary to determine the concerned parameters. The values η and σ are obtained by maximizing the Youden index using a grid search on η = 2r with r∈{−10,−9, …,0} and on different ranges of values for σ, σ = 10r ′ and σ = −1*r′ with r′∈{−5,−4, …,−1}. The LOOCV procedure is also performed and the couple of values leading to the best Youden index is selected.
HR RESP SpO2 ABPMean
195
HR
RESP
SpO2
ABPMean
– 0.24 −0.11 0.12
0.24 – −0.17 0.16
−0.11 −0.17 – −0.01
0.12 0.16 −0.01 –
Computers in Biology and Medicine 102 (2018) 191–199
A. Taoum et al.
Fig. 2. ROC curves for Novelty algorithm followed by data fusions based on performances in (a), and both Simple and WMV error rates functions in (b) and (c), respectively.
of presentation, HR is presented by 1, Resp by 2, SpO2 by 3, ABPMean by 4 and the combination of all the signals by 4D. Fig. 3 shows that the performances of the method with the linear fusion techniques highly fluctuate with the omission of one or two signals; whereas the kernelbased one is more stable with less fluctuation. This is due to the ability of kernel methods to be adjusted to their training data. Moreover, the KRR-based fusion function presents the best Youden indexes for all types of combinations, with the highest one of 65% approximately for the combination of the four signals HR-RESP-SpO2-ABPMean. For all these reasons, only the kernel-based fusion function is considered in the following illustrations.
performance-based fusion remains better than the linear fusion based on error rate in terms of Youden index. Having a specificity of 100% with kernels means that all generated alerts are worth it, without any false alarms. Note that it is also possible to increase the sensitivity of the method, at the cost of the specificity, by tuning differently the parameters. Fig. 2 illustrates subplots of the ROC curves of the proposed algorithm with each of the fusion techniques by taking k ′ = k ′o and varying D within its corresponding interval. ROC curve plots the true positive rate, known as the sensitivity against the false positive rate computed by 1−Sp. The subplots illustrate the curves for the linear fusion techniques based on the performances in (a), the simple and WMV functions of the error rates in (b) and (c) respectively. The areas under the curves (AUC) are equal to 79.11%, 75.67% and 70.98% respectively. It is shown that ROC curves in Fig. 2(a) and (b) have small error bars. Thus Youden index of performance- and simple error-based fusions vary with small fluctuations; however WMV error-based fusion highly fluctuates. Theses curves confirm the results presented in Table 3. The decision fusion based on kernel ridge regression is not illustrated in Fig. 2 because of the random shape of it curve. This is due to the ability of the kernel methods to train themselves for each couple of values of the parameters. This is an additional advantage of the kernel fusion method compared to the linear methods.
3.3. Early predictions The main objective of the proposed approach is to predict ARDS early on in order to administer the right treatment and thus avoid its occurrence. Thus, the correctly identified subjects from the combination HR-Resp-SpO2-ABPMean using the kernel ridge regression are considered in this step. Fig. 4 presents in blue a scatter plot representing the time at which ARDS is predicted against the length of recordings of the signals. As mentioned before, the signals are just considered until the actual onset of ARDS. The plot of Fig. 4 shows that all the predictions are placed below the diagonal y = x. The nearest prediction is far from the diagonal for about 1 h, thus 1 h before the actual ARDS onset. Early warnings of ARDS are then generated in a range of 1–180 h before its occurrence with an average of 39 h or approximately two days and a standard deviation of 42 h. Fig. 4 also shows that the warnings of ARDS are generated in the range of [1,40] hours from the beginning of the recordings, thus in the first two days of monitoring.
3.2. Selection of signals All possible combinations of signals are considered to test the effectiveness of the proposed approach whenever a signal is missing. Fig. 3 illustrates a comparison between the proposed fusion functions for all the possible sub-combinations of signals, starting from the combination of 4-signals, then 3 and finally 2-signals. For the simplicity
Table 3 Performances in (%) of the optimal k ′o in (hours) and Do obtained for the proposed model with each decision fusion technique. Novelty Model +
k ′o
Do
Testing
Se ± ΔSe
fusion based on Performance Simple error rate WMV error rate KRR
Training
10 10 9 11
4.4 4.4 7.2 4.6
69.23 84.62 74.87 65.57
± ± ± ±
Sp ± ΔSp 2.45 1.17 7.4 2.05
92.86 71.43 69.27 99.04
196
± ± ± ±
2.17 2.08 10.2 2.57
Youden
Se
Se
62.09 56.05 44.14 64.61
69.23 84.62 65.38 65.38
78.57 71.43 71.42 100
Computers in Biology and Medicine 102 (2018) 191–199
A. Taoum et al.
Fig. 3. A comparison of the performances, presented by Youden index, between the proposed fusion functions for the different combinations of signals. In this graph, the following simplifications are done 1: HR, 2: RESP, 3: SpO2, 4: ABPMean and 4D is the combination of all the signals.
3.4. Validation on non-invasive ventilated subjects
Table 4 Results of the proposed approach on non-invasive ventilated subjects.
Since the number of subjects included in the study is small, the approach is tested on a larger database. Non-ventilated subjects were extracted from the MIMIC II database. The selection procedure was handled similarly to that described previously in Section 2.2. It leads to 140 ARDS subjects and 135 non-ARDS subjects. Among the ARDS subjects, there are only 72 subjects that had started records before the onset of ARDS and last at least until the occurrence of ARDS. Some subjects were removed due to noise or too short signals. This lead to 50 ARDS subjects. The non-ARDS group is then reduced to have an equal number of subjects, leading to 50 non-ARDS subjects as well. Within the database of 100 subjects, a k-fold cross-validation technique is considered to set training databases and testing databases. The results are illustrated in Table 4. As shown in the table, the novelty detection approach followed with the proposed KRR fusion method showed higher performances than the linear fusion methods with a sensitivity and specificity of 83% and 62% on the training sets and 62.89% and 72.67% on the test sets.
Novelty detection +
Performance Simple error WMV error KRR
Training
Testing
Se
Sp
Acc
Se
Sp
64.26 64.75 78.77 83.03
77.06 67.33 60.62 62.12
70.66 66.04 69.7 72.48
60.4 58 54.8 62.89
72 74 71.2 72.67
Table 5 Performances in % obtained with our approach compared to previous novelty detection technique and state-of-the art methods.
Definition 1 + KRR Novelty detection + SVM Global classification with SVM Global classification with KRR The proposed approach
Se
Sp
Acc
76.92 69.23 73 38 65.38
50 71.43 43 92 100
67.5 70 62.5 57.5 77.5
3.5. Comparison to state-of-the-art methods algorithm of Definition 2 performs better. We then consider the fusion phase and propose to replace the KRR technique followed by the sign computation by SVM. In other words, one starts with the novelty detection proposed algorithm and then performs classification by SVM on the individual decisions. The results of Table 5 show that the proposed
In this section, we compare the proposed approach to other existing methods. We first consider the novelty detection technique and compare the obtained performances to that of Definition 1. The individual decisions are then combined using the KRR fusion method. Table 5 shows the performance indexes. It is obvious that the distance-based
Fig. 4. Early detection results for the combination of HR-Resp-SpO2-ABPMean using KRR fusion function. This scatter plots time of ARDS prediction versus recordings length (hours). 197
Computers in Biology and Medicine 102 (2018) 191–199
A. Taoum et al.
fusion technique based on kernel ridge regression for the fusion of the signals on both types of signals. Performances were stable in the cross validation procedure and they were similar between training and testing subjects. KRR also showed a stability in the performances whenever there are one or more missing signals. In order to validate the proposed model, we selected two types of subjects from the MIMIC II database, namely invasive and non-invasive ventilated subjects. The invasive ventilated dataset was selected based on previous studies in the literature. However, it has a very small number of subjects. Therefore, we proposed to select the non-invasive ventilated dataset. It is worth noting that all the subjects included in this study are intensive care unit patients that developed a severe complication, such as renal failure, cardiac arrest, hypertension, sepsis, pneumonia or others. For the invasive ventilated subjects, the proposed model combined with the kernel ridge regression have succeeded to correctly verify all the stable subjects despite the existing health conditions. In addition, the algorithm tested on the non-invasive ventilated subjects has demonstrated the effectiveness of the non-linear fusion method over the linear techniques. The accuracy of the model on the invasive ventilated subjects was higher than that on the non-invasive subjects. This may refer to the higher number of subjects in the noninvasive ventilated dataset. The non-invasive ventilation helps the patient to regularize their respiration in a non continuous way. This may lead to higher fluctuations in the patients’ vital signs. As an overall performance, the proposed novelty detection algorithm followed by the KRR-based fusion technique showed the best performance using the four extracted signals. Even with the omission of signals, the performance remains the highest presenting small fluctuations. This model has predicted the onset of ARDS in the first two days of recordings, thus earlier to its onset. When comparing with state-of-art methods and the previous ARDS prediction model, this novelty detection - KRR fusion model outperformed them all. The strength of this work is the use of vital sign data, that can be collected from patients at home using non-invasive techniques. In addition, the analysis in real time allows to predict the onset of ARDS in its early stages. As shown, an early warning was generated for the majority of ARDS subjects earlier to onset. Moreover, in the case of the combination of four signals, all the stable subjects were correctly identified. Finally, the model remains stable in term of performances in case of missing data.
fusion technique with a regression followed by a comparison remains better than a direct classification. We then test the effectiveness of the whole proposed model, by comparing it to existing classification methods, such as SVM or a regression method based on KRR followed by a sign computation. Thus, instead of making a decision by comparing rs,i to ps,i as in (5) and then insert fusion functions on decisions, we take rs,i−ps,i as input to the aforementioned methods. Performance indexes are also given in Table 5. Results show clearly the improvement in the performances with the newly proposed approach. The table also shows the accuracy of all methods, that is, the proportion of correctly identified subjects over all subjects independent of their state. The proposed model outperforms the existing classification models in terms of accuracies. Finally, results obtained in this study are compared to those of a previous one using other types of data. Indeed, Ennet et al. developed a prediction model using clinical variables, such as plateau pressure and ventilator settings from MIMIC II clinical database [13]. They reached a sensitivity and a specificity of 60% and 82% for univariate analysis, and their fusion gives performances of 50% and 90% for sensitivity and specificity, respectively; while higher results were obtained in this work with the 4-D combination presented by a sensitivity of 65.38% and a specificity of 100% with only non-invasive continuous physiological signals. 4. Discussion An early-warning model for predicting ARDS is proposed and developed in this paper. The prediction model uses novelty detection and weighted decision fusion algorithms. This detection is crucial in early ARDS phases to implement preventive measures and thereby reduce mortality. It is also appealing in terms of identifying non-ARDS subjects, thus preventing unnecessary and expensive health care. Several studies have been conducted to reveal the physiological changes with ARDS [37]. ARDS has been linked to different clinical features such as severe dyspnea, tachypnea and hypoxemia according to [38]. Other studies on the clinical profile of ARDS has shown that it is associated with breathlessness, hypotension, tachypnea [39] and tachycardia [40]. However, hypertension has been found in ARDS patients in Refs. [41–43]. Although, there are no clear clinical features associated with ARDS, it is evident that ARDS is linked to cardiorespiratory and cardiovascular mechanisms. Therefore, the four vital signs that are considered are the heart rate, the respiratory rate, the oxygen saturation and the mean airway blood pressure. These signals can be collected in real-time using non-invasive and unobtrusive wearable sensors. In order to set the model, one could use classical classifiers, such as Support Vector Machines [34,44]. However, such methods do not consider the subject's normal states, but generate a global unique boundary for decision. In contrast, the proposed approach is subjectbased aiming to detect in real-time by means of a sliding window for each subject whether it deviates from its initial state. It is based on a novelty detection algorithm. Several models of novelty detection perform well on different data. However, the success of such models depends on the choice of the model as well as the statistical properties of the handled data [17]. In most physiological signals, the distribution of data is unknown and do not fit any specific distribution [45]. Moreover, the data histogram varies continuously even for stable subjects. In our previous study, data were assumed to be normally distributed. In the cases that they were not, a gaussianization phase was applied to force the normality. The model proposed in this paper outperforms that previous one being a data-driven model performing in real-time. Furthermore, it is obvious that the distance-based algorithm of Definition 2 performs better than that of Definition 1, and this is due to the non-normality of the data distributions. The distance based model is associated with different decision fusion rules, linear and nonlinear fusions. The results have shown the advantage of a non-linear
5. Conclusion This paper presents a real-time model for the early warning of ARDS onset, based on novelty detection and decision fusions. The ARDS prediction model was tested on physiological records. The prediction of ARDS was evaluated by developing distance-based novelty detection and linear and nonlinear data fusion methods. ARDS was detected in the early stages of its development by considering the combination of the four signals using the kernel ridge regression. Moreover, early warnings are still possible with high performances even with the absence of one or two signals. The strength of this work is the use of vital sign data that can be collected from patients at home using non-invasive techniques. In addition, the analysis in real-time allows the prediction of the onset of ARDS in its early stages. However, this work has some limitations. The size of the data set considered is small and the subjects in the study are patients admitted to the ICU for other disorders. Thus, future work will include more analysis to improve the performance of the decision such as the extraction of different parameters from the signals and the use of probability-based functions. Conflicts of interest The authors declare no conflict of interest. 198
Computers in Biology and Medicine 102 (2018) 191–199
A. Taoum et al.
Acknowledgment
2009, IEEE, 2009, pp. 541–544. [20] M. Saeed, M. Villarroel, A.T. Reisner, G. Clifford, L.-W. Lehman, G. Moody, T. Heldt, T.H. Kyaw, B. Moody, R.G. Mark, Multiparameter intelligent monitoring in intensive care ii (mimic- ii): a public-access intensive care unit database, CCM J. 39 (5) (2011) 952–960, https://doi.org/10.1097/CCM.0b013e31820a92c6. Multiparameter. [21] M.A. Pimentel, D.A. Clifton, L. Clifton, L. Tarassenko, A review of novelty detection, Signal Process. 99 (2014) 215–249, https://doi.org/10.1016/j.sigpro.2013.12.026. [22] Q. Cheng, P.K. Varshney, J.H. Michels, C.M. Belcastro, Fault detection in dynamic systems via decision fusion, IEEE Trans. Aero. Electron. Syst. 44 (1) (2008) 227–242, https://doi.org/10.1109/TAES.2008.4517001. [23] S. Roberts, Extreme value statistics for novelty detection in biomedical data processing, IEE Proc. Sci. Meas. Technol. 147 (2000) 363–367, https://doi.org/10. 1049/ip-smt:20000841. [24] D. Clifton, L. Clifton, L. Tarassenko, P.J. Watkinson, V.S. Barber, J. Salmon, Patientspecific biomedical condition monitoring for post-operative cancer patients, 6th Int. Conf. On Condition Monitoring and Machinery Failure Prevention Technologies, 2009, pp. 424–433. [25] E.M. Knorr, R.T. Ng, V. Tucakov, Distance-based outliers : algorithms and applications, VLDB J. 8 (3–4) (2000) 237–253. [26] P. Perner, Concepts for novelty detection and handling based on a case-based reasoning process scheme, Eng. Appl. Artif. Intell. 22 (1) (2009) 86–91 https://doi. org/10.1016/j.engappai.2008.05.003. [27] M. Markou, S. Singh, Novelty detection: a review - part 1: statistical approaches, Signal Process. 83 (12) (2003) 2481–2497, https://doi.org/10.1016/j.sigpro.2003. 07.018. [28] M. Markou, S. Singh, Novelty detection: a review - part 2: neural network based approaches, Signal Process. 83 (12) (2003) 2499–2521, https://doi.org/10.1016/j. sigpro.2003.07.019. [29] A. Taoum, F. Mourad-Chehade, H. Amoud, A. Chkeir, Z. Fawal, J. Duchêne, Data fusion for predicting ards using the mimic ii physiological database, IEEE 18th Int. Conf. Healthcom, 2016, pp. 1–5, , https://doi.org/10.1109/HealthCom.2016. 7749472. [30] B.S. Duran, P.L. Odell, Cluster Analysis: a Survey vol. 100, Springer-Verlag Berlin Heidelberg, 1974, https://doi.org/10.1007/978-3-642-46309-9. [31] T. Wilk, M. Wozniak, Soft computing methods applied to combination of one-class classifiers, Neurocomputing 75 (1) (2012) 185–193, https://doi.org/10.1016/j. neucom.2011.02.023. [32] T. Meng, L. Lin, M.-L. Shyu, S.-C. Chen, Histology image classification using supervised classification and multimodal fusion, IEEE ISM, IEEE, 2010, pp. 145–152, , https://doi.org/10.1109/ISM.2010.29. [33] A. Mourao, F. Martins, J. Magalhaes, Inverse square rank fusion for multimodal search, 12th Int. Workshop CBMI, IEEE, 2014, pp. 1–6, , https://doi.org/10.1109/ CBMI.2014.6849825. [34] V. Vapnik, Statistical Learning Theory. 1998, Wiley, New York, 1998. [35] S. Mahfouz, F. Mourad-Chehade, P. Honeine, J. Farah, H. Snoussi, Target tracking using machine learning and kalman filter in wireless sensor networks, IEEE Sensor. J. 14 (10) (2014) 3715–3725. [36] B. Scholkopf, R. Herbrich, A.J. Smola, A generalized representer theorem, International Conference on Computational Learning Theory, Springer, 2001, pp. 416–426. [37] A.W. Thille, A. Esteban, P. Fernández-Segoviano, J.-M. Rodriguez, J.-A. Aramburu, P. Vargas-Errázuriz, A. Martín-Pellicer, J.A. Lorente, F. Frutos-Vivar, Chronology of histological lesions in acute respiratory distress syndrome with diffuse alveolar damage: a prospective cohort study of clinical autopsies, Lancet Respiratory Med. 1 (5) (2013) 395–401. [38] O. Gajic, O. Dabbagh, P.K. Park, A. Adesanya, S.Y. Chang, P. Hou, I. Harry Anderson, J.J. Hoth, M.E. Mikkelsen, N.T. Gentile, M.N. Gong, D. Talmor, E. Bajwa, T.R. Watkins, E. Festic, M. Yilmaz, R. Iscimen, D.A. Kaufman, A.M. Esper, R. Sadikot, I. Douglas, J. Sevransky, M. Mon behalf of the U.S. Critical Illness, I. T. G. L. I. P. S. I. (USCIITGLIPS), Early identification of patients at risk of acute lung injury: evaluation of lung injury prediction score in a multicenter cohort study, Am. J. Respir. Crit. Care Med. 183 (4) (2011) 462–470. [39] D. Chaudhury, J. Hasan, S. Paul, I. Ali, A study on clinical profile and outcome of patients with acute respiratory distress syndrome in a tertiary care hospital in north east India, Sepsis 13 (2017) 29–5. [40] P.K. Park, J.W. Cannon, W. Ye, L.H. Blackbourne, J.B. Holcomb, W. Beninati, L.M. Napolitano, Incidence, risk factors, and mortality associated with acute respiratory distress syndrome in combat casualty care, J. Trauma Acute Care Surgery 81 (5) (2016) S150–S156. [41] I.-C. Chen, J. Kuo, W.-J. Ko, H.-C. Shih, C.-D. Kuo, Increased flow resistance and decreased flow rate in patients with acute respiratory distress syndrome: the role of autonomic nervous modulation, J. Chin. Med. Assoc. 79 (1) (2016) 17–24. [42] R. Mac Sweeney, D.F. McAuley, Acute respiratory distress syndrome, Lancet 388 (10058) (2016) 2416–2430. [43] N. Patroniti, S. Isgrò, A. Zanella, Clinical management of severely hypoxemic patients, Curr. Opin. Crit. Care 17 (1) (2011) 50–56. [44] E.D. beyli, Support vector machines for detection of electrocardiographic changes in partial epileptic patients, Eng. Appl. Artif. Intell. 21 (8) (2008) 1196–1203 https://doi.org/10.1016/j.engappai.2008.03.012. [45] K.C. Chua, V. Chandran, U.R. Acharya, C.M. Lim, Application of higher order statistics/spectra in biomedical signals a review, Med. Eng. Phys. 32 (7) (2010) 679–689.
The authors would like to thank the Université de Technologie de Troyes, France and the Lebanese University, Lebanon for their funding of this research. Appendix A. Supplementary data Supplementary data to this article can be found online at https:// doi.org/10.1016/j.compbiomed.2018.09.030. References [1] National heart, lung, and blood institute acute respiratory distress syndrome (ARDS) clinical trials network, efficacy and safety of corticosteroids for persistent acute respiratory distress syndrome, NEJM 354 (16) (2006) 1671–1684, https:// doi.org/10.1056/NEJMoa051693. [2] L.Q. Rong, A. Di Franco, M. Gaudino, Acute respiratory distress syndrome after cardiac surgery, J. Theat. Drama 8 (10) (2016) E1177. [3] D. Ashbaugh, D.B. Bigelow, T. Petty, B. Levine, Acute respiratory distress in adults, Lancet 290 (7511) (1967) 319–323, https://doi.org/10.1016/S0140-6736(67) 90168-7. [4] G.R. Bernard, A. Artigas, K.L. Brigham, J. Carlet, K. Falke, L. Hudson, M. Lamy, J.R. Legall, A. Morris, R. Spragg, The american-european consensus conference on ards. definitions, mechanisms, relevant outcomes, and clinical trial coordination, AJRCCM 149 (1994) 818–824, https://doi.org/10.1164/ajrccm.149.3.7509706. [5] ARDS Definition Task Force, Acute respiratory distress syndrome: the berlin definition, Jama 307 (23) (2012) 2526–2533, https://doi.org/10.1001/jama.2012. 5669. [6] G.D. Rubenfeld, E. Caldwell, E. Peabody, J. Weaver, D.P. Martin, M. Neff, E.J. Stern, L.D. Hudson, Incidence and outcomes of acute lung injury, NEJM 353 (16) (2005) 1685–1693, https://doi.org/10.1056/NEJMoa050333. [7] W. Chen, Y.-Y. Chen, C.-F. Tsai, S.C.-C. Chen, M.-S. Lin, L.B. Ware, C.-M. Chen, Incidence and outcomes of acute respiratory distress syndrome, Medicine 94 (43) (2015) 1–7, https://doi.org/10.1097/MD.0000000000001849. [8] N. Zaglam, P. Jouvet, O. Flechelles, G. Emeriaud, F. Cheriet, Computer-aided diagnosis system for the acute respiratory distress syndrome from chest radiographs, Comput. Biol. Med. 52 (2014) 41–48 https://doi.org/10.1016/j.compbiomed.2014. 06.006. [9] G. Singh, G. Gladdy, T.T. Chandy, N. Sen, Incidence and outcome of acute lung injury and acute respiratory distress syndrome in the surgical intensive care unit, IJCCM 18 (10) (2014) 659–665, https://doi.org/10.4103/0972-5229.142175. [10] J. Xiaoming, A. Malhotra, M. Saeed, R.G. Mark, D. Talmor, Risk factors for acute respiratory distress syndrome in patients mechanically ventilated for greater than 48 hours, Chest 133 (4) (2008) 853–861, https://doi.org/10.1378/chest.07-1121. Risk. [11] S. Yu, D.C. Christiani, B.T. Thompson, E.K. Bajwa, M.N. Gong, Role of diabetes in the development of acute respiratory distress syndrome, CCM J. 41 (12) (2013) 2720–2732, https://doi.org/10.1097/CCM.0b013e318298a2eb. [12] R. Langdon, P.D. Docherty, Y.-S. Chiew, K. Mller, J.G. Chase, Use of basis functions within a non-linear autoregressive model of pulmonary mechanics, Biomed. Signal Process. Control 27 (Supplement C) (2016) 44–50 https://doi.org/10.1016/j.bspc. 2016.01.010. [13] C.M. Ennett, K. Lee, L.J. Eshelman, B. Gross, L. Nielsen, J.J. Frassica, M. Saeed, Predicting respiratory instability in the icu, 30th Int. Conf. Of the IEEE EMBS, IEEE, Vancouver, British Columbia, Canada, 2008, pp. 2848–2851, , https://doi.org/10. 1109/IEMBS.2008.4649796. [14] P. Navarrete-Navarro, M. Ruiz-Bailén, R. Rivera-Fernández, F. Guerrero-López, M.D. Pola-Gallego-de Guzmán, G. Vázquez-Mata, Acute respiratory distress syndrome in trauma patients: icu mortality and prediction factors, ICM J. 26 (11) (2000) 1624–1629, https://doi.org/10.1007/s001340000683. [15] K. Ichikado, M. Suga, H. Muranaka, Y. Gushima, H. Miyakawa, M. Tsubamoto, T. Johkoh, N. Hirata, T. Yoshinaga, Y. Kinoshita, et al., Prediction of prognosis for acute respiratory distress syndrome with thin-section ct: validation in 44 cases 1, Radiology 238 (1) (2006) 321–329. [16] H. Ravishankar, A. Saha, G. Swamy, S. Genc, An early respiratory distress detection method with markov models, 36th Int. Conf. Of the IEEE EMBS, IEEE, Chicago, Illinois, USA, 2014, pp. 3438–3441, , https://doi.org/10.1109/EMBC.2014. 6944362. [17] C. Velardo, S.A. Shah, O. Gibson, H. Rutter, A. Farmer, L. Tarassenko, Automatic generation of personalised alert thresholds for patients with copd, 2nd EUSIPCO, IEEE, Lisbon, Portugal, 2014, pp. 1990–1994. [18] A.L. Goldberger, L.A. Amaral, L. Glass, J.M. Hausdorff, P.C. Ivanov, R.G. Mark, J.E. Mietus, G.B. Moody, C.-K. Peng, H.E. Stanley, Physiobank, physiotoolkit, and physionet : components of a new research resource for complex physiologic signals, Circulation 101 (23) (2000) e215–e220, https://doi.org/10.1161/01.CIR.101.23. e215. [19] G.B. Moody, L.-w. H. Lehman, Predicting acute hypotensive episodes: the 10th annual physionet/computers in cardiology challenge, Computers in Cardiology,
199