- Email: [email protected]
Central blood oxygen saturation vs crying in preterm newborns
Biomedical Signal Processing and Control 7 (2012) 88–92
Contents lists available at ScienceDirect
Biomedical Signal Processing and Control journal homepage: www.elsevier.com/locate/bspc
Central blood oxygen saturation vs crying in preterm newborns S. Orlandi a,∗ , L. Bocchi a , G. Donzelli b , C. Manfredi a a b
Department of Electronics and Telecommunications, Università degli Studi di Firenze, Via S. Marta 3, 50139 Firenze, Italy Department of Paediatrics, Children Hospital A. Meyer, Università degli Studi di Firenze, Firenze, Italy
a r t i c l e
i n f o
Article history: Received 31 October 2010 Received in revised form 7 April 2011 Accepted 8 July 2011 Available online 31 July 2011 Keywords: Oxygen saturation Newborn Cry Automatic cry detection
a b s t r a c t Infant cry characteristics reflect the development and possibly the integrity of the central nervous system. This study evaluates the distress occurring during cry in preterm newborn infants, as related to decrease of central blood oxygenation. A recording system was developed, that allows synchronised, non-invasive monitoring of blood oxygenation and audio recordings of newborn infant’s cry. Cry episodes were identified by an automatic system allowing further analysis of the changes induced by the cry episodes on the oxygen saturation level in the central nervous system. Specifically, decrease in the oxygenation level appears during a cry episode, followed by recovery of the oxygenation after the cry episode is over. In the present work we compare a group of preterm infants with a control group of full term newborns in order to detect possible differences between the two sets of patients. Results indicate that a similar decrease in oxygenation level occurs in both groups of patients, but the recovery time after the crying episode is more stable and rapid in full term newborns than in preterm ones. This could prove useful for clinicians and nurses in the prevention of developmental diseases for this class of patients. © 2011 Elsevier Ltd. All rights reserved.
1. Introduction Infant monitoring is a common procedure in the clinical practice in neonatal critical care units. Specifically, preterm and/or low-birth-weight infants require monitoring because they present respiratory problems which may vary from insufficient ventilation to apnoea. Moreover, very premature babies need monitoring of several vital parameters, as they may have problems with digesting food and gaining weight. Usually, preterm babies are connected to several monitors and sensors that detect changes in a baby’s condition and environment. Great efforts are thus devoted to the development of non-invasive monitoring devices and reliable data analysis techniques. Current medical and technological approaches have significantly increased the survival chances of very preterm infants, and at present there is a growing concern for their developmental and socio-emotional outcomes. Crying is the primary form of child’s communication and is therefore a good candidate for early non-invasive analysis, which may provide information on possible failures or malfunctions as its characteristics reflect the development and the integrity of the central nervous system and the vocal apparatus [1].
∗ Corresponding author. E-mail address: [email protected] (S. Orlandi). 1746-8094/$ – see front matter © 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.bspc.2011.07.003
However, crying also causes stress, in particular in preterm babies, due to their impaired auto-regulation [2–4]. Irregularities in the blood flow and pressure may in fact adversely influence the development of the child [1,4,5]. Some studies have been performed to evaluate both cerebral and peripheral blood oxygenation in the newborn by near infrared spectroscopy (NIRS) and pulsoximetry, also as related to other techniques [6,7]. Newborn cry and cerebral oxygenation flow studies are of great relevance to prevent social and behavioural disorders, such as language and neurocognitive disorders [8]. Research has been developed to study possible differences between full-term and preterm infants in their neuro-physiological maturity and the subsequent impact on their speech development [9]. Previous studies have shown that preterm infants and infants with neurological conditions have different cry characteristics mainly concerning fundamental frequency as compared with healthy full-term infants [9–11]. We have recently shown that the blood oxygenation level in preterm newborns is affected by stress caused by the effort required during crying [10,12]. These studies indicate that the distress effect of crying seems larger on central blood saturation than on peripheral saturation, hence in this paper we will consider only central blood saturation as related to cry. As shown in our preliminary work [9–11], a significant difference between the oxygenation levels before and during the crying episode was detected in preterm infants. As expected, the oxygenation level decreases during the cry episode, pointing out the possibility that the effort related to cry can act as a source of stress
S. Orlandi et al. / Biomedical Signal Processing and Control 7 (2012) 88–92
89
for the patient. Based on these promising results, we extended the study to assess the relevance of the measured changes. In the proposed approach a recording system is used for synchronous recording of audio signals and central blood oxygenation data. After, each record is automatically processed to identify cry episodes. To this aim, we have developed an automatic system based on the Otsu method [13]. The method, suitably modified to take into account the high variability of the recorded signals, is applied to the Short-Term Energy measure (STE) histogram of the audio signal. Detected cry episodes allow for further analysis of the corresponding changes in the oxygen saturation level. Specifically, a decrease in the oxygenation level during a cry episode is detected, followed by a recovery when the cry episode is over. In the present work a group of preterm infants is compared with a control group of full term newborns in order to detect possible differences between the two groups. 2. Methods 2.1. The monitoring system Monitoring was performed collecting data from two different sources: central blood saturation was measured with a NIRS device, and a microphone connected to a laptop was used to record cry emissions. Newborn cry emissions were recorded by means of a unidirectional microphone (Shure SM58), positioned at a fixed distance (25 cm) from the baby’s mouth and equipped with Tascam US144 portable audio/MIDI interface (96 kHz/24-bit recording) and stored on a multimedia laptop on a single channel audio track, with sampling rate Fs = 44 kHz and 16 bit resolution. Each recording lasts at least 15 min, in order to include several crying episodes, with a suitable amount of time both before and after each cry episode. Central blood saturation was measured by means of a NIRS device (somasensors by INVOS 5100 C Somanetics Corp.), with sampling rate of 0.6 samples/s. The NIRS signal is composed of up to four independent channels, each made up of two data, one containing the saturation of oxygen, and the other representing the quality of the signal, useful to detect possible artefacts related to patient movement or poor contact of the sensor with the patient. Specific software has been designed and implemented to allow synchronisation of the output of the two devices by means of a digital connection between the laptop and the output of the NIRS device [10,14]. The software implements simultaneous recording of the audio channel and of the NIRS signal using a RS-232 connection. Moreover, the software allows for managing the patient database, recording anamnesis data and dealing with multiple recording sessions for the same patient. The overall setup used in the experiments is described Fig. 1. All subjects were recorded in a quiet room with low background noise, isolated from the outside and with stable level of illumination, in accordance to the NIRS device requirements. Moreover, good contact between sensors and patient’s skin was carefully checked to avoid artefacts caused by sudden movements. A block diagram of the whole steps involved in data acquisition and processing is reported in Fig. 2.
Patient Data Selection
Monitoring System
Fig. 1. Experimental setup.
2.2. Automatic cry-episodes extraction In previous work, the recordings, lasting from half an hour up to several hours, were manually analysed to find out significant cry episodes [10,14]. Obviously this step is time-consuming and implies subjective judgment. Hence, a method for the automatic identification of cry-episodes was implemented. Cry episodes are automatically extracted from the whole recording applying a specific thresholding method to the Short-Term Energy measure (STE) histogram of the audio signal. This method has the advantage of requiring low computer time without any configuration parameter. The STE of each frame is defined as the squared sum of all samples belonging to that frame. Notice that the input stream was acquired in 16 bit signed format, hence the average level of the signal is 0 on each frame. STE has been measured by splitting the audio signal into 50% partially overlapping frames of fixed length (10 ms). An example of histogram of STE values obtained in a typical audio signal is shown in Fig. 3. The histogram is clearly bimodal, the large peak on the left corresponding to background noise, while the smaller one on the right side representing cry. The two thresholds LT and HT are also shown, that will be described below. First results obtained with a fixed empirical threshold proved to be unsuitable to achieve a satisfactory segmentation, due to varying environmental conditions and cry energy of the patient. Hence, the selection of the optimal threshold to identify sound (cry) and silence (background noise) signal frames was performed using the Otsu method [13], that provides the optimal threshold for the separation of a bimodal histogram minimizing the intra-class variance of the two resulting classes and maximizing the separation between the two classes. However, direct application of the Otsu method resulted in two drawbacks: first, the final part of the events, which usually has lower energy, was often classified as noise; second, in
Recorder rSO2+ Audio Data Fig. 2. Block diagram of the system.
Data Analysis
Statistical Comparison
90
S. Orlandi et al. / Biomedical Signal Processing and Control 7 (2012) 88–92
Fig. 3. Example of STE histogram. Thresholds LT and HT result from the iterative application of the Otsu algorithm.
case of energy oscillations around the threshold level, several disjoint cry episodes were detected instead of a single one. Moreover, the number of noise frames being much larger than the number of cry frames, the single threshold discards a large number of frames belonging to the cry class. This problem has been solved by a two-step application of the Otsu algorithm. A first threshold, HT, is obtained applying the Otsu algorithm to the whole histogram that singles out the high energy crying episodes. A second application of the Otsu algorithm to the histogram of the “noise” class detects a low threshold, LT, capable to distinguish low energy crying episodes and noise ones. A hysteresis thresholding is applied for the segmentation of cry episodes where HT is the upper threshold and LT the lower threshold (Fig. 3). The high energy burst occurring at the beginning of each cry episode triggers the detection that includes all frames up to the point where the energy falls below LT, thus avoiding to “crop” the episode too early due to the lowering of the sound intensity towards the frame end.
2.3. Oxygenation analysis The analysis of the oxygenation data requires the identification of a physiological range of values to be used for comparison purposes. Hence, in a first step, the basal oxygenation level of each subject was evaluated. It was defined here as the average oxygenation level during a suitable period of time during which the child was awake and calm. However, the analysis of the data points out a high variability of the basal oxygenation level, both in full term and in preterm infants. Indeed, on the whole test set we observed an average oxygenation level ranging from 65 to 85%. At the same time, the typical range of variation in the oxygenation level during each recording is approximately of the same order of magnitude. Hence it was found unpractical to analyse the absolute oxygenation level comparing it with a reference value or a range of values. Therefore, in order to assess the change in the oxygenation level during each recording, we considered the difference between the oxygenation level during and after the crying episode and the basal oxygenation of the subject. The basal oxygenation was determined for each recording using the oxygenation values measured in the time interval just before the episode. Three parameters were thus extracted from the oxygenation signal for all the selected crying episodes: the average saturation level before the episode (base level, B), the oxygenation level during the episode (C), and the saturation after a reasonable recovery time (R). Data were analysed in order to compare the oxygen saturation in basal condition (before the crying episode), in case of stress (during the episode), and after the cry episode.
The base oxygenation level B was assumed equal to the average oxygenation value over a period of 15 NIRS samples before the beginning of crying. The cry oxygenation parameter (C) was evaluated using the average oxygenation value over a time span of 18 s, approximately in the middle of the crying episode. Finally, a recovery oxygenation value (R) was evaluated, related to the capability of the patient to recover the base oxygenation level (B). R was obtained by averaging the oxygenation level measured during 90 s starting from the end of the cry episode. Given the high differences in the absolute oxygenation levels between newborns, the analysis was carried out by comparing, on each episode, the variation of the oxygenation level during (C) and after (O) the cry episode with the saturation before the episode, respectively given by: C = C − B
(1)
O = R − B
(2)
We also evaluated the recovery of the oxygenation level (R) occurred during the recovery time: R = R − C
(3)
The number of crying episodes in each recording is highly variable; hence a fixed number of episodes from each recording was selected. The parameters B, R and C were evaluated separately on all episodes related to full term newborns and to preterm ones respectively. A statistical t-test was applied to assess the statistical significance of the measured variations and of the differences between full term and preterm newborns. 3. Results The analysis was carried out on a group of 22 preterm and/or low weight infants and 28 full term infants, with a pregnancy period ranging from 23 to 42 weeks and a weight at birth between 590 g and 4250 g. They were selected by clinicians among patients at the Critical Care Unit of the Children Hospital A. Meyer and Nuovo Ospedale S. Giovanni di Dio, Firenze, Italy. Full term newborns were recorded one day after birth, while preterm newborns could be recorded only about 20–30 days after birth, due to their long staying in the incubator. The data set includes 50 recordings, made up of a variable number of cry episodes. 3.1. Automatic cry-episodes evaluation The assessment of the performance of the automatic cry selection algorithm was tested using a set of 12 recordings lasting 15 min each, where cry episodes were manually labelled. To test the method three different crying episodes, of comparable length, were manually selected from each recording with a convenient period of rest (patient either sleeping or calm) both before and after the episode. The automated cry extraction algorithm was used to mark the starting and ending points of each cry episode, allowing for the selection of 150 episodes used for further analysis. We defined a correctly detected cry episode as an episode overlapping for at least 50% with a manually labelled one. An episode that does not overlap a manually identified one is defined as a false positive, while a manually labelled episode that was not detected is defined as a false negative. Results show that the double threshold algorithm provides a fairly high specificity for the identification of the “sound” (85.77%) and the “silence” (93.87%), episodes showing a clear improvement over the single threshold algorithm, which gave a lower specificity for the intervals of silence (66.33%).
S. Orlandi et al. / Biomedical Signal Processing and Control 7 (2012) 88–92
Fig. 4. Plot of the oxygenation level in a sample signal (full term newborn) and evaluation of the oxygenation parameters. Bold segments show the average saturation level before the episode (base level, B), the oxygenation level during the episode (C), and the saturation after a reasonable recovery time (R). Table 1 Mean and standard deviation of the parameter variations for full term and preterm subjects.
Full-term Preterm
C [%rSO2 ]
R [%rSO2 ]
O [%rSO2 ]
−5.32 ± 4.54 −2,82 ± 4.19
4.00 ± 2.84 0.62 ± 4.50
−1.32 ± 3.47 −2.20 ± 3.22
3.2. Oxygenation evaluation A sample extracted from the data set, relative to a full term newborn, is reported in Fig. 4. The NIRS track is shown in the figure, together with the part of the recording used for the evaluation of the parameters B, C, and R. A graphical representation of C, R and O is also shown. The behaviour shown in the figure commonly occurs in full term newborns: during the cry episode, there is an evident decrease in the saturation level, which is promptly recovered (after about 30 s) when the crying episode is over. Results for the preterm newborns were similar, but with some important differences. As it can be observed in Table 1, the average oxygenation decrease during the cry episode (C) is actually smaller in preterm newborn than in full term subjects, but the recovery after the crying episode (R) is also much reduced. Indeed, the parameter O which measures the decrease in the oxygenation level after the cry episode with respect to the baseline oxygenation before the episode indicates that the recovering capabilities in preterm newborns are noticeably lower than in full term subjects. We analysed the statistical significance of the differences between the values before, during, and after the cry episode using a paired t-test analysis. Results, summarized in Table 2, indicate there is a highly significant difference (p < < 0.01) in the oxygenation level before and during the cry episode (C), both in the full term and in the preterm group. A quite different behaviour is obtained comparing the values measured during the cry episode and after the recovery time (R): in the full term group, the ttest indicates a highly significant statistical difference, while in the preterm group the increase is less pronounced, and is not statistically significant. The same result is confirmed by the comparison between the saturation measured before the cry episode and after the recovery time (O). This difference is highly significant in the preterm group, pointing out that the oxygenation is considTable 2 T-test results of oxygenation levels between pre-term and full-term newborns. Group
PreCry–Cry (C)
Preterm Full-term
0.01 0.01
Cry–Recovery (R) 0.3775 0.01
PreCry–Recovery (O) 0.00018 0.0140
91
Fig. 5. Comparison between the oxygenation level of one preterm and one full term newborn during a cry episode. The full term newborn recovers the baseline oxygenation level in less than 30 s while the preterm newborn needs a much longer time for complete recovering.
erably lower than the one measured before the crying episode, while in the full term group the difference is only marginally significant (0.01 < p < 0.05), suggesting that oxygenation has been recovered, although not completely. An example is shown in Fig. 5, that points out a comparable oxygenation decrease in both groups of patients, but with a recovery time after the crying episode clearly more stable and rapid in the full-term newborn than in the preterm one.
4. Discussion and conclusion The analysis of preterm infant cry may provide information about possible defects or malfunctions as it reflects the development and possibly the integrity of the central nervous system and the vocal apparatus. In this work we have studied the distress occurring during cry in full-term and in pre-term newborn infants to point out possible differences between the two groups of subjects. A new approach for the analysis of the correlation between cerebral oxygen saturation level and crying has been presented based on a completely non-invasive synchronised monitoring system of blood oxygenation and audio recording of newborn infant’s cry. A method to automatically identify cry-episodes during longterm recordings was developed, that allows for picking out cry-episodes from sound recordings, highlighting changes in the oxygen saturation level in the central nervous system. To test the proposed approach, the results obtained with the automatic system were compared with manually obtained ones, giving satisfactory results. Results show a significant decrease of the oxygenation level both in full term and in preterm infants during a cry episode. However, the two groups behave differently during the recovery time after the crying episode. Full term infants can recover almost completely the baseline oxygenation level in less than 30 s, while preterm infants need a longer period of time to achieve full recovery. Moreover, the recovery time after the crying episode was shown to be more stable in full term newborns than in preterm ones. The results could thus be useful for clinicians and nurses to prevent the development of long-lasting diseases for this class of patients. To the author’s knowledge, the analysis proposed here is the first approach to the automatic extraction of cry episodes from long lasting recording as related to central blood oxygenation. Further refinement of the procedure is going on, with the aim of providing a fully automatic cry detection tool, suitably synchronised with other relevant parameters such as central blood saturation [10], ECG, EEG, etc. as an aid to non-invasive monitoring in children hospital intensive care units.
92
S. Orlandi et al. / Biomedical Signal Processing and Control 7 (2012) 88–92
Acknowledgements The authors greatly acknowledge COST Action 2103-Advances in Voice Function Assessment and Ente Cassa di Risparmio di Firenze, LIAB Project 2009, for their contribution to this project. References [1] B.H. Friis, Perinatal brain injury and cerebral blood flow in newborn infant , Acta Paediatrica Scandinavica 74 (1985) 323–331. [2] O. Pryds, A.D. Edwards, Cerebral blood flow in the newborn infant , Archives of Disease in Childhood: Foetal and Neonatal Edition 74 (1) (1996) 63–69. [3] H.C. Lou, N.A. Lassen, B. Frii-Hansen, Impaired autoregulation of cerebral blood flow in the distressed new born infant , Journal of Paediatrics 94 (1979) 118–121. [4] G. Greisen, Cerebral blood flow preterm infant during the first week of life , Acta Paediatrica Scandinavica 75 (1986) 43–51. [5] M. Van De Bor, F.J. Walther, Cerebral blood flow velocity regulation in preterm infant , Biology of the Neonate 59 (1991) 329–335. [6] D.T. Delpy, M.C. Cope, E.B. Cady, J.S. Wyatt, P.A. Hamilton, P.L. Hope, S. Wray, E.O. Reynolds, Cerebral monitoring in newborn infants by magnetic resonance and near infrared spectroscopy , Scandinavian Journal of Clinical Laboratory Investigation 188 (1987) 9–17.
[7] V.M. Miall-Allen, L.S. de Vries, A.G. Whitelaw, Mean arterial blood pressure and neonatal cerebral lesion , Archives of Disease Childhood 62 (1987) 1068– 1069. [8] P.S. Zeskind, Impact of the cry of the infant at risk on psychosocial development, in: R.E. Tremblay, R.G. Barr, R.D.e.V. Peters (Eds.), Encyclopedia on Early Childhood Development, Montreal, Quebec, Centre of Excellence for Early Childhood Development, 2005, pp. 1–7. [9] A.M. Goberman, M.P. Robb, Acoustic examination of preterm and full-term infant cries-the long-time average spectrum , Journal of Speech Language and Hearing Research 42 (1999) 850–886. [10] C. Manfredi, L. Bocchi, S. Orlandi, M. Calisti, L. Spaccaterra, G.P. Donzelli, Non-invasive distress evaluation in preterm newborn infants , in: Proc. 30th IEEE EMBS Annual Int. Conf., Vancouver, British Columbia, Canada (CD-ROM), August 20–24, 2008, 2008. [11] C. Manfredi, L. Bocchi, S. Orlandi, L. Spaccaterra, G.P. Donzelli, High-resolution cry analysis in preterm newborn infants , Medical Engineering and Physics 31 (5) (2009) 528–532. [12] S. Orlandi, L. Bocchi, M. Calisti, G. Donzelli, C. Manfredi, Recovery of oxygen saturation level in newborns , in: Proc. 6th Int. Workshop MAVEBA, Firenze, Italy, December 14–16, 2009, pp. 15–18. [13] N. Otsu, A threshold selection method from gray-level histograms , IEEE Transactions on Systems Man and Cybernetics 9 (1979) 62–66. [14] L. Bocchi, L. Spaccaterra, F. Favilli, L. Favilli, E. Atrei, C. Manfredi, G.P. Donzelli, Monitoring of preterm infants during crying episodes , 11th Mediterranean Conference on Medical and Biomedical Engineering and Computing 16 (2007) 449–452.
Contents lists available at ScienceDirect
Biomedical Signal Processing and Control journal homepage: www.elsevier.com/locate/bspc
Central blood oxygen saturation vs crying in preterm newborns S. Orlandi a,∗ , L. Bocchi a , G. Donzelli b , C. Manfredi a a b
Department of Electronics and Telecommunications, Università degli Studi di Firenze, Via S. Marta 3, 50139 Firenze, Italy Department of Paediatrics, Children Hospital A. Meyer, Università degli Studi di Firenze, Firenze, Italy
a r t i c l e
i n f o
Article history: Received 31 October 2010 Received in revised form 7 April 2011 Accepted 8 July 2011 Available online 31 July 2011 Keywords: Oxygen saturation Newborn Cry Automatic cry detection
a b s t r a c t Infant cry characteristics reflect the development and possibly the integrity of the central nervous system. This study evaluates the distress occurring during cry in preterm newborn infants, as related to decrease of central blood oxygenation. A recording system was developed, that allows synchronised, non-invasive monitoring of blood oxygenation and audio recordings of newborn infant’s cry. Cry episodes were identified by an automatic system allowing further analysis of the changes induced by the cry episodes on the oxygen saturation level in the central nervous system. Specifically, decrease in the oxygenation level appears during a cry episode, followed by recovery of the oxygenation after the cry episode is over. In the present work we compare a group of preterm infants with a control group of full term newborns in order to detect possible differences between the two sets of patients. Results indicate that a similar decrease in oxygenation level occurs in both groups of patients, but the recovery time after the crying episode is more stable and rapid in full term newborns than in preterm ones. This could prove useful for clinicians and nurses in the prevention of developmental diseases for this class of patients. © 2011 Elsevier Ltd. All rights reserved.
1. Introduction Infant monitoring is a common procedure in the clinical practice in neonatal critical care units. Specifically, preterm and/or low-birth-weight infants require monitoring because they present respiratory problems which may vary from insufficient ventilation to apnoea. Moreover, very premature babies need monitoring of several vital parameters, as they may have problems with digesting food and gaining weight. Usually, preterm babies are connected to several monitors and sensors that detect changes in a baby’s condition and environment. Great efforts are thus devoted to the development of non-invasive monitoring devices and reliable data analysis techniques. Current medical and technological approaches have significantly increased the survival chances of very preterm infants, and at present there is a growing concern for their developmental and socio-emotional outcomes. Crying is the primary form of child’s communication and is therefore a good candidate for early non-invasive analysis, which may provide information on possible failures or malfunctions as its characteristics reflect the development and the integrity of the central nervous system and the vocal apparatus [1].
∗ Corresponding author. E-mail address: [email protected] (S. Orlandi). 1746-8094/$ – see front matter © 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.bspc.2011.07.003
However, crying also causes stress, in particular in preterm babies, due to their impaired auto-regulation [2–4]. Irregularities in the blood flow and pressure may in fact adversely influence the development of the child [1,4,5]. Some studies have been performed to evaluate both cerebral and peripheral blood oxygenation in the newborn by near infrared spectroscopy (NIRS) and pulsoximetry, also as related to other techniques [6,7]. Newborn cry and cerebral oxygenation flow studies are of great relevance to prevent social and behavioural disorders, such as language and neurocognitive disorders [8]. Research has been developed to study possible differences between full-term and preterm infants in their neuro-physiological maturity and the subsequent impact on their speech development [9]. Previous studies have shown that preterm infants and infants with neurological conditions have different cry characteristics mainly concerning fundamental frequency as compared with healthy full-term infants [9–11]. We have recently shown that the blood oxygenation level in preterm newborns is affected by stress caused by the effort required during crying [10,12]. These studies indicate that the distress effect of crying seems larger on central blood saturation than on peripheral saturation, hence in this paper we will consider only central blood saturation as related to cry. As shown in our preliminary work [9–11], a significant difference between the oxygenation levels before and during the crying episode was detected in preterm infants. As expected, the oxygenation level decreases during the cry episode, pointing out the possibility that the effort related to cry can act as a source of stress
S. Orlandi et al. / Biomedical Signal Processing and Control 7 (2012) 88–92
89
for the patient. Based on these promising results, we extended the study to assess the relevance of the measured changes. In the proposed approach a recording system is used for synchronous recording of audio signals and central blood oxygenation data. After, each record is automatically processed to identify cry episodes. To this aim, we have developed an automatic system based on the Otsu method [13]. The method, suitably modified to take into account the high variability of the recorded signals, is applied to the Short-Term Energy measure (STE) histogram of the audio signal. Detected cry episodes allow for further analysis of the corresponding changes in the oxygen saturation level. Specifically, a decrease in the oxygenation level during a cry episode is detected, followed by a recovery when the cry episode is over. In the present work a group of preterm infants is compared with a control group of full term newborns in order to detect possible differences between the two groups. 2. Methods 2.1. The monitoring system Monitoring was performed collecting data from two different sources: central blood saturation was measured with a NIRS device, and a microphone connected to a laptop was used to record cry emissions. Newborn cry emissions were recorded by means of a unidirectional microphone (Shure SM58), positioned at a fixed distance (25 cm) from the baby’s mouth and equipped with Tascam US144 portable audio/MIDI interface (96 kHz/24-bit recording) and stored on a multimedia laptop on a single channel audio track, with sampling rate Fs = 44 kHz and 16 bit resolution. Each recording lasts at least 15 min, in order to include several crying episodes, with a suitable amount of time both before and after each cry episode. Central blood saturation was measured by means of a NIRS device (somasensors by INVOS 5100 C Somanetics Corp.), with sampling rate of 0.6 samples/s. The NIRS signal is composed of up to four independent channels, each made up of two data, one containing the saturation of oxygen, and the other representing the quality of the signal, useful to detect possible artefacts related to patient movement or poor contact of the sensor with the patient. Specific software has been designed and implemented to allow synchronisation of the output of the two devices by means of a digital connection between the laptop and the output of the NIRS device [10,14]. The software implements simultaneous recording of the audio channel and of the NIRS signal using a RS-232 connection. Moreover, the software allows for managing the patient database, recording anamnesis data and dealing with multiple recording sessions for the same patient. The overall setup used in the experiments is described Fig. 1. All subjects were recorded in a quiet room with low background noise, isolated from the outside and with stable level of illumination, in accordance to the NIRS device requirements. Moreover, good contact between sensors and patient’s skin was carefully checked to avoid artefacts caused by sudden movements. A block diagram of the whole steps involved in data acquisition and processing is reported in Fig. 2.
Patient Data Selection
Monitoring System
Fig. 1. Experimental setup.
2.2. Automatic cry-episodes extraction In previous work, the recordings, lasting from half an hour up to several hours, were manually analysed to find out significant cry episodes [10,14]. Obviously this step is time-consuming and implies subjective judgment. Hence, a method for the automatic identification of cry-episodes was implemented. Cry episodes are automatically extracted from the whole recording applying a specific thresholding method to the Short-Term Energy measure (STE) histogram of the audio signal. This method has the advantage of requiring low computer time without any configuration parameter. The STE of each frame is defined as the squared sum of all samples belonging to that frame. Notice that the input stream was acquired in 16 bit signed format, hence the average level of the signal is 0 on each frame. STE has been measured by splitting the audio signal into 50% partially overlapping frames of fixed length (10 ms). An example of histogram of STE values obtained in a typical audio signal is shown in Fig. 3. The histogram is clearly bimodal, the large peak on the left corresponding to background noise, while the smaller one on the right side representing cry. The two thresholds LT and HT are also shown, that will be described below. First results obtained with a fixed empirical threshold proved to be unsuitable to achieve a satisfactory segmentation, due to varying environmental conditions and cry energy of the patient. Hence, the selection of the optimal threshold to identify sound (cry) and silence (background noise) signal frames was performed using the Otsu method [13], that provides the optimal threshold for the separation of a bimodal histogram minimizing the intra-class variance of the two resulting classes and maximizing the separation between the two classes. However, direct application of the Otsu method resulted in two drawbacks: first, the final part of the events, which usually has lower energy, was often classified as noise; second, in
Recorder rSO2+ Audio Data Fig. 2. Block diagram of the system.
Data Analysis
Statistical Comparison
90
S. Orlandi et al. / Biomedical Signal Processing and Control 7 (2012) 88–92
Fig. 3. Example of STE histogram. Thresholds LT and HT result from the iterative application of the Otsu algorithm.
case of energy oscillations around the threshold level, several disjoint cry episodes were detected instead of a single one. Moreover, the number of noise frames being much larger than the number of cry frames, the single threshold discards a large number of frames belonging to the cry class. This problem has been solved by a two-step application of the Otsu algorithm. A first threshold, HT, is obtained applying the Otsu algorithm to the whole histogram that singles out the high energy crying episodes. A second application of the Otsu algorithm to the histogram of the “noise” class detects a low threshold, LT, capable to distinguish low energy crying episodes and noise ones. A hysteresis thresholding is applied for the segmentation of cry episodes where HT is the upper threshold and LT the lower threshold (Fig. 3). The high energy burst occurring at the beginning of each cry episode triggers the detection that includes all frames up to the point where the energy falls below LT, thus avoiding to “crop” the episode too early due to the lowering of the sound intensity towards the frame end.
2.3. Oxygenation analysis The analysis of the oxygenation data requires the identification of a physiological range of values to be used for comparison purposes. Hence, in a first step, the basal oxygenation level of each subject was evaluated. It was defined here as the average oxygenation level during a suitable period of time during which the child was awake and calm. However, the analysis of the data points out a high variability of the basal oxygenation level, both in full term and in preterm infants. Indeed, on the whole test set we observed an average oxygenation level ranging from 65 to 85%. At the same time, the typical range of variation in the oxygenation level during each recording is approximately of the same order of magnitude. Hence it was found unpractical to analyse the absolute oxygenation level comparing it with a reference value or a range of values. Therefore, in order to assess the change in the oxygenation level during each recording, we considered the difference between the oxygenation level during and after the crying episode and the basal oxygenation of the subject. The basal oxygenation was determined for each recording using the oxygenation values measured in the time interval just before the episode. Three parameters were thus extracted from the oxygenation signal for all the selected crying episodes: the average saturation level before the episode (base level, B), the oxygenation level during the episode (C), and the saturation after a reasonable recovery time (R). Data were analysed in order to compare the oxygen saturation in basal condition (before the crying episode), in case of stress (during the episode), and after the cry episode.
The base oxygenation level B was assumed equal to the average oxygenation value over a period of 15 NIRS samples before the beginning of crying. The cry oxygenation parameter (C) was evaluated using the average oxygenation value over a time span of 18 s, approximately in the middle of the crying episode. Finally, a recovery oxygenation value (R) was evaluated, related to the capability of the patient to recover the base oxygenation level (B). R was obtained by averaging the oxygenation level measured during 90 s starting from the end of the cry episode. Given the high differences in the absolute oxygenation levels between newborns, the analysis was carried out by comparing, on each episode, the variation of the oxygenation level during (C) and after (O) the cry episode with the saturation before the episode, respectively given by: C = C − B
(1)
O = R − B
(2)
We also evaluated the recovery of the oxygenation level (R) occurred during the recovery time: R = R − C
(3)
The number of crying episodes in each recording is highly variable; hence a fixed number of episodes from each recording was selected. The parameters B, R and C were evaluated separately on all episodes related to full term newborns and to preterm ones respectively. A statistical t-test was applied to assess the statistical significance of the measured variations and of the differences between full term and preterm newborns. 3. Results The analysis was carried out on a group of 22 preterm and/or low weight infants and 28 full term infants, with a pregnancy period ranging from 23 to 42 weeks and a weight at birth between 590 g and 4250 g. They were selected by clinicians among patients at the Critical Care Unit of the Children Hospital A. Meyer and Nuovo Ospedale S. Giovanni di Dio, Firenze, Italy. Full term newborns were recorded one day after birth, while preterm newborns could be recorded only about 20–30 days after birth, due to their long staying in the incubator. The data set includes 50 recordings, made up of a variable number of cry episodes. 3.1. Automatic cry-episodes evaluation The assessment of the performance of the automatic cry selection algorithm was tested using a set of 12 recordings lasting 15 min each, where cry episodes were manually labelled. To test the method three different crying episodes, of comparable length, were manually selected from each recording with a convenient period of rest (patient either sleeping or calm) both before and after the episode. The automated cry extraction algorithm was used to mark the starting and ending points of each cry episode, allowing for the selection of 150 episodes used for further analysis. We defined a correctly detected cry episode as an episode overlapping for at least 50% with a manually labelled one. An episode that does not overlap a manually identified one is defined as a false positive, while a manually labelled episode that was not detected is defined as a false negative. Results show that the double threshold algorithm provides a fairly high specificity for the identification of the “sound” (85.77%) and the “silence” (93.87%), episodes showing a clear improvement over the single threshold algorithm, which gave a lower specificity for the intervals of silence (66.33%).
S. Orlandi et al. / Biomedical Signal Processing and Control 7 (2012) 88–92
Fig. 4. Plot of the oxygenation level in a sample signal (full term newborn) and evaluation of the oxygenation parameters. Bold segments show the average saturation level before the episode (base level, B), the oxygenation level during the episode (C), and the saturation after a reasonable recovery time (R). Table 1 Mean and standard deviation of the parameter variations for full term and preterm subjects.
Full-term Preterm
C [%rSO2 ]
R [%rSO2 ]
O [%rSO2 ]
−5.32 ± 4.54 −2,82 ± 4.19
4.00 ± 2.84 0.62 ± 4.50
−1.32 ± 3.47 −2.20 ± 3.22
3.2. Oxygenation evaluation A sample extracted from the data set, relative to a full term newborn, is reported in Fig. 4. The NIRS track is shown in the figure, together with the part of the recording used for the evaluation of the parameters B, C, and R. A graphical representation of C, R and O is also shown. The behaviour shown in the figure commonly occurs in full term newborns: during the cry episode, there is an evident decrease in the saturation level, which is promptly recovered (after about 30 s) when the crying episode is over. Results for the preterm newborns were similar, but with some important differences. As it can be observed in Table 1, the average oxygenation decrease during the cry episode (C) is actually smaller in preterm newborn than in full term subjects, but the recovery after the crying episode (R) is also much reduced. Indeed, the parameter O which measures the decrease in the oxygenation level after the cry episode with respect to the baseline oxygenation before the episode indicates that the recovering capabilities in preterm newborns are noticeably lower than in full term subjects. We analysed the statistical significance of the differences between the values before, during, and after the cry episode using a paired t-test analysis. Results, summarized in Table 2, indicate there is a highly significant difference (p < < 0.01) in the oxygenation level before and during the cry episode (C), both in the full term and in the preterm group. A quite different behaviour is obtained comparing the values measured during the cry episode and after the recovery time (R): in the full term group, the ttest indicates a highly significant statistical difference, while in the preterm group the increase is less pronounced, and is not statistically significant. The same result is confirmed by the comparison between the saturation measured before the cry episode and after the recovery time (O). This difference is highly significant in the preterm group, pointing out that the oxygenation is considTable 2 T-test results of oxygenation levels between pre-term and full-term newborns. Group
PreCry–Cry (C)
Preterm Full-term
0.01 0.01
Cry–Recovery (R) 0.3775 0.01
PreCry–Recovery (O) 0.00018 0.0140
91
Fig. 5. Comparison between the oxygenation level of one preterm and one full term newborn during a cry episode. The full term newborn recovers the baseline oxygenation level in less than 30 s while the preterm newborn needs a much longer time for complete recovering.
erably lower than the one measured before the crying episode, while in the full term group the difference is only marginally significant (0.01 < p < 0.05), suggesting that oxygenation has been recovered, although not completely. An example is shown in Fig. 5, that points out a comparable oxygenation decrease in both groups of patients, but with a recovery time after the crying episode clearly more stable and rapid in the full-term newborn than in the preterm one.
4. Discussion and conclusion The analysis of preterm infant cry may provide information about possible defects or malfunctions as it reflects the development and possibly the integrity of the central nervous system and the vocal apparatus. In this work we have studied the distress occurring during cry in full-term and in pre-term newborn infants to point out possible differences between the two groups of subjects. A new approach for the analysis of the correlation between cerebral oxygen saturation level and crying has been presented based on a completely non-invasive synchronised monitoring system of blood oxygenation and audio recording of newborn infant’s cry. A method to automatically identify cry-episodes during longterm recordings was developed, that allows for picking out cry-episodes from sound recordings, highlighting changes in the oxygen saturation level in the central nervous system. To test the proposed approach, the results obtained with the automatic system were compared with manually obtained ones, giving satisfactory results. Results show a significant decrease of the oxygenation level both in full term and in preterm infants during a cry episode. However, the two groups behave differently during the recovery time after the crying episode. Full term infants can recover almost completely the baseline oxygenation level in less than 30 s, while preterm infants need a longer period of time to achieve full recovery. Moreover, the recovery time after the crying episode was shown to be more stable in full term newborns than in preterm ones. The results could thus be useful for clinicians and nurses to prevent the development of long-lasting diseases for this class of patients. To the author’s knowledge, the analysis proposed here is the first approach to the automatic extraction of cry episodes from long lasting recording as related to central blood oxygenation. Further refinement of the procedure is going on, with the aim of providing a fully automatic cry detection tool, suitably synchronised with other relevant parameters such as central blood saturation [10], ECG, EEG, etc. as an aid to non-invasive monitoring in children hospital intensive care units.
92
S. Orlandi et al. / Biomedical Signal Processing and Control 7 (2012) 88–92
Acknowledgements The authors greatly acknowledge COST Action 2103-Advances in Voice Function Assessment and Ente Cassa di Risparmio di Firenze, LIAB Project 2009, for their contribution to this project. References [1] B.H. Friis, Perinatal brain injury and cerebral blood flow in newborn infant , Acta Paediatrica Scandinavica 74 (1985) 323–331. [2] O. Pryds, A.D. Edwards, Cerebral blood flow in the newborn infant , Archives of Disease in Childhood: Foetal and Neonatal Edition 74 (1) (1996) 63–69. [3] H.C. Lou, N.A. Lassen, B. Frii-Hansen, Impaired autoregulation of cerebral blood flow in the distressed new born infant , Journal of Paediatrics 94 (1979) 118–121. [4] G. Greisen, Cerebral blood flow preterm infant during the first week of life , Acta Paediatrica Scandinavica 75 (1986) 43–51. [5] M. Van De Bor, F.J. Walther, Cerebral blood flow velocity regulation in preterm infant , Biology of the Neonate 59 (1991) 329–335. [6] D.T. Delpy, M.C. Cope, E.B. Cady, J.S. Wyatt, P.A. Hamilton, P.L. Hope, S. Wray, E.O. Reynolds, Cerebral monitoring in newborn infants by magnetic resonance and near infrared spectroscopy , Scandinavian Journal of Clinical Laboratory Investigation 188 (1987) 9–17.
[7] V.M. Miall-Allen, L.S. de Vries, A.G. Whitelaw, Mean arterial blood pressure and neonatal cerebral lesion , Archives of Disease Childhood 62 (1987) 1068– 1069. [8] P.S. Zeskind, Impact of the cry of the infant at risk on psychosocial development, in: R.E. Tremblay, R.G. Barr, R.D.e.V. Peters (Eds.), Encyclopedia on Early Childhood Development, Montreal, Quebec, Centre of Excellence for Early Childhood Development, 2005, pp. 1–7. [9] A.M. Goberman, M.P. Robb, Acoustic examination of preterm and full-term infant cries-the long-time average spectrum , Journal of Speech Language and Hearing Research 42 (1999) 850–886. [10] C. Manfredi, L. Bocchi, S. Orlandi, M. Calisti, L. Spaccaterra, G.P. Donzelli, Non-invasive distress evaluation in preterm newborn infants , in: Proc. 30th IEEE EMBS Annual Int. Conf., Vancouver, British Columbia, Canada (CD-ROM), August 20–24, 2008, 2008. [11] C. Manfredi, L. Bocchi, S. Orlandi, L. Spaccaterra, G.P. Donzelli, High-resolution cry analysis in preterm newborn infants , Medical Engineering and Physics 31 (5) (2009) 528–532. [12] S. Orlandi, L. Bocchi, M. Calisti, G. Donzelli, C. Manfredi, Recovery of oxygen saturation level in newborns , in: Proc. 6th Int. Workshop MAVEBA, Firenze, Italy, December 14–16, 2009, pp. 15–18. [13] N. Otsu, A threshold selection method from gray-level histograms , IEEE Transactions on Systems Man and Cybernetics 9 (1979) 62–66. [14] L. Bocchi, L. Spaccaterra, F. Favilli, L. Favilli, E. Atrei, C. Manfredi, G.P. Donzelli, Monitoring of preterm infants during crying episodes , 11th Mediterranean Conference on Medical and Biomedical Engineering and Computing 16 (2007) 449–452.