- Email: [email protected]
Performance of three new-generation pulse oximeters during motion and low perfusion in volunteers
Journal of Clinical Anesthesia (2012) 24, 385–391
Original Contribution
Performance of three new-generation pulse oximeters during motion and low perfusion in volunteers☆,☆☆ Nitin Shah MD (Professor of Anesthesiology)a,b,⁎, Hamsa B. Ragaswamy MD (Research Associate)a , Kavitha Govindugari MD (Research Associate)a , Laverne Estanol MS (Director)a,c a
Department of Anesthesiology, Long Beach Veterans Affairs Healthcare System, Long Beach, CA 90822, USA Loma Linda University, Loma Linda, CA 92350, USA c Human Research Protections Program, VA San Diego Healthcare System, San Diego, CA 92161, USA b
Received 24 December 2010; revised 12 October 2011; accepted 25 October 2011
Keywords: Low perfusion; Motion; Oxygen saturation; Pulse oximeter
Abstract Study Objective: To evaluate pulse oximeter performance during motion and induced low perfusion in volunteers. Design: Prospective volunteer study. Setting: Direct Observation unit. Subjects: 10 healthy adult volunteers. Interventions: Ten volunteers were monitored with three different pulse oximeters while they underwent desaturation to about 75% oxygen saturation (SpO2) and performed machine-generated (MG) and volunteer-generated (VG) hand movements with the test hand, keeping the control hand stationary. Measurements: SpO2 and pulse rate readings from the motion (test) and stationary (control) hands were recorded as well as the number of times and the duration that the oximeters connected to the test hands did not report a reading. Sensitivity, specificity, performance index for SpO2, and pulse rate (PR) were calculated for each pulse oximeter by comparing performance of the test hand with the control hand. Main Results: During both MG and VG motion, the Masimo Radical had higher SpO2 specificity (93% and 97%) than the Nellcor N-600 (67% and 77%) or the Datex-Ohmeda TruSat (83% and 82%). The Masimo Radical also had higher SpO2 sensitivity (100% and 95%) than the Nellcor N-600 (65% and 50%) or the Datex-Ohmeda TruSat (20% and 15%) during both MG and VG motion. During MG motion, the Masimo Radical had the lowest PR failure rate (0%) compared with the Nellcor N-600 (22.2%) and Datex-Ohmeda TruSat (1.3%). However, during VG motion, the Masimo Radical had the lowest SpO2 failure rate (0%) of the three devices (Nellcor N-600 16.4% and Datex-Ohmeda TruSat 1.7%). Both the Masimo Radical and the Datex-Ohmeda TruSat had lower PR failure rates (0% and 4.4%) than the Nellcor N-600 (33.9%). There were no significant differences in SpO2 or PR performance index between the three devices.
☆
Financial Support: Supported in part by Masimo Corporation, Irvine, CA, USA, which provided equipment and a grant for the study. Conflict of interest: Dr. Shah is a Speaker Board Member, Advisory Board Member, & Stock holder in Masimo Corporation, Irvine, CA, USA. ⁎ Correspondence: Nitin Shah, MD, Department of Anesthesiology, Long Beach Veterans Administration Healthcare System, 5901 E. 7th St., Long Beach, CA 90822, USA. Tel.: 562-826-8000 ext 3264; fax: 562 826-5991. E-mail address: [email protected] (N. Shah). ☆☆
0952-8180/$ – see front matter. Published by Elsevier Inc. doi:10.1016/j.jclinane.2011.10.012
386
N. Shah et al. Conclusions: The Masimo Radical had higher SpO2 sensitivity and specificity than the Nellcor N-600 and Datex-Ohmeda TruSat during conditions of motion and induced low perfusion in this volunteer study. Published by Elsevier Inc.
1. Introduction Oxygen saturation (SpO2) is the estimate of the oxygen saturation of hemoglobin. Continuous monitoring of SpO2 enables clinicians to detect hypoxemia at an early stage and intervene to prevent any dire consequences. Pulse oximeters provide immediate, continuous, and noninvasive monitoring of SpO2 without the need for calibration and frequent sensor replacement [1]. Conventional pulse oximeters have significant limitations, including high rates of false alarms and failed measurements when used on patients with low perfusion or during patient motion [2-6]. This high false-alarm rate incurs the risk of desensitizing clinical care givers to true alarms [7]. Failed measurements are another significant limitation. Earlier work showed that pulse oximeters failed in the operating room in 2.5% of 10,312 patients randomized to be monitored [8]. Pulse oximeter failure rates are even higher in critical care, intensive care, and Postanesthesia Care Unit (PACU) environments [9]. New-generation pulse oximeters are designed to overcome these limitations. The newergeneration pulse oximeters have fewer false alarms and higher accuracy than conventional pulse oximeters [10-13]. In this study, we evaluated the performance of three newgeneration pulse oximeters [Masimo Radical (V 5.0), Nellcor N-600 (V 1.1.2.0), and Datex-Ohmeda TruSat] in a cool environment to mimic reduced perfusion and during induced motion. Both machine-generated (MG) motion (standardized and reproducible) and volunteer-generated (VG) motion (more challenging for pulse oximeter sensors) were used [14]. Both hypoxic and normoxic conditions were studied.
2. Materials and methods After Institutional Review Board of the Long Beach VAHS, CA approval and informed consent, 10 healthy, ASA physical status 1 volunteers (5 women), ranging in age from 22 to 30 years (mean age 26 yrs), were studied. Earlier studies of pulse oximeter performance studied 10 healthy volunteers and had obtained statistically significant results [13,14]. All subjects were nonsmokers with no evidence of vascular or other systemic disease. Each subject was monitored with three oximeter sensors: the Masimo Radical (V 5.0; Masimo Corp., Irvine, CA, USA), the Nellcor N-600 (v1.1.2.0; Covidien, Boulder CO, USA), and the GE DatexOhmeda TruSat (GE Healthcare Technologies, Waukesha, WI, USA), one each on digits 2, 3, and 4 of the moving “test”
hand, and one each of the same make and model on the digits of the stationary “control” hand. Sensor assignments to the individual digits were rotated among the subjects. All sensors were of the disposable adhesive type to minimize sensor displacement during hand motion. The averaging time for each oximeter was set at 8 seconds and so were essentially equivalent for all devices. All devices were new, purchased for use in the study. The test hand was strapped to a motorized motion table, which produced repeatable and continuous hand motions. This motion table, which was similar to one used in previous studies [14,15], moved the hand up and down while the elbow remained fixed. It was configured so that the fingertips either tapped or rubbed on a smooth surface. The amplitude of the motion was ± 2 cm, and the frequency was either fixed or randomly varied (aperiodic) within the range of 1 to 4 Hz. Previous studies found that these motions caused any pulse oximeter to fail at least occasionally [14,15]. The peripheral perfusion of healthy, awake volunteers is clearly not that of typical surgical patients. To partially compensate for this fact, we maintained our room temperature in the 16°C - 18°C range and kept the subject's arms exposed. Skin temperature was monitored by a thermocouple on the fifth digit. This method was employed in other studies to evaluate the performance of pulse oximeters during low perfusion [13,15,16]. The oximeter sensors were applied to each subject and connected to their respective instruments. Once applied, the sensors were optically isolated from each other by a black nonconductive plastic covering. The oximeters were connected via serial data ports to a computerized data logger that recorded pulse rate (PR) and SpO2 values from all instruments and perfusion index from the Masimo pulse oximeter once per second. After recording room air control values with both hands stationary, the motion table was activated, and two minutes of data were recorded for each of two motions: 1) fingers tapping at 3 Hz or at a frequency that varied randomly between 1 and 3 Hz, and 2) fingers rubbing at the same frequencies. Once the two motions were completed and all SpO2 values returned to baseline, the sensors were moved to different test fingers and the series was repeated twice so that all three test digits were monitored with each test oximeter. In the next series of tests on each subject, a hypoxemic episode was induced during each motion period. For this purpose, subjects were outfitted with a disposable rebreathing circuit with a CO2 absorber (CO2 rebreathing system; King Systems, Noblesville, IN, USA). The rebreathing circuit removes CO2 from the exhaled air of the subject,
Three new-generation pulse oximeters leaving the O2 and other gases available for rebreathing. As the subject continues to breathe through the rebreather apparatus, the gas mixture will contain progressively less O2 and thus gradually induces hypoxia. A TOSCA sensor (PtcCO2 + Masimo Radical pulse oximeter; Radiometer America, Philadelphia PA, USA) was placed on the right ear to rapidly detect SpO2 changes during hypoxia and guide administration of 100% O2 after desaturation. A TOSCA ear oximeter monitors both SpO2 and transcutaneous CO2 values. This requires the machine to heat the tissues underneath the sensor to 42°C to allow rapid diffusion of CO2 from arterial blood through the surrounding tissues to the sensor interface. The TOSCA ear sensor has a faster reaction time for monitoring SpO2 compared with pulse oximeter finger sensors, which allowed us to minimize the time when volunteers were hypoxic. After baseline data were recorded during normoxemia, subjects were instructed to place the rebreathing circuit mouthpiece in their mouth and breathe room air through their nose and exhale through their mouth to fill a 3-liter bag. Once the bag was filled, a nose clip was placed on the subject so that they breathed the recirculated gas mixture from the 3-liter bag. Subjects continued to breathe from the bag until SpO2 reached approximately 75%, as measured by the TOSCA ear sensor. Volunteers were instructed to remove the mouthpiece if unpleasant symptoms developed at any time during rebreathing. During hypoxemia, data were recorded during two types of motion: 1) random tapping, which was accompanied by the disconnecting and reconnecting of the sensors at the start of motion, and 2) random rubbing. Once SpO2 reached 75%, volunteers were given 100% O2 until SpO2 reached 100% on all control monitors, at which time the motion was terminated. Each rapid desaturation/resaturation experiment was completed within three minutes. Continuous verbal contact was maintained with the subjects. All subjects were able to complete the protocol without difficulty. The protocol during hypoxemia included the additional step of disconnecting and reconnecting all test sensors from their respective instruments after the motion had begun. This disconnect-reconnect experiment simulated the clinical scenario of placing an oximeter sensor on a patient who is already moving when the sensor is applied. We calculated the failure rate for each device by dividing the amount of time that each instrument displayed no SpO2 value by the total time of the test. To combine measures of accuracy and reliability, we also calculated the performance index for each device by dividing the amount of time the instrument displayed a current SpO2 value that was within 7% of the simultaneous control value by the total time of the test. Performance index was a method used to evaluate pulse oximeter performance in a previous study [13]. The error margin of 7% was chosen because SpO2 changes rapidly over a wide range in these experiments. A smaller margin of error will reduce the performance index of all instruments, and a larger margin will increase it. The performance index
387 for PR accuracy was calculated in the same manner as for SpO2, but with a margin of error of 10% compared with the control value. The oximeter's ability to detect hypoxemic events is quantified by sensitivity and specificity – sensitivity being the probability of detecting hypoxemia and specificity being the probability of detecting normoxia (lack of false alarms). Formally, sensitivity is the number of true positives divided by the number of true positives plus the number of false negatives. For the purposes of this study, sensitivity was calculated by dividing the number of events (tests) in which the test device read b 90% when the control device read b 90% during hypoxia (true positives) by the number of events (tests) the test device read b 90% plus the number of times the test device read N 90% when the control device read b 90% during hypoxia (false negatives). Sensitivity was calculated from the 20 machine-generated and 20 volunteergenerated motion tests with desaturation. Conversely, specificity, which is formerly calculated by dividing the number of true negatives by the number of true negatives plus the number of false positives, for the purposes of this study was calculated by dividing the number of events that the test device reported SpO2 values N 90% when the control device also reported values N 90% during normoxia (true negatives) by the number of true negatives plus the number of false positives during the 60 machine-generated and 60 volunteer-generated motion tests. We also compared the effect of two types of motions, machine-generated and volunteer-generated, on performance of the pulse oximeters by measuring the duration of missed events, number of false alarms, failure rate, and recovery time. Missed events were calculated by counting the number of tests in which the test device reported normoxia (SpO2 N 90%) when the control device reported a desaturation event b 90% SpO2 during 20 volunteer- and 20 machine-generated motion tests. False alarms were calculated by counting the number of times the test device alarmed for a SpO2 value b 90%, during normoxia as determined by the control device, during the 60 volunteer- and 60 machine-generated motion tests. Failure rate was calculated by dividing the amount of time the test device displayed no reading by the total time of the test. Recovery time was defined as the time required for the test device to display a SpO2 value within 7% of the control value after the cessation of motion. The duration of each experimental condition when subjects were breathing room air (mechanical tapping and rubbing and voluntary tapping and rubbing) was two minutes, with each condition repeated twice so that all test digits were monitored with each test oximeter. All test periods were of the same duration so that results could be compared, except for the recovery time tests. These tests were completed within three minutes, the exact duration of which depended on how long it took for a device to recover from a failure to provide a measurement. The time to desaturate was the same for all pulse oximeters tested. However, the time to recovery from desaturation varied
388
N. Shah et al.
among the pulse oximeters. Since each subject wore sensors for all the devices at the same time, the duration of each experimental condition for each device was identical, except for the recovery time. Data from all subjects were combined. Analysis of variance (ANOVA) was performed, with Fisher's post hoc test, to compare performance index, missed events, false alarms, and failure rate results for the three oximeters. In addition, ANOVA with Fisher's post hoc test, and Chisquare analysis as appropriate, were used to compare sensitivity and specificity of the three oximeters and to compare the effect of machine-generated and volunteergenerated motions on missed events, false alarms, failure rate, and recovery time. The mean ± 1 SD was calculated for subject fingertip temperature and a Mann-Whitney Rank Sum test for nonparametric data was used to compare the perfusion index between the test and control hands. P b 0.05 was considered statistically significant for all statistical tests.
3. Results A total of 160 motion tests were performed (80 with machine-generated motion, 80 with volunteer-generated motion, 40 with desaturation, and 120 on room air) on 10 study volunteers. Missed events were counted out of 40 tests during hypoxemia (20 with machine motion and 20 with volunteer motion), and false alarms were counted out of 120 tests during normoxia (60 with machine motion and 60 with volunteer motion). The range in average temperatures for each experimental condition for all subjects was 18.9° C - 27.7° C, with an average fingertip temperature (± 1 SD) for all subjects during all conditions of 21.4° C ± 3.3° C. The median perfusion index for all subjects during all conditions was 0.95 (0.63 at
Table 1
the first quartile) in the control hand and 1.16 (0.873 at the first quartile) in the motion hand during non-motion conditions. The difference was significant (P b 0.001). The performance index of the pulse oximeters (Table 1) for SpO2 was 97.5%, 72.3%, and 83.2% for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat during machine motion and 98.5%, 73.1%, and 81.9% during volunteer motion, respectively, but the differences were not significant. Likewise for PR, the performance index of the pulse oximeters was 82.9%, 61.0%, and 78.0% during machine motion and 88.5%, 60.3%, and 73.6% during volunteer motion for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat, respectively, but differences were not significant. The Masimo Radical had failure rates of 0% during both machine and volunteer motions for both SpO2 and PR (Table 1). The SpO2 failure rate was significantly lower for the Masimo Radical than for the other two devices (P b 0.05) during volunteer-generated motion, but no significant differences were noted during the machine-generated motion tests. The PR failure rate of the Masimo Radical was significantly lower than that of the Nellcor N-600 and Datex-Ohmeda TruSat during machine-generated motion (P b 0.05), whereas during volunteer-generated motion the PR failure rates of both the Masimo Radical and DatexOhmeda TruSat were significantly lower than the Nellcor N-600 (P b 0.005). Table 2 shows that during machine-generated motion, specificity (probability of detecting normoxia during the 60 machine-generated motion tests) was 93%, 67%, and 83% for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat, respectively. During volunteer motion, specificity was 97%, 77%, and 82% for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat, respectively. During both machine- and volunteer-generated motion, the Masimo
Performance index and failure rate (FR) during machine-generated and volunteer-generated motion
Machine-generated motion (80 tests) Device
SpO2 performance index (%)
Masimo Radical (V 5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
97.5 72.3 83.2
SpO2 FR (%) 0 9.3 1.3
PR performance index (%)
PR FR (%)
82.9 61.0 78.0
0 22.2 ⁎ 1.7 ⁎
Volunteer-generated motion (80 tests) Device
SpO2 performance index (%)
SpO2 FR (%)
PR performance index (%)
PR FR (%)
Masimo Radical (V 5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
98.5 73.1 81.9
0 16.4 ⁎ 1.7 ⁎
88.5 60.3 73.6
0† 33.9 4.4 †
PR = pulse rate; performance index = duration of time the test device displayed oxygen saturation (SpO2) within 7% or PR value with 10% of the control device divided by the total time of the test; failure rate = duration of time the test device displayed no SpO2 or PR value divided by the total time of the test. ⁎ P b 0.05 vs Masimo Radical (V 5.0). † P b 0.005 vs Nellcor N-600 (V 1.1.2.0).
Three new-generation pulse oximeters Table 2
389
Oxygen saturation (SpO2) sensitivity and specificity during machine-generated and volunteer-generated motion
Machine-generated motion (20 tests for missed events/sensitivity and 60 tests for false alarms/specificity) Device
Missed events
Sensitivity (%)
False alarms
Specificity (%)
Masimo Radical (V5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
0/20 7/20 16/20
100 65 ⁎ 20 ⁎
4/60 20/60 10/60
93 67 ⁎ 83 ⁎
Device
Missed events
Sensitivity (%)
False alarms
Specificity (%)
Masimo Radical (V 5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
1/20 10/20 17/20
95 50 ⁎ 15 ⁎
2/60 14/60 11/60
97 77 ⁎ 82 ⁎
Volunteer-generated motion
Missed events = number of tests in which the test device reported normoxia (SpO2 N 90%) when the control device reported a desaturation event b 90% SpO2. ⁎ P b 0.05 vs Masimo Radical (V5.0).
Radical had higher specificity than either the Nellcor N-600 or Datex-Ohmeda TruSat (P b 0.05). Similar results were obtained for sensitivity of the pulse oximeters – the ability to detect hypoxemia. Sensitivity was 100%, 65%, and 20% for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat during the 20 machine motion tests, and 95%, 50%, and 15% during the 20 volunteer motion tests, respectively, with the Masimo Radical having significantly higher sensitivity than either the Nellcor N-600 or Datex-Ohmeda TruSat (P b 0.05). There was no significant difference in sensitivity or specificity between the Nellcor N-600 and Datex-Ohmeda TruSat. When the effects of the two types of motions on performance of the pulse oximeters were compared (Table 3), there were no significant differences in failure rate or recovery time for both SpO2 and PR. Failure rates of the pulse oximeters while recovering from motion (Table 4) for SpO2 were 1.3%, 25.0%, and 15.0% during machine motion and 3.8%, 31.3%, and 23.8% during volunteer motion for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat, respectively. The difference in SpO2 failure rates between the Masimo Radical and both the Nellcor N-600 and the DatexOhmeda TruSat was statistically significant (P b 0.05). For PR, the Masimo Radical had a significantly lower failure rate than the Nellcor N-600 during both machineand volunteer-generated motion (P b 0.05). The Masimo Radical also had a significantly lower PR failure rate
Table 3
than the Datex-Ohmeda TruSat during volunteer motion (P b 0.05), but differences were not significant during machine-generated motion. Recovery times (mean and range) from the failed events for SpO2 were 21 (13-27 sec), 14 (6-36 sec), and 42 (9-180 sec) seconds during machine motion and 17 (12-21 sec), 13 (6-24 sec), and 43 (12-288 sec) seconds during volunteer motion for the Masimo Radical, Nellcor N-600, and DatexOhmeda TruSat, respectively. The Masimo Radical had a significantly shorter recovery time than the Datex-Ohmeda TruSat during both machine-generated and volunteergenerated motion tests (P b 0.05), but the difference between the Masimo Radical and the Nellcor N-600 or the Nellcor N-600 and the Datex-Ohmeda TrueSat devices were not significant. Results for PR recovery times were similar. The Masimo Radical had a significantly shorter PR recovery time than the Datex-Ohmeda TruSat during both machinegenerated and volunteer-generated motion tests (P b 0.05), but the difference between the Masimo Radical and the Nellcor N-600 or the Nellcor N-600 and the Datex-Ohmeda TrueSat devices was not significant.
4. Discussion Our study involved volunteer subjects in a laboratory setting. Such studies allow investigators to use a more rigorous protocol while maintaining control over variables.
Machine-generated (MG) versus volunteer-generated (VG) motion: effects on pulse oximeter performance
Motion
SpO2 (%) RT (min)
PR RT (min)
SpO2 missed events (min)
PR false alarms (min)
SpO2 failure (min)
PR failure (min)
MG VG
0.44 ± 0.80 0.60 ± 1.1
0.65 ± 0.92 0.87 ± 1.1
2.5 ± 2.8 2.3 ± 2.3
3.8 ± 2.7 3.5 ± 2.7
0.35 ± 1.1 0.60 ± 1.6
0.80 ± 1.8 1.3 ± 2.3
Data are means± SD. Differences were not significant. SpO2 = oxygen saturation, RT = recovery time, PR = pulse rate.
390 Table 4
N. Shah et al. Recovery time (RT) and failure rate (FR) while recovering during machine-generated and volunteer-generated motion
Machine-generated motion (80 tests) Device
SpO2 RT [sec; means (ranges)]
SpO2 no. times fail/total
SpO2 FR
PR RT [sec; means (ranges)]
PR no. times fail/total
PR FR
Masimo Radical (V 5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
21.0 (13-27) 14.3 (6-36) 42.0 (9-180) ⁎
1/80 20/80 12/80
1.3% 25.0% ⁎ 15.0% ⁎
14.5 (6-24) 21.4 (6-39) 44.3 (12-168) ⁎
12/80 22/80 12/80
15.0% 27.5% ⁎ 15.0%
Device
SpO2 RT [sec; means (ranges)]
SpO2 no. times fail/total
SpO2 FR
PR RT [sec; means (ranges)]
PR no. times fail/total
PR FR
Masimo Radical (V5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
17.0 (12-21) 13.0 (6-24) 42.8 (12-288) ⁎
3/80 25/80 19/80
3.8% 31.3% ⁎ 23.8% ⁎
21.3 (6-48) 20.8 (3-36) 40.2 (12-270) ⁎
9/80 35/80 17/80
11.3% 43.8% ⁎ 21.3% ⁎
Volunteer-generated motion
PR = pulse rate, recovery time = time required (sec) for test device to display oxygen saturation (SpO2) value within 7%, or PR value within 10% of control device value after cessation of motion. ⁎ P b 0.05 vs Masimo Radical (V 5.0).
The major disadvantage of laboratory studies is that they may not fully replicate the characteristics of complex clinical settings such as the different types of motions exhibited by patient populations [17]. The potential of unrecognized hypoxemia, however, mandates that research in volunteers mimic actual clinical scenarios as closely as possible. Earlier studies have described the use of active motions to test pulse oximeter devices [15,18]. In our study we incorporated two types of motions, machine-generated and volunteer-generated motions. We found no significant differences among them on the performance of the pulse oximeters such as for the number of missed events, false alarms, failure rates, and recovery times. To simulate this clinical scenario, we tested the performance of pulse oximeters during motion during hypoxic conditions in the presence of reduced perfusion induced by lowering the room temperature. Other researchers have reported perfusion values in healthy subjects during room temperature conditions of about 5.0% [16]. Our subjects, therefore, had a nearly threefold decrease in perfusion index compared with what has been found in healthy subjects in room temperature conditions, indicating reduced peripheral perfusion of the digits during the testing period. This study showed that during experimental motion conditions the Masimo Radical had the highest SpO2 sensitivity and specificity during normoxia and the lowest SpO2 failure rate during hypoxemia compared with the Datex-Ohmeda TruSat and Nellcor N-600 devices. No significant differences were observed in SpO2 or PR performance index. When comparing recovery time among the pulse oximeters, mean SpO2 and PR recovery time were significantly shorter for the Masimo Radical than for the Datex-Ohmeda TruSat, but there was no difference between the Masimo Radical and the Nellcor N-600 or the Nellcor N-600 and the Datex-Ohmeda devices. Investigators have used numerous methodologies in the laboratory and the clinical environment to compare both
conventional and new-generation “motion-tolerant” pulse oximeter technologies during conditions of motion. In general, motion-tolerant technologies have had fewer data dropouts and false alarms, and better overall performance compared with conventional technologies. In studies that compared motion-tolerant technologies, some results have been less consistent, in part because of the various software versions of the pulse oximeter technologies and in part because of methodological weaknesses. Notable issues with previous studies include comparing PR with electrocardiography heart rate, comparing one pulse oximeter to other pulse oximeters to calculate true and false alarms, inconsistent sensitivity mode selection between various pulse oximeters, and the use of reusable finger clips instead of adhesive sensors. However, we found that our results were consistent with previous work using similar methods with earlier versions of pulse oximeters [13]. Our study was limited by its lack of use of arterial blood gas (ABG) measurements as the comparative standard. Comparisons with ABG measurements are required to evaluate the absolute accuracy of SpO2 values obtained from a pulse oximeter. Our study was also limited by its use of volunteers rather than hospitalized subjects, as the physiological characteristics of each group may be different. Our study also relied upon machine-generated or deliberate volunteer-generated motion; these types of motion may differ from naturally occurring motion in a clinical environment. Due to these limitations, we strongly urge clinicians evaluating various pulse oximeters to perform their own side-by-side comparisons in clinically relevant scenarios, using some form of automated data collection for objective recording and analysis. In volunteers in a cold environment during machine- and volunteer-generated motion, the Masimo Radical had higher SpO2 sensitivity and specificity than the Nellcor N-600 and Datex-Ohmeda TruSat. The Masimo Radical had lower SpO2 and PR failure rates during volunteer motion during
Three new-generation pulse oximeters both normoxia and hypoxia, but the results were less consistent during machine-generated motion. There were no significant differences in SpO2 or PR performance index between the three devices.
References [1] Poets CF, Southall DP. Noninvasive monitoring of oxygenation in infants and children: practical considerations and areas of concern. Pediatrics 1994;93:737-46. [2] Lawson D, Norley I, Korbon G, Loeb R, Ellis J. Blood flow limits and pulse oximeter signal detection. Anesthesiology 1987;67:599-603. [3] Severinghaus JW, Spellman MJ Jr. Pulse oximeter failure thresholds in hypotension and vasoconstriction. Anesthesiology 1990;73:532-7. [4] Wilson S. Conscious sedation and pulse oximetry: false alarms? Pediatr Dent 1990;12:228-32. [5] Plummer JL, Zakaria AZ, Ilsley AH, Fronsko RR, Owen H. Evaluation of the influence of movement on saturation readings from pulse oximeters. Anaesthesia 1995;50:423-6. [6] Wiklund L, Hök B, Ståhl K, Jordeby-Jönsson A. Post anesthesia monitoring revisited: incidence of true and false alarms from different monitoring devices. J Clin Anesth 1994;6:182-8. [7] Lawless ST. Crying wolf: false alarms in a pediatric intensive care unit. Crit Care Med 1994;22:981-5. [8] Moller JT, Johannessen NW, Espersen K, et al. Randomized evaluation of pulse oximetry in 20,802 patients: II. Perioperative events and postoperative complications. Anesthesiology 1993;78:445-53.
391 [9] Brown LT, Purcell GJ, Traugott FM. Hypoxaemia during postoperative recovery using continuous pulse oximetry. Anaesth Intensive Care 1990;18:509-16. [10] Lie C, Kehlet H, Rosenberg J. Comparison of the Nellcor N-200 and N-3000 pulse oximeters during simulated postoperative activities. Anaesthesia 1997;52:450-2. [11] Lutter NO, Urankar S, Kroeber S. False alarm rates of three thirdgeneration pulse oximeters in PACU, ICU, and IABP patients. Anesth Analg 2002;94(1 Suppl):S69-75. [12] Malviya S, Reynolds PI, Voepel-Lewis T, et al. False alarms and sensitivity of conventional pulse oximetry versus the Masimo SET technology in the pediatric postoperative anesthesia care unit. Anesth Analg 2000;90:1336-40. [13] Barker SJ. “Motion-resistant” pulse oximetry: a comparison of new and old models. Anesth Analg 2002;95:967-72. [14] Gehring H, Hornberger C, Matz H, Konecny E, Schmucker P. The effects of motion artifact and low perfusion on the performance of a new generation of pulse oximeters in volunteers undergoing hypoxemia. Respir Care 2002;47:48-60. [15] Barker SJ, Shah NK. The effects of motion on the performance of pulse oximeters in volunteers (revised publication). Anesthesiology 1997;86:101-8. [16] Nishiyama T. Pulse oximeters demonstrate different responses during hypothermia and changes in perfusion. Can J Anaesth 2006;53:136-8. [17] Jopling MW, Mannheimer PD, Bebout DE. Issues in the laboratory evaluation of pulse oximeter performance. Anesth Analg 2002;94(1 Suppl):S62-68. [18] Barker SJ. Standardization of the testing of pulse oximeter performance. Anesth Analg 2002;94(1 Suppl):S17-20.
Original Contribution
Performance of three new-generation pulse oximeters during motion and low perfusion in volunteers☆,☆☆ Nitin Shah MD (Professor of Anesthesiology)a,b,⁎, Hamsa B. Ragaswamy MD (Research Associate)a , Kavitha Govindugari MD (Research Associate)a , Laverne Estanol MS (Director)a,c a
Department of Anesthesiology, Long Beach Veterans Affairs Healthcare System, Long Beach, CA 90822, USA Loma Linda University, Loma Linda, CA 92350, USA c Human Research Protections Program, VA San Diego Healthcare System, San Diego, CA 92161, USA b
Received 24 December 2010; revised 12 October 2011; accepted 25 October 2011
Keywords: Low perfusion; Motion; Oxygen saturation; Pulse oximeter
Abstract Study Objective: To evaluate pulse oximeter performance during motion and induced low perfusion in volunteers. Design: Prospective volunteer study. Setting: Direct Observation unit. Subjects: 10 healthy adult volunteers. Interventions: Ten volunteers were monitored with three different pulse oximeters while they underwent desaturation to about 75% oxygen saturation (SpO2) and performed machine-generated (MG) and volunteer-generated (VG) hand movements with the test hand, keeping the control hand stationary. Measurements: SpO2 and pulse rate readings from the motion (test) and stationary (control) hands were recorded as well as the number of times and the duration that the oximeters connected to the test hands did not report a reading. Sensitivity, specificity, performance index for SpO2, and pulse rate (PR) were calculated for each pulse oximeter by comparing performance of the test hand with the control hand. Main Results: During both MG and VG motion, the Masimo Radical had higher SpO2 specificity (93% and 97%) than the Nellcor N-600 (67% and 77%) or the Datex-Ohmeda TruSat (83% and 82%). The Masimo Radical also had higher SpO2 sensitivity (100% and 95%) than the Nellcor N-600 (65% and 50%) or the Datex-Ohmeda TruSat (20% and 15%) during both MG and VG motion. During MG motion, the Masimo Radical had the lowest PR failure rate (0%) compared with the Nellcor N-600 (22.2%) and Datex-Ohmeda TruSat (1.3%). However, during VG motion, the Masimo Radical had the lowest SpO2 failure rate (0%) of the three devices (Nellcor N-600 16.4% and Datex-Ohmeda TruSat 1.7%). Both the Masimo Radical and the Datex-Ohmeda TruSat had lower PR failure rates (0% and 4.4%) than the Nellcor N-600 (33.9%). There were no significant differences in SpO2 or PR performance index between the three devices.
☆
Financial Support: Supported in part by Masimo Corporation, Irvine, CA, USA, which provided equipment and a grant for the study. Conflict of interest: Dr. Shah is a Speaker Board Member, Advisory Board Member, & Stock holder in Masimo Corporation, Irvine, CA, USA. ⁎ Correspondence: Nitin Shah, MD, Department of Anesthesiology, Long Beach Veterans Administration Healthcare System, 5901 E. 7th St., Long Beach, CA 90822, USA. Tel.: 562-826-8000 ext 3264; fax: 562 826-5991. E-mail address: [email protected] (N. Shah). ☆☆
0952-8180/$ – see front matter. Published by Elsevier Inc. doi:10.1016/j.jclinane.2011.10.012
386
N. Shah et al. Conclusions: The Masimo Radical had higher SpO2 sensitivity and specificity than the Nellcor N-600 and Datex-Ohmeda TruSat during conditions of motion and induced low perfusion in this volunteer study. Published by Elsevier Inc.
1. Introduction Oxygen saturation (SpO2) is the estimate of the oxygen saturation of hemoglobin. Continuous monitoring of SpO2 enables clinicians to detect hypoxemia at an early stage and intervene to prevent any dire consequences. Pulse oximeters provide immediate, continuous, and noninvasive monitoring of SpO2 without the need for calibration and frequent sensor replacement [1]. Conventional pulse oximeters have significant limitations, including high rates of false alarms and failed measurements when used on patients with low perfusion or during patient motion [2-6]. This high false-alarm rate incurs the risk of desensitizing clinical care givers to true alarms [7]. Failed measurements are another significant limitation. Earlier work showed that pulse oximeters failed in the operating room in 2.5% of 10,312 patients randomized to be monitored [8]. Pulse oximeter failure rates are even higher in critical care, intensive care, and Postanesthesia Care Unit (PACU) environments [9]. New-generation pulse oximeters are designed to overcome these limitations. The newergeneration pulse oximeters have fewer false alarms and higher accuracy than conventional pulse oximeters [10-13]. In this study, we evaluated the performance of three newgeneration pulse oximeters [Masimo Radical (V 5.0), Nellcor N-600 (V 1.1.2.0), and Datex-Ohmeda TruSat] in a cool environment to mimic reduced perfusion and during induced motion. Both machine-generated (MG) motion (standardized and reproducible) and volunteer-generated (VG) motion (more challenging for pulse oximeter sensors) were used [14]. Both hypoxic and normoxic conditions were studied.
2. Materials and methods After Institutional Review Board of the Long Beach VAHS, CA approval and informed consent, 10 healthy, ASA physical status 1 volunteers (5 women), ranging in age from 22 to 30 years (mean age 26 yrs), were studied. Earlier studies of pulse oximeter performance studied 10 healthy volunteers and had obtained statistically significant results [13,14]. All subjects were nonsmokers with no evidence of vascular or other systemic disease. Each subject was monitored with three oximeter sensors: the Masimo Radical (V 5.0; Masimo Corp., Irvine, CA, USA), the Nellcor N-600 (v1.1.2.0; Covidien, Boulder CO, USA), and the GE DatexOhmeda TruSat (GE Healthcare Technologies, Waukesha, WI, USA), one each on digits 2, 3, and 4 of the moving “test”
hand, and one each of the same make and model on the digits of the stationary “control” hand. Sensor assignments to the individual digits were rotated among the subjects. All sensors were of the disposable adhesive type to minimize sensor displacement during hand motion. The averaging time for each oximeter was set at 8 seconds and so were essentially equivalent for all devices. All devices were new, purchased for use in the study. The test hand was strapped to a motorized motion table, which produced repeatable and continuous hand motions. This motion table, which was similar to one used in previous studies [14,15], moved the hand up and down while the elbow remained fixed. It was configured so that the fingertips either tapped or rubbed on a smooth surface. The amplitude of the motion was ± 2 cm, and the frequency was either fixed or randomly varied (aperiodic) within the range of 1 to 4 Hz. Previous studies found that these motions caused any pulse oximeter to fail at least occasionally [14,15]. The peripheral perfusion of healthy, awake volunteers is clearly not that of typical surgical patients. To partially compensate for this fact, we maintained our room temperature in the 16°C - 18°C range and kept the subject's arms exposed. Skin temperature was monitored by a thermocouple on the fifth digit. This method was employed in other studies to evaluate the performance of pulse oximeters during low perfusion [13,15,16]. The oximeter sensors were applied to each subject and connected to their respective instruments. Once applied, the sensors were optically isolated from each other by a black nonconductive plastic covering. The oximeters were connected via serial data ports to a computerized data logger that recorded pulse rate (PR) and SpO2 values from all instruments and perfusion index from the Masimo pulse oximeter once per second. After recording room air control values with both hands stationary, the motion table was activated, and two minutes of data were recorded for each of two motions: 1) fingers tapping at 3 Hz or at a frequency that varied randomly between 1 and 3 Hz, and 2) fingers rubbing at the same frequencies. Once the two motions were completed and all SpO2 values returned to baseline, the sensors were moved to different test fingers and the series was repeated twice so that all three test digits were monitored with each test oximeter. In the next series of tests on each subject, a hypoxemic episode was induced during each motion period. For this purpose, subjects were outfitted with a disposable rebreathing circuit with a CO2 absorber (CO2 rebreathing system; King Systems, Noblesville, IN, USA). The rebreathing circuit removes CO2 from the exhaled air of the subject,
Three new-generation pulse oximeters leaving the O2 and other gases available for rebreathing. As the subject continues to breathe through the rebreather apparatus, the gas mixture will contain progressively less O2 and thus gradually induces hypoxia. A TOSCA sensor (PtcCO2 + Masimo Radical pulse oximeter; Radiometer America, Philadelphia PA, USA) was placed on the right ear to rapidly detect SpO2 changes during hypoxia and guide administration of 100% O2 after desaturation. A TOSCA ear oximeter monitors both SpO2 and transcutaneous CO2 values. This requires the machine to heat the tissues underneath the sensor to 42°C to allow rapid diffusion of CO2 from arterial blood through the surrounding tissues to the sensor interface. The TOSCA ear sensor has a faster reaction time for monitoring SpO2 compared with pulse oximeter finger sensors, which allowed us to minimize the time when volunteers were hypoxic. After baseline data were recorded during normoxemia, subjects were instructed to place the rebreathing circuit mouthpiece in their mouth and breathe room air through their nose and exhale through their mouth to fill a 3-liter bag. Once the bag was filled, a nose clip was placed on the subject so that they breathed the recirculated gas mixture from the 3-liter bag. Subjects continued to breathe from the bag until SpO2 reached approximately 75%, as measured by the TOSCA ear sensor. Volunteers were instructed to remove the mouthpiece if unpleasant symptoms developed at any time during rebreathing. During hypoxemia, data were recorded during two types of motion: 1) random tapping, which was accompanied by the disconnecting and reconnecting of the sensors at the start of motion, and 2) random rubbing. Once SpO2 reached 75%, volunteers were given 100% O2 until SpO2 reached 100% on all control monitors, at which time the motion was terminated. Each rapid desaturation/resaturation experiment was completed within three minutes. Continuous verbal contact was maintained with the subjects. All subjects were able to complete the protocol without difficulty. The protocol during hypoxemia included the additional step of disconnecting and reconnecting all test sensors from their respective instruments after the motion had begun. This disconnect-reconnect experiment simulated the clinical scenario of placing an oximeter sensor on a patient who is already moving when the sensor is applied. We calculated the failure rate for each device by dividing the amount of time that each instrument displayed no SpO2 value by the total time of the test. To combine measures of accuracy and reliability, we also calculated the performance index for each device by dividing the amount of time the instrument displayed a current SpO2 value that was within 7% of the simultaneous control value by the total time of the test. Performance index was a method used to evaluate pulse oximeter performance in a previous study [13]. The error margin of 7% was chosen because SpO2 changes rapidly over a wide range in these experiments. A smaller margin of error will reduce the performance index of all instruments, and a larger margin will increase it. The performance index
387 for PR accuracy was calculated in the same manner as for SpO2, but with a margin of error of 10% compared with the control value. The oximeter's ability to detect hypoxemic events is quantified by sensitivity and specificity – sensitivity being the probability of detecting hypoxemia and specificity being the probability of detecting normoxia (lack of false alarms). Formally, sensitivity is the number of true positives divided by the number of true positives plus the number of false negatives. For the purposes of this study, sensitivity was calculated by dividing the number of events (tests) in which the test device read b 90% when the control device read b 90% during hypoxia (true positives) by the number of events (tests) the test device read b 90% plus the number of times the test device read N 90% when the control device read b 90% during hypoxia (false negatives). Sensitivity was calculated from the 20 machine-generated and 20 volunteergenerated motion tests with desaturation. Conversely, specificity, which is formerly calculated by dividing the number of true negatives by the number of true negatives plus the number of false positives, for the purposes of this study was calculated by dividing the number of events that the test device reported SpO2 values N 90% when the control device also reported values N 90% during normoxia (true negatives) by the number of true negatives plus the number of false positives during the 60 machine-generated and 60 volunteer-generated motion tests. We also compared the effect of two types of motions, machine-generated and volunteer-generated, on performance of the pulse oximeters by measuring the duration of missed events, number of false alarms, failure rate, and recovery time. Missed events were calculated by counting the number of tests in which the test device reported normoxia (SpO2 N 90%) when the control device reported a desaturation event b 90% SpO2 during 20 volunteer- and 20 machine-generated motion tests. False alarms were calculated by counting the number of times the test device alarmed for a SpO2 value b 90%, during normoxia as determined by the control device, during the 60 volunteer- and 60 machine-generated motion tests. Failure rate was calculated by dividing the amount of time the test device displayed no reading by the total time of the test. Recovery time was defined as the time required for the test device to display a SpO2 value within 7% of the control value after the cessation of motion. The duration of each experimental condition when subjects were breathing room air (mechanical tapping and rubbing and voluntary tapping and rubbing) was two minutes, with each condition repeated twice so that all test digits were monitored with each test oximeter. All test periods were of the same duration so that results could be compared, except for the recovery time tests. These tests were completed within three minutes, the exact duration of which depended on how long it took for a device to recover from a failure to provide a measurement. The time to desaturate was the same for all pulse oximeters tested. However, the time to recovery from desaturation varied
388
N. Shah et al.
among the pulse oximeters. Since each subject wore sensors for all the devices at the same time, the duration of each experimental condition for each device was identical, except for the recovery time. Data from all subjects were combined. Analysis of variance (ANOVA) was performed, with Fisher's post hoc test, to compare performance index, missed events, false alarms, and failure rate results for the three oximeters. In addition, ANOVA with Fisher's post hoc test, and Chisquare analysis as appropriate, were used to compare sensitivity and specificity of the three oximeters and to compare the effect of machine-generated and volunteergenerated motions on missed events, false alarms, failure rate, and recovery time. The mean ± 1 SD was calculated for subject fingertip temperature and a Mann-Whitney Rank Sum test for nonparametric data was used to compare the perfusion index between the test and control hands. P b 0.05 was considered statistically significant for all statistical tests.
3. Results A total of 160 motion tests were performed (80 with machine-generated motion, 80 with volunteer-generated motion, 40 with desaturation, and 120 on room air) on 10 study volunteers. Missed events were counted out of 40 tests during hypoxemia (20 with machine motion and 20 with volunteer motion), and false alarms were counted out of 120 tests during normoxia (60 with machine motion and 60 with volunteer motion). The range in average temperatures for each experimental condition for all subjects was 18.9° C - 27.7° C, with an average fingertip temperature (± 1 SD) for all subjects during all conditions of 21.4° C ± 3.3° C. The median perfusion index for all subjects during all conditions was 0.95 (0.63 at
Table 1
the first quartile) in the control hand and 1.16 (0.873 at the first quartile) in the motion hand during non-motion conditions. The difference was significant (P b 0.001). The performance index of the pulse oximeters (Table 1) for SpO2 was 97.5%, 72.3%, and 83.2% for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat during machine motion and 98.5%, 73.1%, and 81.9% during volunteer motion, respectively, but the differences were not significant. Likewise for PR, the performance index of the pulse oximeters was 82.9%, 61.0%, and 78.0% during machine motion and 88.5%, 60.3%, and 73.6% during volunteer motion for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat, respectively, but differences were not significant. The Masimo Radical had failure rates of 0% during both machine and volunteer motions for both SpO2 and PR (Table 1). The SpO2 failure rate was significantly lower for the Masimo Radical than for the other two devices (P b 0.05) during volunteer-generated motion, but no significant differences were noted during the machine-generated motion tests. The PR failure rate of the Masimo Radical was significantly lower than that of the Nellcor N-600 and Datex-Ohmeda TruSat during machine-generated motion (P b 0.05), whereas during volunteer-generated motion the PR failure rates of both the Masimo Radical and DatexOhmeda TruSat were significantly lower than the Nellcor N-600 (P b 0.005). Table 2 shows that during machine-generated motion, specificity (probability of detecting normoxia during the 60 machine-generated motion tests) was 93%, 67%, and 83% for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat, respectively. During volunteer motion, specificity was 97%, 77%, and 82% for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat, respectively. During both machine- and volunteer-generated motion, the Masimo
Performance index and failure rate (FR) during machine-generated and volunteer-generated motion
Machine-generated motion (80 tests) Device
SpO2 performance index (%)
Masimo Radical (V 5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
97.5 72.3 83.2
SpO2 FR (%) 0 9.3 1.3
PR performance index (%)
PR FR (%)
82.9 61.0 78.0
0 22.2 ⁎ 1.7 ⁎
Volunteer-generated motion (80 tests) Device
SpO2 performance index (%)
SpO2 FR (%)
PR performance index (%)
PR FR (%)
Masimo Radical (V 5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
98.5 73.1 81.9
0 16.4 ⁎ 1.7 ⁎
88.5 60.3 73.6
0† 33.9 4.4 †
PR = pulse rate; performance index = duration of time the test device displayed oxygen saturation (SpO2) within 7% or PR value with 10% of the control device divided by the total time of the test; failure rate = duration of time the test device displayed no SpO2 or PR value divided by the total time of the test. ⁎ P b 0.05 vs Masimo Radical (V 5.0). † P b 0.005 vs Nellcor N-600 (V 1.1.2.0).
Three new-generation pulse oximeters Table 2
389
Oxygen saturation (SpO2) sensitivity and specificity during machine-generated and volunteer-generated motion
Machine-generated motion (20 tests for missed events/sensitivity and 60 tests for false alarms/specificity) Device
Missed events
Sensitivity (%)
False alarms
Specificity (%)
Masimo Radical (V5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
0/20 7/20 16/20
100 65 ⁎ 20 ⁎
4/60 20/60 10/60
93 67 ⁎ 83 ⁎
Device
Missed events
Sensitivity (%)
False alarms
Specificity (%)
Masimo Radical (V 5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
1/20 10/20 17/20
95 50 ⁎ 15 ⁎
2/60 14/60 11/60
97 77 ⁎ 82 ⁎
Volunteer-generated motion
Missed events = number of tests in which the test device reported normoxia (SpO2 N 90%) when the control device reported a desaturation event b 90% SpO2. ⁎ P b 0.05 vs Masimo Radical (V5.0).
Radical had higher specificity than either the Nellcor N-600 or Datex-Ohmeda TruSat (P b 0.05). Similar results were obtained for sensitivity of the pulse oximeters – the ability to detect hypoxemia. Sensitivity was 100%, 65%, and 20% for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat during the 20 machine motion tests, and 95%, 50%, and 15% during the 20 volunteer motion tests, respectively, with the Masimo Radical having significantly higher sensitivity than either the Nellcor N-600 or Datex-Ohmeda TruSat (P b 0.05). There was no significant difference in sensitivity or specificity between the Nellcor N-600 and Datex-Ohmeda TruSat. When the effects of the two types of motions on performance of the pulse oximeters were compared (Table 3), there were no significant differences in failure rate or recovery time for both SpO2 and PR. Failure rates of the pulse oximeters while recovering from motion (Table 4) for SpO2 were 1.3%, 25.0%, and 15.0% during machine motion and 3.8%, 31.3%, and 23.8% during volunteer motion for the Masimo Radical, Nellcor N-600, and Datex-Ohmeda TruSat, respectively. The difference in SpO2 failure rates between the Masimo Radical and both the Nellcor N-600 and the DatexOhmeda TruSat was statistically significant (P b 0.05). For PR, the Masimo Radical had a significantly lower failure rate than the Nellcor N-600 during both machineand volunteer-generated motion (P b 0.05). The Masimo Radical also had a significantly lower PR failure rate
Table 3
than the Datex-Ohmeda TruSat during volunteer motion (P b 0.05), but differences were not significant during machine-generated motion. Recovery times (mean and range) from the failed events for SpO2 were 21 (13-27 sec), 14 (6-36 sec), and 42 (9-180 sec) seconds during machine motion and 17 (12-21 sec), 13 (6-24 sec), and 43 (12-288 sec) seconds during volunteer motion for the Masimo Radical, Nellcor N-600, and DatexOhmeda TruSat, respectively. The Masimo Radical had a significantly shorter recovery time than the Datex-Ohmeda TruSat during both machine-generated and volunteergenerated motion tests (P b 0.05), but the difference between the Masimo Radical and the Nellcor N-600 or the Nellcor N-600 and the Datex-Ohmeda TrueSat devices were not significant. Results for PR recovery times were similar. The Masimo Radical had a significantly shorter PR recovery time than the Datex-Ohmeda TruSat during both machinegenerated and volunteer-generated motion tests (P b 0.05), but the difference between the Masimo Radical and the Nellcor N-600 or the Nellcor N-600 and the Datex-Ohmeda TrueSat devices was not significant.
4. Discussion Our study involved volunteer subjects in a laboratory setting. Such studies allow investigators to use a more rigorous protocol while maintaining control over variables.
Machine-generated (MG) versus volunteer-generated (VG) motion: effects on pulse oximeter performance
Motion
SpO2 (%) RT (min)
PR RT (min)
SpO2 missed events (min)
PR false alarms (min)
SpO2 failure (min)
PR failure (min)
MG VG
0.44 ± 0.80 0.60 ± 1.1
0.65 ± 0.92 0.87 ± 1.1
2.5 ± 2.8 2.3 ± 2.3
3.8 ± 2.7 3.5 ± 2.7
0.35 ± 1.1 0.60 ± 1.6
0.80 ± 1.8 1.3 ± 2.3
Data are means± SD. Differences were not significant. SpO2 = oxygen saturation, RT = recovery time, PR = pulse rate.
390 Table 4
N. Shah et al. Recovery time (RT) and failure rate (FR) while recovering during machine-generated and volunteer-generated motion
Machine-generated motion (80 tests) Device
SpO2 RT [sec; means (ranges)]
SpO2 no. times fail/total
SpO2 FR
PR RT [sec; means (ranges)]
PR no. times fail/total
PR FR
Masimo Radical (V 5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
21.0 (13-27) 14.3 (6-36) 42.0 (9-180) ⁎
1/80 20/80 12/80
1.3% 25.0% ⁎ 15.0% ⁎
14.5 (6-24) 21.4 (6-39) 44.3 (12-168) ⁎
12/80 22/80 12/80
15.0% 27.5% ⁎ 15.0%
Device
SpO2 RT [sec; means (ranges)]
SpO2 no. times fail/total
SpO2 FR
PR RT [sec; means (ranges)]
PR no. times fail/total
PR FR
Masimo Radical (V5.0) Nellcor N-600 (V 1.1.2.0) Datex-Ohmeda TruSat
17.0 (12-21) 13.0 (6-24) 42.8 (12-288) ⁎
3/80 25/80 19/80
3.8% 31.3% ⁎ 23.8% ⁎
21.3 (6-48) 20.8 (3-36) 40.2 (12-270) ⁎
9/80 35/80 17/80
11.3% 43.8% ⁎ 21.3% ⁎
Volunteer-generated motion
PR = pulse rate, recovery time = time required (sec) for test device to display oxygen saturation (SpO2) value within 7%, or PR value within 10% of control device value after cessation of motion. ⁎ P b 0.05 vs Masimo Radical (V 5.0).
The major disadvantage of laboratory studies is that they may not fully replicate the characteristics of complex clinical settings such as the different types of motions exhibited by patient populations [17]. The potential of unrecognized hypoxemia, however, mandates that research in volunteers mimic actual clinical scenarios as closely as possible. Earlier studies have described the use of active motions to test pulse oximeter devices [15,18]. In our study we incorporated two types of motions, machine-generated and volunteer-generated motions. We found no significant differences among them on the performance of the pulse oximeters such as for the number of missed events, false alarms, failure rates, and recovery times. To simulate this clinical scenario, we tested the performance of pulse oximeters during motion during hypoxic conditions in the presence of reduced perfusion induced by lowering the room temperature. Other researchers have reported perfusion values in healthy subjects during room temperature conditions of about 5.0% [16]. Our subjects, therefore, had a nearly threefold decrease in perfusion index compared with what has been found in healthy subjects in room temperature conditions, indicating reduced peripheral perfusion of the digits during the testing period. This study showed that during experimental motion conditions the Masimo Radical had the highest SpO2 sensitivity and specificity during normoxia and the lowest SpO2 failure rate during hypoxemia compared with the Datex-Ohmeda TruSat and Nellcor N-600 devices. No significant differences were observed in SpO2 or PR performance index. When comparing recovery time among the pulse oximeters, mean SpO2 and PR recovery time were significantly shorter for the Masimo Radical than for the Datex-Ohmeda TruSat, but there was no difference between the Masimo Radical and the Nellcor N-600 or the Nellcor N-600 and the Datex-Ohmeda devices. Investigators have used numerous methodologies in the laboratory and the clinical environment to compare both
conventional and new-generation “motion-tolerant” pulse oximeter technologies during conditions of motion. In general, motion-tolerant technologies have had fewer data dropouts and false alarms, and better overall performance compared with conventional technologies. In studies that compared motion-tolerant technologies, some results have been less consistent, in part because of the various software versions of the pulse oximeter technologies and in part because of methodological weaknesses. Notable issues with previous studies include comparing PR with electrocardiography heart rate, comparing one pulse oximeter to other pulse oximeters to calculate true and false alarms, inconsistent sensitivity mode selection between various pulse oximeters, and the use of reusable finger clips instead of adhesive sensors. However, we found that our results were consistent with previous work using similar methods with earlier versions of pulse oximeters [13]. Our study was limited by its lack of use of arterial blood gas (ABG) measurements as the comparative standard. Comparisons with ABG measurements are required to evaluate the absolute accuracy of SpO2 values obtained from a pulse oximeter. Our study was also limited by its use of volunteers rather than hospitalized subjects, as the physiological characteristics of each group may be different. Our study also relied upon machine-generated or deliberate volunteer-generated motion; these types of motion may differ from naturally occurring motion in a clinical environment. Due to these limitations, we strongly urge clinicians evaluating various pulse oximeters to perform their own side-by-side comparisons in clinically relevant scenarios, using some form of automated data collection for objective recording and analysis. In volunteers in a cold environment during machine- and volunteer-generated motion, the Masimo Radical had higher SpO2 sensitivity and specificity than the Nellcor N-600 and Datex-Ohmeda TruSat. The Masimo Radical had lower SpO2 and PR failure rates during volunteer motion during
Three new-generation pulse oximeters both normoxia and hypoxia, but the results were less consistent during machine-generated motion. There were no significant differences in SpO2 or PR performance index between the three devices.
References [1] Poets CF, Southall DP. Noninvasive monitoring of oxygenation in infants and children: practical considerations and areas of concern. Pediatrics 1994;93:737-46. [2] Lawson D, Norley I, Korbon G, Loeb R, Ellis J. Blood flow limits and pulse oximeter signal detection. Anesthesiology 1987;67:599-603. [3] Severinghaus JW, Spellman MJ Jr. Pulse oximeter failure thresholds in hypotension and vasoconstriction. Anesthesiology 1990;73:532-7. [4] Wilson S. Conscious sedation and pulse oximetry: false alarms? Pediatr Dent 1990;12:228-32. [5] Plummer JL, Zakaria AZ, Ilsley AH, Fronsko RR, Owen H. Evaluation of the influence of movement on saturation readings from pulse oximeters. Anaesthesia 1995;50:423-6. [6] Wiklund L, Hök B, Ståhl K, Jordeby-Jönsson A. Post anesthesia monitoring revisited: incidence of true and false alarms from different monitoring devices. J Clin Anesth 1994;6:182-8. [7] Lawless ST. Crying wolf: false alarms in a pediatric intensive care unit. Crit Care Med 1994;22:981-5. [8] Moller JT, Johannessen NW, Espersen K, et al. Randomized evaluation of pulse oximetry in 20,802 patients: II. Perioperative events and postoperative complications. Anesthesiology 1993;78:445-53.
391 [9] Brown LT, Purcell GJ, Traugott FM. Hypoxaemia during postoperative recovery using continuous pulse oximetry. Anaesth Intensive Care 1990;18:509-16. [10] Lie C, Kehlet H, Rosenberg J. Comparison of the Nellcor N-200 and N-3000 pulse oximeters during simulated postoperative activities. Anaesthesia 1997;52:450-2. [11] Lutter NO, Urankar S, Kroeber S. False alarm rates of three thirdgeneration pulse oximeters in PACU, ICU, and IABP patients. Anesth Analg 2002;94(1 Suppl):S69-75. [12] Malviya S, Reynolds PI, Voepel-Lewis T, et al. False alarms and sensitivity of conventional pulse oximetry versus the Masimo SET technology in the pediatric postoperative anesthesia care unit. Anesth Analg 2000;90:1336-40. [13] Barker SJ. “Motion-resistant” pulse oximetry: a comparison of new and old models. Anesth Analg 2002;95:967-72. [14] Gehring H, Hornberger C, Matz H, Konecny E, Schmucker P. The effects of motion artifact and low perfusion on the performance of a new generation of pulse oximeters in volunteers undergoing hypoxemia. Respir Care 2002;47:48-60. [15] Barker SJ, Shah NK. The effects of motion on the performance of pulse oximeters in volunteers (revised publication). Anesthesiology 1997;86:101-8. [16] Nishiyama T. Pulse oximeters demonstrate different responses during hypothermia and changes in perfusion. Can J Anaesth 2006;53:136-8. [17] Jopling MW, Mannheimer PD, Bebout DE. Issues in the laboratory evaluation of pulse oximeter performance. Anesth Analg 2002;94(1 Suppl):S62-68. [18] Barker SJ. Standardization of the testing of pulse oximeter performance. Anesth Analg 2002;94(1 Suppl):S17-20.