- Email: [email protected]
Long wavelength video detection of fire in ship compartments
ARTICLE IN PRESS
Fire Safety Journal 41 (2006) 315–320 www.elsevier.com/locate/firesaf
Long wavelength video detection of fire in ship compartments Jeffrey C. Owrutskya,, Daniel A. Steinhurstb, Christian P. Minorb, Susan L. Rose-Pehrssona, Frederick W. Williamsa, Daniel T. Gottukc a
Chemistry Division, US Naval Research Laboratory, Washington, DC 20375, USA b Nova Research, Inc., Alexandria, VA 22308, USA c Hughes Associates, Inc., Baltimore, MD 21227, USA
Received 3 March 2005; received in revised form 24 October 2005; accepted 15 November 2005 Available online 23 March 2006
Abstract This paper describes progress using filtered, long wavelength video image-based detection (LWVD) of events in laboratory tests and full scale fire testing within the Volume Sensor Program at the U.S. Naval Research Laboratory (NRL). This effort toward developing a real-time, remote sensing detection system utilizes video image detection (VID) systems based on cameras that operate in the visible region, which were developed for detecting smoke and have recently been adapted to detecting fire. However, VID systems are not effective at detecting fire outside the direct line of sight of the camera. Our studies demonstrate that long wavelength imaging achieves effective detection of reflected flame emission compared to visible video images. A system that combines visible and long wavelength image capabilities may be more accurate and sensitive than either alone. Our LWVD approach exploits the long wavelength response of standard CCD arrays used in many cameras. A long pass filter (typically in the range 700–900 nm) increases the contrast for flaming and hot objects and suppresses the normal video image of the space, thereby effectively providing a degree of thermal imaging. There is more emission from hot objects in this spectral region than in the visible region (o600 nm). Testing has demonstrated the detection of objects heated to 400 1C or higher. A simple luminosity-based algorithm was developed and used to evaluate camera/filter combinations for fire, smoke and nuisance (false) event detection and response times. r 2006 Elsevier Ltd. All rights reserved. Keywords: Video image fire detection; Fire detection; Near infrared; Long wavelength; Camera; Flame detection
1. Introduction Long wavelength video image-based detection (LWVD) of fire has been developed in the Volume Sensor (VS) Program at the U.S. Naval Research Laboratory (NRL). In the VS Program, which is a component of the Advanced Damage Countermeasures Program in the Platform Protection Future Naval Capability, the primary goal is to employ optical detection methods for inexpensive, remote and real-time monitoring of ship spaces, including the detection of fire and smoke. Previous efforts at NRL have demonstrated significant improvements in accuracy, sensitivity and response time in fire and smoke detection using multicriteria approaches which combine various Corresponding author
E-mail address: [email protected] (J.C. Owrutsky). 0379-7112/$ - see front matter r 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.firesaf.2005.11.011
sensors and utilize neural network algorithms [1,2]. The response time of these methods, however, is inherently limited by their reliance on point detectors (e.g., of heat, smoke or gases, such as CO or CO2), which depend on molecular or thermal diffusion from the fire or smoke source to the sensor. Optical sensors do not depend on diffusion, so they can provide stand-off detection and possibly faster response times to more quickly identify fires or other hazardous conditions. The central element of the VS Program is the identification of fire, smoke and other conditions using video image detection (VID) systems in which the images are analyzed by machine vision algorithms. Progress in this area includes testing and evaluation of commercial systems [3,4]. These video fire detection systems are most effective at identifying smoke and direct line of sight (LOS) fires and are less successful for identifying small or obstructed flames from
ARTICLE IN PRESS 316
J.C. Owrutsky et al. / Fire Safety Journal 41 (2006) 315–320
reflected emission or hot objects. Video detection systems can respond quickly, but a common limitation is that they require a direct LOS between the source and detector for optimum performance. It is not feasible for practical or economical reasons to deploy enough cameras to ensure complete direct LOS coverage throughout the compartment space, especially in crowded spaces that are typical for naval ships. A possible remedy being explored is to detect reflected radiation as described below using LWVD. In addition, other kinds of sensors, including acoustic and single element optical detectors, are being incorporated into the VS system as described elsewhere [4]. The LWVD is one component of the multisensor, data fusion-based VS system currently under development at NRL. As currently conceived for deployment on future U.S. Navy ships, there could be one or more VS system in each compartment of the ship, therefore it is important to minimize the unit cost. The goal of developing an inexpensive yet effective system motivated using near infrared (NIR) imaging or LWVD. The LWVD uses commonly available, inexpensive cameras. These can be simple surveillance cameras (o$100) or commercial cameras (o$1000), both of which cost much less than mid infrared cameras (3–12 mm, 4$15,000). The latter are almost certainly better for thermal imaging. For example, they have a much lower minimum detectable temperature and can distinguish between body temperature and room temperature. Although mid infrared cameras outperform LWVD, the latter provides a desirable trade off between cost and performance for the applications described here. Long wavelength or NIR emission radiation detectors have been used previously for fire detection. Lloyd et al. reported [5,6] and patented [7] approaches intended to detect reflected near infrared emission using several narrowband detectors without any imaging capability. The results demonstrated the feasibility of detecting reflected NIR emission. In addition, NIR image detection has been applied in background free environments, such as for monitoring forest fires from terrestrial-based [8] and satellite images [9], tunnels [10], as well as aircraft cargo surveillance [11]. The latter study included a detailed characterization using CCD cameras operating in the NIR for remote temperature measurement and demonstrated a minimum detectable temperature of about 350 1C. NIR detection of forest fires and other quiescent environments is effective in part because there are few interferences or nuisances to complicate the detection. A narrowbandfiltered (1140 nm) NIR imaging technique that includes image analysis has been patented as a method for improved fire and hot object detection by reducing false alarms in protected areas [12]. In our LWVD approach to fire detection, we use a long pass filter (LPF) positioned in front of a standard CCD array camera. The LPF transmits light with wavelengths longer than a cutoff, typically between 700 and 900 nm, which increases the contrast for fire, flame and hot objects by suppressing the normal video images of the space. This
provides modest thermal imaging because there is more emission from hot objects in this spectral region than in the visible (o600 nm). Videos of long wavelength emission are demonstrated to be particularly beneficial for providing high contrast and therefore straightforward LOS detection for flames and hot objects, where the latter includes bulkheads heated by obstructed fires, and for identifying obstructed fire and flame based on reflected light. 2. Approach The LWVD method is implemented using long wavelength images acquired in real time and digitally recorded with various long pass filters (typically, 720–850 nm) using both a Sony camcorder (DCR-TRV27) in NightshotTM mode and inexpensive ‘‘bullet’’ cameras (Si-SPECO (CVC130R)). Cameras that operate in this wavelength region are often referred to as nightvision cameras and we use this nomenclature in this paper. This report will focus on results from full scale fire testing in the CVNX and VS1 test series [13] conducted aboard the ex-USS SHADWELL in April 2003 and the VS2 test series conducted at the facilities of Hughes Associates, Inc., in Baltimore, MD in 2003–2004 [14]. The video signal from a nightvision camera was converted from analog to digital video (DV) format with a Firewire video adapter (Dazzle Hollywood DV Bridge) or to digital .AVI format with a USB video adapter (Belkin USB VideoBus II) for suitable input into a computer. A program coded in Mathworks’ numerical analysis software suite, MATLAB v6.5 (Release 13), was used to control the video input acquisition from the cameras and to analyze the video images. The analysis was carried out using a straightforward luminosity-based algorithm developed at NRL explicitly for analysis of nightvision images [13]. The luminosity algorithm was designed and implemented to capture the enhanced sensitivity of the nightvision cameras to the thermal emission of fires, hot objects, and especially flame emission reflected off walls and around obstructions from a source fire not in the field of view (FOV) of the camera. A brief outline of the algorithm is presented here; a more detailed description of the luminosity algorithm is available in an NRL memorandum report [13]. The LWVD algorithm detects fire events by comparing the luminosity, L, which is the sum of the pixel intensities, of the current video frame to the sum of a reference luminosity, Lb, and an alarm threshold, Lth. The algorithm tracks the number of frames with L4Lb+Lth and then alarms when a persistence criteria is met. Application of the algorithm in CVNX and VS1 tests has shown that a fire event generally increases the luminosity of a video frame by an amount independent of the background illumination. The reference luminosity is chosen as the luminosity of a frame 30 s from the beginning video acquisition or of the .AVI file, generally early enough in the test to consist entirely of background features. To mitigate
ARTICLE IN PRESS J.C. Owrutsky et al. / Fire Safety Journal 41 (2006) 315–320
317
the effects of large variations observed in the background luminosity, a nonlinear relationship between the reference Lb and the alarm threshold is used: pffiffiffiffiffiffiffi Lth ¼ 2 Lb which yields proportionally smaller thresholds for larger background luminosities. Persistence of the L4Lb+Lth condition is used to discriminate against spurious bright nuisances such as a flash of light or a reflective object rapidly moving through the space. A frame with L4Lb+Lth will increment the alarm count, while a frame with LoLb þ Lth will decrement this count, but never to a value less than zero. An event alarm is generated when the alarm count reaches 75. Given the rate of video frame processing, the algorithm’s minimum response time for event detection is 5 s. A maximum response, or reset alarm, time is not necessary for the analysis of recorded tests. Alarms were indicated in real time and alarm times were recorded to files for later retrieval and compilation into a database. A bitmap background video image was also stored at the start of each test. Luminosity time series data were recorded for the entire test.
Fig. 1. Camera video from a test on the ex-USS SHADWELL. Regular and nightvision still images before and during a flaming event: (a) regular camera at start of test; (b) nightvision camera at start of test; (c) regular camera during period of open flame just outside the camera FOV; and (d) nightvision camera during period of open flame within the camera FOV.
3. Results The CVNX, VS1, and VS2 test series have produced a ‘library’ of more than 300 tests of fire and smoke sources and nuisances in real or simulated ship compartments. These tests have included heated bulkheads due to fires in the adjacent spaces, flaming fires, both in and out of the field of view, smoldering sources and various nuisance events, such as grinding, welding, and activity and motion within the compartment. Evaluation of this ‘library’ using LWVD has shown that the LWVD approach is useful for detection of fire and smoke sources typical of US Navy environments, especially for reflected fire emission and hot objects. Also, LWVD-detected luminosity was correlated with thermocouple temperature measurements of a heated bulkhead to demonstrate detection of objects hotter than 400 1C. Several general capabilities of the LWVD method were apparent from visual inspection of the video streams as the tests were being conducted. First, flaming fires are detected with greater sensitivity with filtered nightvision cameras than with regular cameras because there is more emission from hot objects at the longer wavelengths detected by the nightvision cameras. NIR emission from flames is easily visible to the nightvision cameras, which is not always the case for regular video cameras. The point is demonstrated in Fig. 1, which consists of several panels of images extracted from the videos from a test. Panels (a) and (b) show images from a test aboard the ex-USS SHADWELL for the collocated regular cameras and filtered nightvision cameras, respectively, prior to source ignition. The images in panels (c) and (d) are from the same cameras several minutes later while the cardboard box flaming source is
burning in the lower right-hand corner, within the camera FOV for the nightvision camera and just out of the camera FOV for the regular camera. The flame is evident in both types of video. Emission from the flame can be seen on the surface of the nearest cabinet in the regular video image, but a more dramatic change is observed in the nightvision camera image, in which the lower right-hand quadrant is brightly illuminated. Although this example is somewhat biased because the fire is in the FOV of the nightvision camera and not the regular camera, it nevertheless demonstrates the high sensitivity of filtered LWVD. The LWVD images are more informative so that less is required of the image analysis for detection and identification. A simple luminosity algorithm would be much less effective for regular video images. Another example is shown in Fig. 2 for a source that is completely outside the FOV of all cameras. The source for this test was several cardboard boxes placed on the deck below and behind the FOV of the camera. Panels (a) and (b) show images obtained prior to ignition of the source from the regular and nightvision cameras, respectively. The images in panels (c) and (d) were acquired several minutes after ignition when the source was fully engulfed in flame. Little or no difference can be seen between the regular images, with the exception of what appears to be smoke in the upper left-hand portion of the image. There is, however, a marked difference between the two nightvision images. NIR emission from the flame illuminates the entire area within the nightvision camera FOV. In the nightvision video, the NIR illumination fluctuates with the same temporal profile as the flame itself. This suggests that reflected NIR light could be used to detect flames that are
ARTICLE IN PRESS 318
J.C. Owrutsky et al. / Fire Safety Journal 41 (2006) 315–320
Fig. 2. Regular and nightvision images before test ignition and during a flaming event outside the camera FOV: (a) regular camera at start of test; (b) nightvision camera at start of test; (c) regular camera during period of open flame outside the camera FOV; and (d) nightvision camera during period of open flame within the camera FOV.
out of the camera FOV based on time-series analysis of the camera video alone. A second general observation is the ability of the nightvision cameras to act as an inexpensive form of thermal-imaging camera. In fact, they worked so well that during the CVNX test series, the ex-USS SHADWELL control room crew used the nightvision cameras as monitors for the progress of tests involving fires in adjacent compartments. For several tests in the CVNX test series, heptane spray fires were set up outside the test compartment and directed at a test compartment bulkhead to simulate a fire in an adjacent compartment. If the bulkhead behaves similar to a blackbody emitter, the NIR emission from the bulkhead should increase with increasing temperature. In addition to monitoring the luminosity with the LWVD system, the bulkhead temperature was measured with thermocouples attached to the bulkhead. Fig. 3 contains several still images from the nightvision camera as the bulkhead temperature increased during the test. NIR emission of the bulkhead is seen to grow in both size and intensity as the temperature increased. Before ignition (in panel (a)), the image is dark. After 279 s as shown in panel (b), a small illuminated region is seen and the maximum temperature of the bulkhead is 370 1C. In (c) and (d), 737 and 1243 s after ignition, respectively, the luminosity is higher and the temperatures measured are 540 and 565 1C. There is no indication from the regular video images (not shown) that the bulkhead temperature is elevated. Hot object identification is a capability of LWVD that cannot be achieved with regular video. The results of a hot bulkhead test for which images are shown in Fig. 3 were used to estimate the minimum
Fig. 3. Still images of nightvision video from test of a hot bulkhead before and at three times during the test: (a) nightvision camera at start of test; (b) nightvision camera 00:04:39 into test; (c) nightvision camera 00:12:17 into test; (d) nightvision camera 00:20:43 into test, effectively the end of the test.
detectable temperature (Tmin) of the LWVD system using the luminosity algorithm. Tmin was found to be about 400 1C by comparing the luminosity (intensity) measured for a small area of the images to the actual bulkhead temperature as recorded by thermocouples (TC). The results for the shipboard test were verified in laboratory measurements carried out using a blackbody emission source using the LWVD system and a narrowband-filtered (at 800 nm) single element detector. The Tmin of about 400 1C is similar to the value reported in the study by Sentenac et al. [11] using CCD camera temperature sensing for hangar bay surveillance. A final general observation is that there are tradeoffs in flame and hot object detection with smoke detection, depending on the LPF used. While the nightvision cameras are more sensitive to flames and other hot objects when longer wavelength LPFs are used, they suppress more of the visible image and increase the contrast for NIR emission, and consequently they degrade the sensitivity for detecting smoke. There is actually some flexibility in how one might utilize the enhanced NIR sensitivity in imaging applications. An important consideration in terms of cost, sensitivity and coverage is how many cameras to use. For a single camera version, increased sensitivity in the NIR while retaining the image for smoke detection would be implemented with light filtering (shorter wavelength LPF, perhaps 650–750 nm). An approach that would yield more sensitivity albeit with more complexity would be to use two cameras, one in the visible for smoke and a heavily (4900 nm) filtered nightvision camera for better contrast and more effective hot object and reflected flame detection. Another advantage to heavy filtering is that it could be used in spaces where privacy is an issue (e.g., berthing compartments), i.e., where visible image collection may not be desirable/allowable, but the availability of video fire protection would still be beneficial.
ARTICLE IN PRESS J.C. Owrutsky et al. / Fire Safety Journal 41 (2006) 315–320 Table 1 Summary of NRL LWVD for the VS1 test series Regular video
Total tests Total alarms Correct alarms Percentage (%) False alarm Percentage (%)
LWVD
VID1
VID2
VID3
40 34 27 96 7 21
40 23 20 71 3 13
40 3 3 11 0 0
40 26 24 86 2 8
23 13 57
3 3 100
26 14 54
Mutual alarms LWVD faster? Percentage (%)
Table 2 Summary of NRL LWVD for the CVNX test series Total tests LWVD alarms Correct alarms Percentage correct alarm
22 18 18 82%
Total tests VID1 alarms Correct alarms Percentage correct alarm
24 11 11 50%
No. of mutual alarms LWVD faster? Percentage
8 7 88%
For a more quantitative assessment, test results are given in Table 1 for a comparison of the alarm statistics and relative response times from the nightvision camera/ LWVD systems with those for the three regular camera/ commercial VID systems that were also evaluated during the VS1 test series [13]. The VIDS are generically referenced as VID1, VID2 and VID3 in this paper. See Ref. [13] for further details. The LWVD system performed well, with a 96% correct alarm percentage (No. of correct alarms/No. of non-nuisance sources) for the VS1 test series and a 21% false alarm rate (No. of false alarms/Total no. of alarms). The LWVD appears to be more sensitive to flaming and smoldering sources, but suffers from correspondingly higher false alarm sensitivity. For sources where more than one system went into alarm, the LWVD system reached an alarm condition faster than the commercial systems at least 50% of the time for the VID1 and the VID3 systems and 100% of the time for the VID2 system. The VID2 system alarmed on only three tests, so the statistical significance of that percentage is unclear. Table 2 presents the summary results for the nightvision/LWVD and regular/VID1 systems from the CVNX test series. The LWVD detected 18 while VID1 detected eleven out of the 22 total source fires, or 82 and 50% detection, respectively. There were no nuisance sources in the CVNX test matrix, so the nuisance rejection was not measured for the CVNX test series. For tests where both systems went into alarm, the LWVD alarmed faster for 88% of the sources. Comparison
319
of the system response times for tests with mutual alarms indicate that the LWVD system, as configured, responded on average two to three times faster than the other VID systems for flaming fires and approximately two times slower for smoldering events based on the VS1 results. Caution is advised in drawing conclusions from these results, in particular for the smoldering events where the LWVD and regular cameras are not necessarily detecting the same smoke events. For example, the NIR emission of a cartridge heater used to initiate a smoldering source could be detected in addition to or instead of the smoke, as described in more detail in Ref. [13]. One should also note that the LWVDs nuisance rejection algorithm is very immature in comparison to the regular VIDS and the result of any improvements would most likely lead to slower response times. In the initial stages of the LWVD/VIDS evaluation, the nightvision camera video was analyzed offline with the LWVD algorithm and the regular camera video was analyzed in real-time by the commercial VIDS. Therefore, it was not clear whether performance differences were due primarily to the different cameras, the different algorithms or a combination of the two. One way to address this problem was to analyze the same video sources with all available algorithm/system combinations. At the request of NRL, the vendor for the VID1 system post-processed the nightvision video from the entire VS1 test series (tests VS11 through VS1-40) using their VID product. The results are detailed in Ref. [13] in addition to the VID1 analysis results of the regular video. For the VID1 fire algorithm operating on video from the small test compartment, the NIR and regular cameras had the same accuracy; 62.5% of the fire sources were detected correctly with 10% false alarms. In the medium-sized test compartment, 100% of the sources were correctly detected with the nightvision video while the regular video correctly detected only 12.5%. For the VID1 smoke algorithm, the correctly detected events were 23.5% and 88.2%, respectively for the NIR and the regular cameras in the small test compartment with false alarm percentages of 50% and 0%. In the medium-sized test compartment, the smoldering event detection percentages were 55.6% and 83.3% correct detection and 45% and 0% false alarms for the NIR and regular cameras, respectively. The assessment by the vendor agrees with the results of the NRL LWVD algorithm analysis and also with basic inspection of the video. Compared to regular cameras, NIR cameras are better at flaming event detection and worse at smoldering event detection. 4. Conclusions LWVD offers an attractive and cost-effective augmentation to the standard implementation of VIDS technologies as seen in currently available commercial systems. As with other optical detection methods, LWVD provides a remote, stand-off sensing capability, so that it can in principle respond more quickly than point-type detectors
ARTICLE IN PRESS 320
J.C. Owrutsky et al. / Fire Safety Journal 41 (2006) 315–320
that rely on diffusive transport to operate. The NRL LWVD system emphasizes detection of fire and hot objects both within the camera FOV and outside the camera FOV. The NIR radiation from flaming and hot objects is sufficiently intense in the observation band of the nightvision cameras (700–1000 nm) to quickly detect fires and hot objects such as overheated cables and ship bulkheads heated by a fire in an adjacent compartment. As part of the NRL LWVD system, a simple total luminosity-based, machine vision algorithm has been developed to support the nightvision camera development. Since LPFs suppress the visible image detected with nightvision cameras, smoke is not easily detected by the LWVD system. The LWVD system already shows promise as an inexpensive pseudothermal imaging system for remote monitoring of surface temperatures. Both spatial and time-series analysis will be incorporated into an enhanced version of the LWVD algorithm in a parallel effort to the planned testing. Acknowledgements Support for this work provided by the Office of Naval Research, Future Naval Capabilities. References [1] Rose-Pehrsson SL, Hart SJ, Street TT, Williams FW, Hammond MH, Gottuk DT, et al. Early warning fire detection system using a probabilistic neural network. Fire Technol 2003;39:147–71. [2] Pfister G. Multisensor/multicriteria fire detection: a new trend rapidly becomes state of the art. Fire Technol Second Quart 1997;33:115.
[3] Gottuk DT, Lynch JA, Rose-Pehrsson SL, Owrutsky JC, Williams FW. AUBE ‘04 Proceedings of the eleventh international conference on automatic fire detection. Duisburg, Germany. [4] Rose-Pehrsson SL, Owrutsky, JC, Wales, SC, Williams, FW, Farley, JP, Steinhurst, DA, et al. AUBE ‘04 Proceedings of the eleventh international conference on automatic fire detection. Duisburg, Germany. [5] Zhu Y-J, Lloyd A, Sivathanu Y, Gore J. Experimental and numerical evaluation of a near infrared fire detector. In: Slaughter KC, editor. Fire research and engineering second (2nd) international conference (ICFRE2). 1998. p. 512. [6] Lloyd AC, Zhu YJ, Tseng LK, Gore JP, Sivanthanu YR. Fire detection using reflected near infrared radiation and source temperature discrimination. NIST GCR 1998;98:747. [7] Sivathanu Y, Joseph RK, Tseng L, Gore JP, Lloyd A. Flame and smoke detector. United States patent 6,111,511. 29 August 2000. [8] Thomas PJ. Near-infrared forest-fire detection concept. Appl Opt 1993;32:5348. [9] Lasaponara R, Cuomo V, Macchiato MF, Simoniello T. A selfadaptive algorithm based on AVHRR multitemporal data analysis for small active fire detection. Int J Remote Sens 2003;24:1723. [10] Wieser D, Brupbacher T. Smoke detection in tunnels using video images. NIST SP 965. February 2001. [11] Sentenac T, Le Maoult Y, Orteu J- J. Evaluation of a charge-coupleddevice-based video sensor for aircraft cargo surveillance. Opt Eng 2002;41:796–810. [12] Chan WS, Burge JW. Imaging flame detection system. United States patent 5,937,077. 10 August 1999. [13] Steinhurst DA, Minor CP, Owrutsky JC, Rose-Pehrsson SL, Gottuk DT, Williams FW, et al. Long wavelength video-based event detection preliminary results from the CVNX and VS1 test series, ex-USS SHADWELL 7–25 April 2003. NRL/MM-MR/6110—038733. 31 December 2003. [14] Lynch JA, Gottuk DT, Rose-Pehrsson SL, Owrutsky JC, Steinhurst DA, Minor CP, et al. Volume sensor development test series 2— lighting conditions camera settings and spectral and acoustic signatures. NRL/MR/6180—04-8843. 24 November 2004.
Fire Safety Journal 41 (2006) 315–320 www.elsevier.com/locate/firesaf
Long wavelength video detection of fire in ship compartments Jeffrey C. Owrutskya,, Daniel A. Steinhurstb, Christian P. Minorb, Susan L. Rose-Pehrssona, Frederick W. Williamsa, Daniel T. Gottukc a
Chemistry Division, US Naval Research Laboratory, Washington, DC 20375, USA b Nova Research, Inc., Alexandria, VA 22308, USA c Hughes Associates, Inc., Baltimore, MD 21227, USA
Received 3 March 2005; received in revised form 24 October 2005; accepted 15 November 2005 Available online 23 March 2006
Abstract This paper describes progress using filtered, long wavelength video image-based detection (LWVD) of events in laboratory tests and full scale fire testing within the Volume Sensor Program at the U.S. Naval Research Laboratory (NRL). This effort toward developing a real-time, remote sensing detection system utilizes video image detection (VID) systems based on cameras that operate in the visible region, which were developed for detecting smoke and have recently been adapted to detecting fire. However, VID systems are not effective at detecting fire outside the direct line of sight of the camera. Our studies demonstrate that long wavelength imaging achieves effective detection of reflected flame emission compared to visible video images. A system that combines visible and long wavelength image capabilities may be more accurate and sensitive than either alone. Our LWVD approach exploits the long wavelength response of standard CCD arrays used in many cameras. A long pass filter (typically in the range 700–900 nm) increases the contrast for flaming and hot objects and suppresses the normal video image of the space, thereby effectively providing a degree of thermal imaging. There is more emission from hot objects in this spectral region than in the visible region (o600 nm). Testing has demonstrated the detection of objects heated to 400 1C or higher. A simple luminosity-based algorithm was developed and used to evaluate camera/filter combinations for fire, smoke and nuisance (false) event detection and response times. r 2006 Elsevier Ltd. All rights reserved. Keywords: Video image fire detection; Fire detection; Near infrared; Long wavelength; Camera; Flame detection
1. Introduction Long wavelength video image-based detection (LWVD) of fire has been developed in the Volume Sensor (VS) Program at the U.S. Naval Research Laboratory (NRL). In the VS Program, which is a component of the Advanced Damage Countermeasures Program in the Platform Protection Future Naval Capability, the primary goal is to employ optical detection methods for inexpensive, remote and real-time monitoring of ship spaces, including the detection of fire and smoke. Previous efforts at NRL have demonstrated significant improvements in accuracy, sensitivity and response time in fire and smoke detection using multicriteria approaches which combine various Corresponding author
E-mail address: [email protected] (J.C. Owrutsky). 0379-7112/$ - see front matter r 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.firesaf.2005.11.011
sensors and utilize neural network algorithms [1,2]. The response time of these methods, however, is inherently limited by their reliance on point detectors (e.g., of heat, smoke or gases, such as CO or CO2), which depend on molecular or thermal diffusion from the fire or smoke source to the sensor. Optical sensors do not depend on diffusion, so they can provide stand-off detection and possibly faster response times to more quickly identify fires or other hazardous conditions. The central element of the VS Program is the identification of fire, smoke and other conditions using video image detection (VID) systems in which the images are analyzed by machine vision algorithms. Progress in this area includes testing and evaluation of commercial systems [3,4]. These video fire detection systems are most effective at identifying smoke and direct line of sight (LOS) fires and are less successful for identifying small or obstructed flames from
ARTICLE IN PRESS 316
J.C. Owrutsky et al. / Fire Safety Journal 41 (2006) 315–320
reflected emission or hot objects. Video detection systems can respond quickly, but a common limitation is that they require a direct LOS between the source and detector for optimum performance. It is not feasible for practical or economical reasons to deploy enough cameras to ensure complete direct LOS coverage throughout the compartment space, especially in crowded spaces that are typical for naval ships. A possible remedy being explored is to detect reflected radiation as described below using LWVD. In addition, other kinds of sensors, including acoustic and single element optical detectors, are being incorporated into the VS system as described elsewhere [4]. The LWVD is one component of the multisensor, data fusion-based VS system currently under development at NRL. As currently conceived for deployment on future U.S. Navy ships, there could be one or more VS system in each compartment of the ship, therefore it is important to minimize the unit cost. The goal of developing an inexpensive yet effective system motivated using near infrared (NIR) imaging or LWVD. The LWVD uses commonly available, inexpensive cameras. These can be simple surveillance cameras (o$100) or commercial cameras (o$1000), both of which cost much less than mid infrared cameras (3–12 mm, 4$15,000). The latter are almost certainly better for thermal imaging. For example, they have a much lower minimum detectable temperature and can distinguish between body temperature and room temperature. Although mid infrared cameras outperform LWVD, the latter provides a desirable trade off between cost and performance for the applications described here. Long wavelength or NIR emission radiation detectors have been used previously for fire detection. Lloyd et al. reported [5,6] and patented [7] approaches intended to detect reflected near infrared emission using several narrowband detectors without any imaging capability. The results demonstrated the feasibility of detecting reflected NIR emission. In addition, NIR image detection has been applied in background free environments, such as for monitoring forest fires from terrestrial-based [8] and satellite images [9], tunnels [10], as well as aircraft cargo surveillance [11]. The latter study included a detailed characterization using CCD cameras operating in the NIR for remote temperature measurement and demonstrated a minimum detectable temperature of about 350 1C. NIR detection of forest fires and other quiescent environments is effective in part because there are few interferences or nuisances to complicate the detection. A narrowbandfiltered (1140 nm) NIR imaging technique that includes image analysis has been patented as a method for improved fire and hot object detection by reducing false alarms in protected areas [12]. In our LWVD approach to fire detection, we use a long pass filter (LPF) positioned in front of a standard CCD array camera. The LPF transmits light with wavelengths longer than a cutoff, typically between 700 and 900 nm, which increases the contrast for fire, flame and hot objects by suppressing the normal video images of the space. This
provides modest thermal imaging because there is more emission from hot objects in this spectral region than in the visible (o600 nm). Videos of long wavelength emission are demonstrated to be particularly beneficial for providing high contrast and therefore straightforward LOS detection for flames and hot objects, where the latter includes bulkheads heated by obstructed fires, and for identifying obstructed fire and flame based on reflected light. 2. Approach The LWVD method is implemented using long wavelength images acquired in real time and digitally recorded with various long pass filters (typically, 720–850 nm) using both a Sony camcorder (DCR-TRV27) in NightshotTM mode and inexpensive ‘‘bullet’’ cameras (Si-SPECO (CVC130R)). Cameras that operate in this wavelength region are often referred to as nightvision cameras and we use this nomenclature in this paper. This report will focus on results from full scale fire testing in the CVNX and VS1 test series [13] conducted aboard the ex-USS SHADWELL in April 2003 and the VS2 test series conducted at the facilities of Hughes Associates, Inc., in Baltimore, MD in 2003–2004 [14]. The video signal from a nightvision camera was converted from analog to digital video (DV) format with a Firewire video adapter (Dazzle Hollywood DV Bridge) or to digital .AVI format with a USB video adapter (Belkin USB VideoBus II) for suitable input into a computer. A program coded in Mathworks’ numerical analysis software suite, MATLAB v6.5 (Release 13), was used to control the video input acquisition from the cameras and to analyze the video images. The analysis was carried out using a straightforward luminosity-based algorithm developed at NRL explicitly for analysis of nightvision images [13]. The luminosity algorithm was designed and implemented to capture the enhanced sensitivity of the nightvision cameras to the thermal emission of fires, hot objects, and especially flame emission reflected off walls and around obstructions from a source fire not in the field of view (FOV) of the camera. A brief outline of the algorithm is presented here; a more detailed description of the luminosity algorithm is available in an NRL memorandum report [13]. The LWVD algorithm detects fire events by comparing the luminosity, L, which is the sum of the pixel intensities, of the current video frame to the sum of a reference luminosity, Lb, and an alarm threshold, Lth. The algorithm tracks the number of frames with L4Lb+Lth and then alarms when a persistence criteria is met. Application of the algorithm in CVNX and VS1 tests has shown that a fire event generally increases the luminosity of a video frame by an amount independent of the background illumination. The reference luminosity is chosen as the luminosity of a frame 30 s from the beginning video acquisition or of the .AVI file, generally early enough in the test to consist entirely of background features. To mitigate
ARTICLE IN PRESS J.C. Owrutsky et al. / Fire Safety Journal 41 (2006) 315–320
317
the effects of large variations observed in the background luminosity, a nonlinear relationship between the reference Lb and the alarm threshold is used: pffiffiffiffiffiffiffi Lth ¼ 2 Lb which yields proportionally smaller thresholds for larger background luminosities. Persistence of the L4Lb+Lth condition is used to discriminate against spurious bright nuisances such as a flash of light or a reflective object rapidly moving through the space. A frame with L4Lb+Lth will increment the alarm count, while a frame with LoLb þ Lth will decrement this count, but never to a value less than zero. An event alarm is generated when the alarm count reaches 75. Given the rate of video frame processing, the algorithm’s minimum response time for event detection is 5 s. A maximum response, or reset alarm, time is not necessary for the analysis of recorded tests. Alarms were indicated in real time and alarm times were recorded to files for later retrieval and compilation into a database. A bitmap background video image was also stored at the start of each test. Luminosity time series data were recorded for the entire test.
Fig. 1. Camera video from a test on the ex-USS SHADWELL. Regular and nightvision still images before and during a flaming event: (a) regular camera at start of test; (b) nightvision camera at start of test; (c) regular camera during period of open flame just outside the camera FOV; and (d) nightvision camera during period of open flame within the camera FOV.
3. Results The CVNX, VS1, and VS2 test series have produced a ‘library’ of more than 300 tests of fire and smoke sources and nuisances in real or simulated ship compartments. These tests have included heated bulkheads due to fires in the adjacent spaces, flaming fires, both in and out of the field of view, smoldering sources and various nuisance events, such as grinding, welding, and activity and motion within the compartment. Evaluation of this ‘library’ using LWVD has shown that the LWVD approach is useful for detection of fire and smoke sources typical of US Navy environments, especially for reflected fire emission and hot objects. Also, LWVD-detected luminosity was correlated with thermocouple temperature measurements of a heated bulkhead to demonstrate detection of objects hotter than 400 1C. Several general capabilities of the LWVD method were apparent from visual inspection of the video streams as the tests were being conducted. First, flaming fires are detected with greater sensitivity with filtered nightvision cameras than with regular cameras because there is more emission from hot objects at the longer wavelengths detected by the nightvision cameras. NIR emission from flames is easily visible to the nightvision cameras, which is not always the case for regular video cameras. The point is demonstrated in Fig. 1, which consists of several panels of images extracted from the videos from a test. Panels (a) and (b) show images from a test aboard the ex-USS SHADWELL for the collocated regular cameras and filtered nightvision cameras, respectively, prior to source ignition. The images in panels (c) and (d) are from the same cameras several minutes later while the cardboard box flaming source is
burning in the lower right-hand corner, within the camera FOV for the nightvision camera and just out of the camera FOV for the regular camera. The flame is evident in both types of video. Emission from the flame can be seen on the surface of the nearest cabinet in the regular video image, but a more dramatic change is observed in the nightvision camera image, in which the lower right-hand quadrant is brightly illuminated. Although this example is somewhat biased because the fire is in the FOV of the nightvision camera and not the regular camera, it nevertheless demonstrates the high sensitivity of filtered LWVD. The LWVD images are more informative so that less is required of the image analysis for detection and identification. A simple luminosity algorithm would be much less effective for regular video images. Another example is shown in Fig. 2 for a source that is completely outside the FOV of all cameras. The source for this test was several cardboard boxes placed on the deck below and behind the FOV of the camera. Panels (a) and (b) show images obtained prior to ignition of the source from the regular and nightvision cameras, respectively. The images in panels (c) and (d) were acquired several minutes after ignition when the source was fully engulfed in flame. Little or no difference can be seen between the regular images, with the exception of what appears to be smoke in the upper left-hand portion of the image. There is, however, a marked difference between the two nightvision images. NIR emission from the flame illuminates the entire area within the nightvision camera FOV. In the nightvision video, the NIR illumination fluctuates with the same temporal profile as the flame itself. This suggests that reflected NIR light could be used to detect flames that are
ARTICLE IN PRESS 318
J.C. Owrutsky et al. / Fire Safety Journal 41 (2006) 315–320
Fig. 2. Regular and nightvision images before test ignition and during a flaming event outside the camera FOV: (a) regular camera at start of test; (b) nightvision camera at start of test; (c) regular camera during period of open flame outside the camera FOV; and (d) nightvision camera during period of open flame within the camera FOV.
out of the camera FOV based on time-series analysis of the camera video alone. A second general observation is the ability of the nightvision cameras to act as an inexpensive form of thermal-imaging camera. In fact, they worked so well that during the CVNX test series, the ex-USS SHADWELL control room crew used the nightvision cameras as monitors for the progress of tests involving fires in adjacent compartments. For several tests in the CVNX test series, heptane spray fires were set up outside the test compartment and directed at a test compartment bulkhead to simulate a fire in an adjacent compartment. If the bulkhead behaves similar to a blackbody emitter, the NIR emission from the bulkhead should increase with increasing temperature. In addition to monitoring the luminosity with the LWVD system, the bulkhead temperature was measured with thermocouples attached to the bulkhead. Fig. 3 contains several still images from the nightvision camera as the bulkhead temperature increased during the test. NIR emission of the bulkhead is seen to grow in both size and intensity as the temperature increased. Before ignition (in panel (a)), the image is dark. After 279 s as shown in panel (b), a small illuminated region is seen and the maximum temperature of the bulkhead is 370 1C. In (c) and (d), 737 and 1243 s after ignition, respectively, the luminosity is higher and the temperatures measured are 540 and 565 1C. There is no indication from the regular video images (not shown) that the bulkhead temperature is elevated. Hot object identification is a capability of LWVD that cannot be achieved with regular video. The results of a hot bulkhead test for which images are shown in Fig. 3 were used to estimate the minimum
Fig. 3. Still images of nightvision video from test of a hot bulkhead before and at three times during the test: (a) nightvision camera at start of test; (b) nightvision camera 00:04:39 into test; (c) nightvision camera 00:12:17 into test; (d) nightvision camera 00:20:43 into test, effectively the end of the test.
detectable temperature (Tmin) of the LWVD system using the luminosity algorithm. Tmin was found to be about 400 1C by comparing the luminosity (intensity) measured for a small area of the images to the actual bulkhead temperature as recorded by thermocouples (TC). The results for the shipboard test were verified in laboratory measurements carried out using a blackbody emission source using the LWVD system and a narrowband-filtered (at 800 nm) single element detector. The Tmin of about 400 1C is similar to the value reported in the study by Sentenac et al. [11] using CCD camera temperature sensing for hangar bay surveillance. A final general observation is that there are tradeoffs in flame and hot object detection with smoke detection, depending on the LPF used. While the nightvision cameras are more sensitive to flames and other hot objects when longer wavelength LPFs are used, they suppress more of the visible image and increase the contrast for NIR emission, and consequently they degrade the sensitivity for detecting smoke. There is actually some flexibility in how one might utilize the enhanced NIR sensitivity in imaging applications. An important consideration in terms of cost, sensitivity and coverage is how many cameras to use. For a single camera version, increased sensitivity in the NIR while retaining the image for smoke detection would be implemented with light filtering (shorter wavelength LPF, perhaps 650–750 nm). An approach that would yield more sensitivity albeit with more complexity would be to use two cameras, one in the visible for smoke and a heavily (4900 nm) filtered nightvision camera for better contrast and more effective hot object and reflected flame detection. Another advantage to heavy filtering is that it could be used in spaces where privacy is an issue (e.g., berthing compartments), i.e., where visible image collection may not be desirable/allowable, but the availability of video fire protection would still be beneficial.
ARTICLE IN PRESS J.C. Owrutsky et al. / Fire Safety Journal 41 (2006) 315–320 Table 1 Summary of NRL LWVD for the VS1 test series Regular video
Total tests Total alarms Correct alarms Percentage (%) False alarm Percentage (%)
LWVD
VID1
VID2
VID3
40 34 27 96 7 21
40 23 20 71 3 13
40 3 3 11 0 0
40 26 24 86 2 8
23 13 57
3 3 100
26 14 54
Mutual alarms LWVD faster? Percentage (%)
Table 2 Summary of NRL LWVD for the CVNX test series Total tests LWVD alarms Correct alarms Percentage correct alarm
22 18 18 82%
Total tests VID1 alarms Correct alarms Percentage correct alarm
24 11 11 50%
No. of mutual alarms LWVD faster? Percentage
8 7 88%
For a more quantitative assessment, test results are given in Table 1 for a comparison of the alarm statistics and relative response times from the nightvision camera/ LWVD systems with those for the three regular camera/ commercial VID systems that were also evaluated during the VS1 test series [13]. The VIDS are generically referenced as VID1, VID2 and VID3 in this paper. See Ref. [13] for further details. The LWVD system performed well, with a 96% correct alarm percentage (No. of correct alarms/No. of non-nuisance sources) for the VS1 test series and a 21% false alarm rate (No. of false alarms/Total no. of alarms). The LWVD appears to be more sensitive to flaming and smoldering sources, but suffers from correspondingly higher false alarm sensitivity. For sources where more than one system went into alarm, the LWVD system reached an alarm condition faster than the commercial systems at least 50% of the time for the VID1 and the VID3 systems and 100% of the time for the VID2 system. The VID2 system alarmed on only three tests, so the statistical significance of that percentage is unclear. Table 2 presents the summary results for the nightvision/LWVD and regular/VID1 systems from the CVNX test series. The LWVD detected 18 while VID1 detected eleven out of the 22 total source fires, or 82 and 50% detection, respectively. There were no nuisance sources in the CVNX test matrix, so the nuisance rejection was not measured for the CVNX test series. For tests where both systems went into alarm, the LWVD alarmed faster for 88% of the sources. Comparison
319
of the system response times for tests with mutual alarms indicate that the LWVD system, as configured, responded on average two to three times faster than the other VID systems for flaming fires and approximately two times slower for smoldering events based on the VS1 results. Caution is advised in drawing conclusions from these results, in particular for the smoldering events where the LWVD and regular cameras are not necessarily detecting the same smoke events. For example, the NIR emission of a cartridge heater used to initiate a smoldering source could be detected in addition to or instead of the smoke, as described in more detail in Ref. [13]. One should also note that the LWVDs nuisance rejection algorithm is very immature in comparison to the regular VIDS and the result of any improvements would most likely lead to slower response times. In the initial stages of the LWVD/VIDS evaluation, the nightvision camera video was analyzed offline with the LWVD algorithm and the regular camera video was analyzed in real-time by the commercial VIDS. Therefore, it was not clear whether performance differences were due primarily to the different cameras, the different algorithms or a combination of the two. One way to address this problem was to analyze the same video sources with all available algorithm/system combinations. At the request of NRL, the vendor for the VID1 system post-processed the nightvision video from the entire VS1 test series (tests VS11 through VS1-40) using their VID product. The results are detailed in Ref. [13] in addition to the VID1 analysis results of the regular video. For the VID1 fire algorithm operating on video from the small test compartment, the NIR and regular cameras had the same accuracy; 62.5% of the fire sources were detected correctly with 10% false alarms. In the medium-sized test compartment, 100% of the sources were correctly detected with the nightvision video while the regular video correctly detected only 12.5%. For the VID1 smoke algorithm, the correctly detected events were 23.5% and 88.2%, respectively for the NIR and the regular cameras in the small test compartment with false alarm percentages of 50% and 0%. In the medium-sized test compartment, the smoldering event detection percentages were 55.6% and 83.3% correct detection and 45% and 0% false alarms for the NIR and regular cameras, respectively. The assessment by the vendor agrees with the results of the NRL LWVD algorithm analysis and also with basic inspection of the video. Compared to regular cameras, NIR cameras are better at flaming event detection and worse at smoldering event detection. 4. Conclusions LWVD offers an attractive and cost-effective augmentation to the standard implementation of VIDS technologies as seen in currently available commercial systems. As with other optical detection methods, LWVD provides a remote, stand-off sensing capability, so that it can in principle respond more quickly than point-type detectors
ARTICLE IN PRESS 320
J.C. Owrutsky et al. / Fire Safety Journal 41 (2006) 315–320
that rely on diffusive transport to operate. The NRL LWVD system emphasizes detection of fire and hot objects both within the camera FOV and outside the camera FOV. The NIR radiation from flaming and hot objects is sufficiently intense in the observation band of the nightvision cameras (700–1000 nm) to quickly detect fires and hot objects such as overheated cables and ship bulkheads heated by a fire in an adjacent compartment. As part of the NRL LWVD system, a simple total luminosity-based, machine vision algorithm has been developed to support the nightvision camera development. Since LPFs suppress the visible image detected with nightvision cameras, smoke is not easily detected by the LWVD system. The LWVD system already shows promise as an inexpensive pseudothermal imaging system for remote monitoring of surface temperatures. Both spatial and time-series analysis will be incorporated into an enhanced version of the LWVD algorithm in a parallel effort to the planned testing. Acknowledgements Support for this work provided by the Office of Naval Research, Future Naval Capabilities. References [1] Rose-Pehrsson SL, Hart SJ, Street TT, Williams FW, Hammond MH, Gottuk DT, et al. Early warning fire detection system using a probabilistic neural network. Fire Technol 2003;39:147–71. [2] Pfister G. Multisensor/multicriteria fire detection: a new trend rapidly becomes state of the art. Fire Technol Second Quart 1997;33:115.
[3] Gottuk DT, Lynch JA, Rose-Pehrsson SL, Owrutsky JC, Williams FW. AUBE ‘04 Proceedings of the eleventh international conference on automatic fire detection. Duisburg, Germany. [4] Rose-Pehrsson SL, Owrutsky, JC, Wales, SC, Williams, FW, Farley, JP, Steinhurst, DA, et al. AUBE ‘04 Proceedings of the eleventh international conference on automatic fire detection. Duisburg, Germany. [5] Zhu Y-J, Lloyd A, Sivathanu Y, Gore J. Experimental and numerical evaluation of a near infrared fire detector. In: Slaughter KC, editor. Fire research and engineering second (2nd) international conference (ICFRE2). 1998. p. 512. [6] Lloyd AC, Zhu YJ, Tseng LK, Gore JP, Sivanthanu YR. Fire detection using reflected near infrared radiation and source temperature discrimination. NIST GCR 1998;98:747. [7] Sivathanu Y, Joseph RK, Tseng L, Gore JP, Lloyd A. Flame and smoke detector. United States patent 6,111,511. 29 August 2000. [8] Thomas PJ. Near-infrared forest-fire detection concept. Appl Opt 1993;32:5348. [9] Lasaponara R, Cuomo V, Macchiato MF, Simoniello T. A selfadaptive algorithm based on AVHRR multitemporal data analysis for small active fire detection. Int J Remote Sens 2003;24:1723. [10] Wieser D, Brupbacher T. Smoke detection in tunnels using video images. NIST SP 965. February 2001. [11] Sentenac T, Le Maoult Y, Orteu J- J. Evaluation of a charge-coupleddevice-based video sensor for aircraft cargo surveillance. Opt Eng 2002;41:796–810. [12] Chan WS, Burge JW. Imaging flame detection system. United States patent 5,937,077. 10 August 1999. [13] Steinhurst DA, Minor CP, Owrutsky JC, Rose-Pehrsson SL, Gottuk DT, Williams FW, et al. Long wavelength video-based event detection preliminary results from the CVNX and VS1 test series, ex-USS SHADWELL 7–25 April 2003. NRL/MM-MR/6110—038733. 31 December 2003. [14] Lynch JA, Gottuk DT, Rose-Pehrsson SL, Owrutsky JC, Steinhurst DA, Minor CP, et al. Volume sensor development test series 2— lighting conditions camera settings and spectral and acoustic signatures. NRL/MR/6180—04-8843. 24 November 2004.