- Email: [email protected]
Visual inspection of a combustion process in a thermoelectric plant
Signal Processing 80 (2000) 1577}1589
Visual inspection of a combustion process in a thermoelectric plant Jorge S. Marques *, Pedro M. Jorge IST/ISR, Torre, Norte, Piso 7, Av. Rovisco Pais, 1049-001 Lisbon, Portugal ISEL, R. Cons. Emn& dio Navarro, 1949-014 Lisbon, Portugal Received 8 February 1999; received in revised form 20 December 1999
Abstract Infrared images provide useful information to inspect the status of combustion processes. The #ame geometry and intensity depend on the combustion status and can be used for control and monitoring purposes. Flame segmentation is di$cult since the background intensity is sometimes higher than the #ame intensity, therefore requiring the use of sophisticated image analysis algorithms. This paper describes methods to analyze infrared images of industrial #ames and to characterize the #ame geometry. A segmentation algorithm is proposed to separate the #ame region from the background using an image formation model, a background model and the available shape information. Segmentation algorithms (e.g., active contours) usually assume solid objects with sharp boundaries. This is not true in the case of #ame images. The #ame is nonhomogeneous and it has a fuzzy boundary. To circumvent this di$culty multiple contours are used to characterize the #ame geometry. The #ame shape is then obtained by robust estimation methods, using a model of the image formation process inside the combustion chamber. The proposed algorithm is evaluated and used to monitor the #ame characteristics in a boiler of a thermoelectric plant. 2000 Elsevier Science B.V. All rights reserved. Zusammenfassung Infrarotbilder liefern nuK tzliche Information zur Untersuchung von VerbrennungsvorgaK ngen. Die Flammengeometrie und -intensitaK t haK ngen vom Zustand der Verbrennnung ab und koK nnen fuK r Steuer- und UG berwachungszwecke verwendet werden. Eine Flammensegmentierung ist schwierig, da die HintergrundintensitaK t manchmal hoK her ist als die FlammenintensitaK t. Deshalb werden komplizierte Bildanalysealgorithmen benoK tigt. In dieser Arbeit werden Methoden zur Analyse von Infrarotbildern industrieller Flammen und zur Charakterisierung der Flammengeometrie beschrieben. Es wird ein Segmentierungsalgorithmus zur Abgrenzung der Flammenregion vom Hintergrund vorgeschlagen, der ein Bildformungsmodell, ein Hintergrundmodell und verfuK gbare Gestaltinformation benuK tzt. Segmentierungsalgorithmen (z.B. mittels aktiver Konturen) beruhen uK blicherweise auf der Annahme fester Objekte mit scharfen Begrenzungen. Diese Annahmen tre!en auf Flammenbilder nicht zu. Die Flamme ist inhomogen mit unscharfer Begrenzung. Um diese Schwierigkeit zu umgehen, werden zur Charakterisierung der Flammengeometrie mehrfache Konturen verwendet. Die Flammenform wird dann mittels robuster SchaK tzmethoden erhalten, wobei ein Modell des Bildformungsprozesses innerhalb des Verbrennungsraums verwendet wird. Der vorgeschlagene Algorithmus wird ausgewertet und zur UG berwachung der Flammencharakteristik im Boiler eines WaK rmekraftwerks verwendet. 2000 Elsevier Science B.V. All rights reserved.
* Corresponding author. E-mail addresses: [email protected] (J.S. Marques), [email protected] (P.M. Jorge). 0165-1684/00/$ - see front matter 2000 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 5 - 1 6 8 4 ( 0 0 ) 0 0 0 5 7 - 8
1578
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
Re2 sume2 Les images infrarouges fournissent une information utile pour l'inspection de l'eH tat de processus de combustion. La geH omeH trie de la #amme et son intensiteH deH pendant de l'eH tat de la combustion et peuvent e( tre utiliseH es a` des "ns de contro( le et de surveillance. La segmentation de la #amme est di$cile car l'intensiteH de l'arrie`re-plan est quelquefois supeH rieure a` l'intensiteH de la #amme, ce qui impose l'utilisation d'algorithmes d'analyse d'image sophistiqueH s. Cet article deH crit des meH thodes d'analyse d'images infrarouges de #ammes industrielles et de caracteH risation de la geH omeH trie des #ammes. Un algorithme de segmentation, utilisant un mode`le de formation de l'image, un mode`le de l'arrie`re-plan et l'information de forme disponible, est proposeH pour seH parer la reH gion de #amme de l'arrie`re-plan. Des algorithmes de segmentation (tels que celui des contours actifs) font en geH neH ral l'hypothe`se d'objets solides ayant des frontie`res nettes. Ceci n'est pas vrai dans le cas d'images de #ammes. Une #amme et non-homoge`ne et a une frontie`re #oue. A"n de contourner cette di$culteH des contours multiples sont employeH s pour caracteH riser la geH omeH trie de la #amme. La forme de la #amme est obtenue ensuite a` l'aide de meH thodes d'estimation robustes, sur la base d'un mode`le du processus de formation de l'image a` l'inteH rieur de la chambre de combustion. L'algorithme proposeH est eH valueH et utiliseH pour surveiller les caracteH ristiques des #ammes dans la chaudie`re d'une centrale thermo-eH lectrique. 2000 Elsevier Science B.V. All rights reserved. Keywords: Image segmentation; Active contours; Visual inspection; Bayesian methods
1. Introduction Flame images are a useful source of information to evaluate the status of combustion processes in industrial plants. Most monitoring systems used in industry measure input variables characterizing the fuel, the temperature or the oxygen prior to the combustion. These are indirect measurements since they do not attempt to evaluate the combustion output: the chemical species and their spacial concentration inside the combustion chamber, the geometry of the #ame and the global e$ciency of the combustion process. Visual information (e.g., infra-red images) is used by specialists to tune the process and to periodically monitor its status (typically, once a month). Automatic measurement of #ame characteristics and its integration in the control loop is an active research area. A prototype tested in laboratory conditions is described in [5,23,24]. However, #ame measurement in largescale industrial facilities is still an open problem. Previous work in this direction can be found in [10,26]. Optical #ame measurement requires the segmentation of #ame images to separate the #ame from the background. This is a di$cult problem where classic image segmentation methods fail (e.g., thresholding, region growing [19]). Standard active contours methods which have found a wide spread application in object tracking (e.g., see [4,15]) do
not solve this problem either. The reason is simple. These methods were developed for the segmentation of solid objects and either they assume that the object is homogeneous or that it has sharp boundaries. This is not true in the case of industrial #ame images. The #ame is nonhomogeneous and it has a fuzzy boundary since the image intensity smoothly decreases from the #ame center to the periphery. It is not easy to de"ne a crisp boundary separating the #ame (space region where combustion occurs) from the background. Furthermore, the background intensity is often higher than the #ame intensity. For example, this happens inside boilers with walls illuminated by multiple #ames and covered with high-re#ectivity materials. This paper describes an algorithm for #ame segmentation in infra-red (IR) images based on two contours: an inner contour (boundary of the high intensity region) and an outer contour estimated in a Bayesian framework. The segmented #ame is then used to compute a set of geometric parameters: the #ame center, the #ame area and the distance from the #ame to the burner. A set of tests conducted at Barreiro thermoelectric plant, are presented to study the variation of the #ame parameters under controlled changes of the process variables. An air cooled infra-red camera was used to obtain images of the #ame inside a combustion chamber of the power plant. These images were obtained during the normal operation of the power plant, subject to
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
1579
the scheduled steam and energy instantaneous production. Therefore, they illustrate the application of the proposed methods with real data under industrial operating conditions. The paper is organized as follows. Section 2 discusses the geometric characterization of the #ame, Section 3 addresses #ame boundary extraction in a Bayesian framework and describes the #ame segmentation algorithm, Section 4 presents experimental results and Section 5 concludes the paper.
2. Geometric measurement of industrial 6ames The #ame shape is closely related to the spatial distribution of the chemical species produced during the combustion process. In industrial boilers, we can only visualize limited regions of the #ame from the available inspection windows. The inspection windows are usually located at the lateral walls of the combustion chamber. Typically, they are organized in rows and columns at di!erent distances from the top wall and at di!erent heights (see Fig. 1). Since the distance from the lenses to the #ame is much smaller than the #ame length, it is only possible to monitor part of the #ame. Furthermore, special cameras are needed to operate under such high temperatures even though the camera body is left outside the boiler (only the lenses are inside).
Fig. 1. Combustion chamber.
Fig. 2. Flame image.
Fig. 2 shows the image of an industrial #ame inside the boiler of a thermo-electric plant. This image and all the others used in this work were obtained with a cooled infra-red camera at Barreiro power plant. Fig. 2 displays the view from a lateral window near the top wall showing the burner and the #ame cone (beginning of the #ame). Changes in combustion inputs modify the geometry of the #ame. Two types of parameters will be measured: the distance from the #ame to the burner and the #ame shape, characterized by its center and area (see Fig. 3). The computation of these features raises the problem of deciding where is the #ame boundary. We shall consider two boundaries to characterize the #ame: an inner boundary, associated to the high-intensity region of the #ame, and an external boundary which de"nes the space region where combustion occurs. Each of these regions is characterized by a set of geometric measurements (area, center, distance to the burner) which can be used to monitor the combustion process. The estimation of the #ame boundaries is not simple. Inside the inner boundary, the image intensity is high (it is often saturated). However, this property is not enough to discriminate it from the
1580
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
3. Flame segmentation
Fig. 3. Geometric features.
other saturated regions of the image (e.g., the top wall or the other #ames inside the boiler) as shown in Fig. 4. The estimation of the external boundary is also di$cult since the di!erence between the #ame and the background is subtle. The next section describes a Bayesian segmentation method which tries to overcome these di$culties.
We wish to segment the #ame image into three regions: an inner region, an outer region and a transition region, which are characterized as follows. In the inner region only the #ame intensity is observed (the background has no in#uence inside this region). Its computation is performed as described in Section 2. In the outer region no #ame is observed. Inside this region, the image is not in#uenced by the #ame and all we can see is the top wall of the boiler and the burners (background). Finally, the transition region has a more complex structure since the image depends on the #ame intensity and on the background. We shall assume that the image intensity smoothly decays from the saturation level to the background image when we move from the inner boundary to the external boundary of the #ame. The #ame is characterized by these two boundaries. For the sake of simplicity, the #ame boundaries are de"ned by a "nite set of parameters u ,2, u+ , v ,2, v+ , (vG *uG ) which represent the distances of 2M boundary points to a reference point C chosen by the user (see Fig. 5). The boundary points are obtained by intersecting the #ame
Fig. 4. Thresholding results: (a) original image; (b) binary image.
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
1581
compute the estimate of the unknown parameters from the a posteriori probability p(x/I). This can be done in a number of ways, e.g., by using the mean (minimum mean-square error criterion) or the mode (maximum a posteriori criterion) of the distribution as estimates of the unknown parameters [21,25]. In this paper we shall adopt the maximum a posteriori (MAP) estimate of x de"ned by x( "arg max p(I/x)p(x) V
(2)
or by x( "arg max log p(I/x)#log p(x) V
Fig. 5. Boundary representation.
boundaries with M straight lines containing C, with equally spaced directions in the interval [!p, p[ (M"120 in all the experiments). For the sake of simplicity, the unknown parameters are organized in a sequence of 2D vectors x"(x , x ,2, x+ ), (1) where xG "(uG , vG ). The #ame boundaries de"ned in this way belong to a vector space of dimension 2M. Other parametric models have been used to represent plane curves, e.g., splines [7,16], Fourier shape models [22], landmark points [6,12]. The radial model used in this paper is however more adequate since it is able to e$ciently represent the #ame geometry and it allows to incorporate the a priori knowledge about the #ame shape and the hard restrictions imposed by the image boundaries. Flame segmentation is now converted into a parameter estimation problem: given an observed image I, what are the best contours (parameter con"guration) which separate the #ame from the background according to a given model. This problem is addressed in a Bayesian context, assuming that the image I and the con"guration of the coupled contours, x, are random variables with known probability distribution. Bayesian methods
(3)
provided that p(I/x)p(x)'0 for all x. In (3), p(I/x) is the probability density function of the observed image given the #ame boundary (image model) and p(x) is the shape prior which contains the a priori knowledge about the shape parameters. Related approaches were used in [8,9]. To compute the MAP estimate, an image model and a prior distribution have to be speci"ed, using the available information about the image statistics and the shape con"guration. This is the subject of the next two sections. 3.1. Image model The observed image contains two basic components: the #ame and the background (see Fig. 2). We shall distinguish three regions in the image where each of these components have a di!erent strength: the inner region, which is in#uenced by the #ame intensity only; the outer region which depends on the background intensity only and the transition region in#uenced by both the #ame and the background. Fig. 6 shows the intensity pro"le along a radial direction of Fig. 2, as a function of the distance to the reference point. The intensity pro"le is saturated in the interval [0,40]. This interval belongs to the inner region of the #ame. Then the intensity pro"le decreases until it reaches the background level at vG "76. This point is the boundary between the transition region and the outer region. Finally, the image pro"le follows the background pro"le if the distance is increased.
1582
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
Fig. 6. Intensity pro"les: (a) observed data; (b) model (solid line) and background pro"le (dashed line).
We shall assume that the observed image I is the superposition of a deterministic image IM with a white noise image = with Gaussian distribution. Let IG , IM G and =G , be the intensity pro"les of these images along the ith direction. Then, IG "IM G #=G ,
(4)
where IM G is constant in the saturated region, it is equal to the background pro"le in the outer region and it evolves according to a straight line in the transition region, i.e.,
S,
s)uG , S!BG (vG ) IM G (s;xG )" S! (s!uG ), uG (s)vG , (5) vG !uG BG (s), s'vG , where BG is the intensity pro"le of the background in the ith direction, xG "(uG , vG ) de"ned the boundary of the transition region and s is the distance to C. Fig. 6b shows the model pro"le (solid line) with vG "76 and the background pro"le (dashed line) for a speci"c direction. A background image (see Fig. 7) was used to obtain the background pro"le. This example shows that there is a good agreement between the intensity pro"le and the model provided that the boundary parameters, uG , vG , are accurately estimated. To de"ne a probabilistic model for the observed data let us assume that the image I is well repre-
Fig. 7. Background image.
sented by the set of radial intensity pro"les, i.e., I"(I , I ,2, I+ ) and let us also assume independence of the intensity pro"les, i.e., + p(I/x)" p(IG /xG ). G
(6)
Since the ith intensity pro"le is the sum of a deterministic pro"le with Gaussian white noise, its
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
1583
density function is given by
1 p(IG /xG )a exp ! "I (s)!IM G (s;xG )" ds . 2p G
(7)
Therefore,
1 + log p(I/x)"K! "IG (s)!IM G (s;xG )" ds. (8) 2p G In practice, the intensity pro"les will be sampled at equally spaced points and the integrals will be replaced by sums of the sampled squared error. The background image is obtained by editing the average image of the combustion chamber under normal operation conditions. The chamber walls are partially ocluded by the #ame. Therefore, in the #ame region the walls are reconstructed using the available knowledge of the boiler geometry and interpolated intensity values. This task is done only once for each boiler window. 3.2. Contour model The image model is insu$cient to obtain meaningful #ame estimates. There is little information to discriminate the #ame from the background in some regions, e.g., in high-intensity regions of the top wall, illuminated by multiple #ames. To overcome this di$culty, we have to rely on the a priori information available about the #ame shape. We shall require that (i) both boundaries are smooth and (ii) the external boundary is close to an average #ame shape, de"ned by the user or estimated from a large set of images. A simple approach consists of assuming that the x, is an unidimensional Markov random "eld with Gibbs density function [2,11]
1 p(x)" exp ! < (x) , ! Z !
(9)
where Z"exp+! !
Fig. 8. Average external boundary.
are smooth and the external boundary is close to the average shape v , shown in Fig. 8. Zero-order and "rst-order cliques are used to model this information. Zero-order cliques are used to measure the distance of the #ame boundary with respect to the average shape. The potential of a zero-order clique CG "+vG , is de"ned by the Lorentzian function [3]
1 v !v G G
(10)
as shown in Fig. 9, where v G is the ith component of the average shape v and a is a scale parameter. The probability density function associated with a zero-order clique is 1 pG (vG )J 1#((vG !v G )/a)
(11)
which has longer tails than the ubiquitous Gaussian distribution. Therefore, outliers (boundary points far from the average con"guration v G ) have smaller in#uence on the estimates. This allows a robust estimation of the #ame boundary [14]. The average shape is de"ned by the user, using an
1584
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
relaxation algorithm which performs a minimization with respect to a single variable in each iteration. Let +i , i ,2, iR ,2, be a deterministic or random sequence of sites (indices), such that each site is repeated an in"nite number of times. At the tth iteration x( GR is obtained by minimizing the objective function with respect to xGR , keeping the other variables constant. Let iR "k. Then,
1 "I (s)!IM (s;xI )" ds x( I "arg minVI 2p I
(v !v ) I #log 1# I 2a
# b(uI !u I )#c(vI !v I ),
Fig. 9. Lorentzian function.
average image of the #ame under normal operating conditions. To increase the smoothness of the #ame contours, "rst-order cliques CG "+i!1, i, are used. The clique potentials are de"ned by
+ 1 vG !v G log p(x)"C! log 1# 2 a G !b(uG !uG\ )!c(vG !vG\ ).
(13)
3.3. Optimization The computation of the MAP estimates by (3) requires the optimization of a non-convex function with a large number of variables [20]. Several methods have been proposed to tackle this problem (e.g. Metropolis algorithm [17], Gibbs sampler [11] or iterated conditional modes [2]). We have chosen the iterated conditional modes (ICM) algorithm proposed by Besag in [2]. ICM is a deterministic
(14)
where x I "(u I , v I )"(x( I\ #x( I> ) and x( G , iOk, are the most recent shape estimates. In the spirit of the ICM algorithm we will optimize (14) with respect to uI "rst and with respect to vI afterwards. These are unidirectional optimization operations which can be easily performed (e.g., by sampling the objective function and searching for the minimum). The algorithm is initialized using the maximum likelihood estimate of the #ame boundary and recursion (14) is repeated until convergence is observed. Typically, convergence is obtained in less then 20 iterations in this problem.
4. Experimental results Experimental tests were carried out using a set of 3355 IR images obtained at Barreiro thermoelectric plant. The goal of these tests is twofold: to assess the performance of the #ame segmentation algorithm and to study the relationship between the #ame measurements (see Section 2) and the process variables to determine if the geometry of the #ame contains useful information for on-line monitoring of the combustion process. The images used in these tests characterize the operation of a combustion chamber under 15 different operating conditions. In each test, video sequences were obtained from four inspection windows close to the burners (see Fig. 1), i.e., the video sequences allow to characterize the operation of four burners under 15 operating conditions.
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589 Table 1 Reference variables < (mm/s)
O (%)
P (bar)
20
1.1
9.8
Table 1 shows the values of the fuel viscosity (< ), excess oxygen (O ) and atomization steam pressure (P ) used as reference. In each test, the power plant was controlled to modify one of these variables, keeping the others as invariant as possible. The regulation is not always possible since the tests were performed during the normal duty of the power plant and therefore subject to the restrictions imposed by power and steam instantaneous consumption. More information would be obtained if we had considered all possible combinations of the process variables, modi"ed with given step. A comprehensive study is done in [24] in a laboratory environment, for a process with two input variables (oxygen, gas). This methodology leads to a high number of tests which are useful but di$cult to perform in an industrial environment (here the problem becomes even harder due to the fact that a third input variable (fuel viscosity) is considered). Fig. 10 shows the estimates of the coupled boundaries obtained by the method described in this paper. The images show the #ame produced by four burners observed from close inspection windows. The #ame boundaries displayed in the "gure are in good agreement with the contours which were manually de"ned by a specialist. The importance of the prior model on the "nal estimates was con"rmed. Both the smoothness and the average shape prior play relevant roles. Without the average shape information the algorithm fails to produce meaningful estimates when the #ame is surrounded by a high-intensity background (e.g., near the burner) since there is not enough information to compute the boundary between both regions. The smoothness prior is also relevant since it avoids the ringing e!ects which would be present otherwise and helps to discriminate the #ame from the background. The prior parameters a,b,c were chosen by trial and test.
1585
The experimental results are insensitive to small changes of these parameters. In fact the same parameters were used in all the tests performed for all the windows, i.e., no adjustment was needed every time the experimental conditions or the camera position changed. A second set of tests was performed to study the relationship between the #ame parameters (distance to the burner, area and center of the #ame) and the process variables (fuel viscosity, excess oxygen and atomization steam pressure). Figs. 11a}d show the evolution of the #ame measurements as a function of the fuel viscosity. Each "gure displays the results obtained for with the inner boundary (dashed line) and with the external boundary (solid line). For the sake of simplicity the maximum of each parameter was normalized to 1. We can conclude from this "gure that the shape measurements change, specially in the case of the inner boundary, when the set point characteristics are modi"ed: when the viscosity decreases (increase of temperature), the #ame becomes closer to the wall (combustion starts earlier) and the visible area of the #ame increases. This evolution corresponds to the expected behavior of the #ame under a change of the fuel viscosity. The average values of the process variables during these experiments are shown in Table 2. The fuel viscosity increased as desired. The other variables which should have stayed constant show however small changes. These changes are hard to avoid since the production requirements vary during the tests. Experiments were also performed to evaluate the dependence of the #ame features on the other variables. It was observed in these tests that changes in the excess oxygen and, in a less extent, changes in the atomization steam pressure also produce noticeable modi"cations of the #ame measurements.
5. Conclusions The optical inspection of #ames was addressed as a tool for monitoring industrial processes. The advantage of this approach is to measure the output of the combustion process and its spatial distribution. This is a di$cult task for three main reasons:
1586
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
Fig. 10. Boundary estimates of four di!erent #ames.
(i) the process variables we wish to monitor are encoded in a complex way in the #ame geometry; (ii) we can only observe a small region of the #ame from a single vantage point, i.e., our measurements are local and (iii) the segmentation of the #ame images obtained inside a combustion chamber is a challenging image analysis problem.
This paper presents methods to estimate the shape of non-solid objects, including several novel features: E Multiple-model representation. The #ame geometry is represented by two contours which are used to represent the fuzzy boundary of the #ame. The contours are jointly estimated.
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
1587
Fig. 11. Flame measurements as a function of viscosity: (a) distance to the burner; (b) #ame center: horizontal displacement; (c) #ame center: vertical displacement; (d) #ame area (results obtained using the inner boundary (dashed line) and external boundary (solid line)). Table 2 Process variables Test
< (mm/s)
O (%)
P (bar)
1 2 3
20 27 37
1.1 1.0 1.1
9.8 10.0 9.7
E Image formation model. An image formation model is proposed in the paper to characterize the #ame image. Most works in the area of active contours do not use image formation models. They assume that the object boundary is associated with sudden intensity changes. This approach does not work in this problem since it is tailored for solid objects and not for transparent ones. E Robust estimation. Robust estimation is used to deal with the cluttered background. Non-Gaussian data distributions, with long tails, are used to reduce the in#uence of outliers. (Most active contours models are based on Gaussian assumptions. Recent works have addressed the robustness problem using complex nonlinear "ltering
methods such as the Condensation algorithm [4]. The method used in this paper is di!erent.) Flame is characterized by two contours: an inner contour bounding the high-intensity region of the #ame, and an external contour, de"ning the space region where combustion occurs. The geometric features used to characterize the #ame are easily computed from these boundaries which have to be estimated from the observed IR images. The computation of the #ame boundaries is di$cult since it cannot be achieved by simple thresholding or segmentation methods: the intensity of the inner region is similar to the intensity of the top wall and the di!erences between the #ame and the background in the vicinity of the external boundary, are subtle. In some locations, the two regions are virtually indistinguishable. A Bayesian approach is adopted in this paper to achieve a joint estimation of the #ame boundaries. An image model and a shape prior are required. The image model takes into account the information available about the background and the image formation noise. It is assumed that the #ame intensity is superimposed on a known background image. The #ame region is split into two subsets: a high-intensity region
1588
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
containing the #ame center and a transition region between the inner region and the background. This model provides an accurate description of the observed images. However, this information is not enough to obtain meaningful shape estimates. A shape prior is needed. For the sake of simplicity, it is assumed that the boundaries are uni-dimensional Markov random "elds with clique potentials chosen in order to obtain smooth contours and, in the case of the external boundary, the prior assigns high probabilities to shapes close to the average #ame shape. This prior makes the external boundary estimates follow the average contour when there is no information available in the observations (this happens at speci"c locations where the #ame is indistinguishable from the background). Lorentzian functions are used to de"ne the clique potentials in order to allow large deviations from the average contour. Experimental tests were carried out to evaluate the #ame segmentation algorithm and to characterize the relationship between the #ame measurements and the process variables. Theses tests were performed using a database of 3355 IR images obtained at Barreiro thermoelectric plant and characterize the operation of 4 burners (#ames) in 15 operating conditions. Good segmentation results were achieved with the proposed algorithm. The estimated boundary is free of artifacts and provides good estimates of both contours. The tests performed under di!erent conditions show that #ame characteristics are modi"ed by the variation of the fuel viscosity, the excess oxygen and, in less extent, by changes of the atomization steam pressure. These changes are more apparent in the case of the inner contour estimates. As an example, it was concluded that a reduction of the viscosity reduces the distance from the #ame to the burner. The outer boundary is useful when the whole #ame is observed. In this study, only a small part of the #ame is imaged (due to physical restrictions) and the inner boundary carries more information than the outer boundary in this context. This is a useful conclusion which can be extended to the cases where only a small part of the #ame is observed. In practice, we still have to compute both boundaries since they are simultaneously estimated
by the MAP method. The geometric characteristics of the #ame extracted from the IR images can be used to detect deviations with respect to speci"c operating conditions and may also be useful to discriminate normal operating conditions from abnormal or dangerous ones.
Acknowledgements We thank the anonymous reviewers for helpful suggestions. This work was supported by CPPE under the contract CHAMA. The data-acquisition tests at Barreiro thermoelectric plant were planned and executed by the CPPE sta!. Special thanks are due to Engs. AntoH nio Gonc7 alves and Alfredo Parada of CPPE who conducted the tests, to Eng. QueiroH s dos Santos for support and helpful suggestions and to Prof. J. Miranda Lemos of IST who suggested this project and provided stimulating ideas.
References [1] Y. Aloimonos, A. Rosenfeld, Principles of computer vision, in: Handbook of Pattern Recognition and Image Processing, 1994. [2] J. Besag, On the statistical analysis of dirty picture, J. Roy. Stat. Soc. B 48 (1986) 259}302. [3] M. Black, A. Rangaranjan, The outlier process: unifying line processes and robust statistic, IEEE Proc. Comput. Vision Pattern Recognition (1994) 15}22. [4] A. Blake, M. Isard, Active Contours, Springer, Berlin, 1998. [5] H. Burkhardt, L. Oest, W. Tao, Vision-guided #ame control, SENSOR 95, Nurberg, 1995. [6] T. Cootes, C. Taylor, D. Cooper, J. Graham, Active shape models } their training and application, Comput. Vision Image Understanding 61 (1995) 38}59. [7] R. Curwen, A. Blake, Dynamic contours: real-time active splines, in: Active Vision, MIT Press, Cambridge, MA, 1992. [8] M. Figueiredo, J. Leita o, Bayesian estimation of ventricular contours in angiographic images, IEEE Trans. Med. Imaging 11 (3) (September 1992) 416}429. [9] N. Friedland, A. Rosenfeld, An integrated approach to 2D object recognition, Pattern Recognition 30 (3) (1997) 525}535. [10] B. Georgel, B. Lavaissiere, P. Jacques, J. Labarre, Flamenco: an image processing system for #ame analysis in a combustion chamber, Technical Report, EDF, 1995.
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589 [11] S. Geman, D. Geman, Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images, IEEE Trans Pattern Anal. Mach. Intell. 6 (6) (1984) 721}741. [12] U. Grenander, Y. Chow, D. Keenan, Hands, A Pattern Theoretical Study of Biological Shapes, Springer, Berlin, 1991. [13] B. Horn, Robot Vision, MIT Press, Cambridge, MA, 1986. [14] P. Huber, Robust Statistics, Wiley, New York, 1981. [15] M. Kass, A. Witkin, D. Terzopoulos, Snakes: active contour models, Int. J. Comput. Vision 1 (1987) 259}268. [16] S. Menet, P. Saint-Marc, G. Medioni, B-Snakes: implementation and applications to stereo, DARPA, 1990, pp. 720}726. [17] N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, E. Teller, Equation of state calculation by fast computing machines, J. Chem. Phys. 21 (1953) 1087}1091. [18] T. Poggio, V. Torre, C. Koch, Computational vision and regularization theory, Nature 317 (1985) 314}319. [19] W. Pratt, Digital Image Processing, Wiley, New York, 1991.
1589
[20] W. Press, S. Teukolsky, W. Vetterling, B. Flannery, Numerical Recipes in C, Cambridge University Press, Cambridge, 1994. [21] B. Ripley, Pattern Recognition and Neural Networks, Cambridge University Press, Cambridge, 1996. [22] L. Staib, J. Duncan, Deformable fourier models for surface "nding in 3D images, Conference On Visualisation in Biomedical Computing, SPIE, 1994, pp. 90}104. [23] W. Tao, H. Burkhardt, An e!ective image thresholding method using a fuzzy compactness measure, International Conference on Pattern Recognition, Vol. I, 1994, pp. 47}51. [24] W. Tao, H. Burkhardt, Vision-guided #ame control using fuzzy logic and neural networks (invited paper), Part. & Part. Systems Characterization 12 (1995) 87}94. [25] H. Van Trees, Detection, Estimation and Modulation Theory, Wiley, New York, 1968. [26] J. Santos-Victor, A system for the analysis and classi"cation of industrial #ames, M.Sc. Thesis, Instituto Superior Tecnico, 1991 (in Portuguese).
Visual inspection of a combustion process in a thermoelectric plant Jorge S. Marques *, Pedro M. Jorge IST/ISR, Torre, Norte, Piso 7, Av. Rovisco Pais, 1049-001 Lisbon, Portugal ISEL, R. Cons. Emn& dio Navarro, 1949-014 Lisbon, Portugal Received 8 February 1999; received in revised form 20 December 1999
Abstract Infrared images provide useful information to inspect the status of combustion processes. The #ame geometry and intensity depend on the combustion status and can be used for control and monitoring purposes. Flame segmentation is di$cult since the background intensity is sometimes higher than the #ame intensity, therefore requiring the use of sophisticated image analysis algorithms. This paper describes methods to analyze infrared images of industrial #ames and to characterize the #ame geometry. A segmentation algorithm is proposed to separate the #ame region from the background using an image formation model, a background model and the available shape information. Segmentation algorithms (e.g., active contours) usually assume solid objects with sharp boundaries. This is not true in the case of #ame images. The #ame is nonhomogeneous and it has a fuzzy boundary. To circumvent this di$culty multiple contours are used to characterize the #ame geometry. The #ame shape is then obtained by robust estimation methods, using a model of the image formation process inside the combustion chamber. The proposed algorithm is evaluated and used to monitor the #ame characteristics in a boiler of a thermoelectric plant. 2000 Elsevier Science B.V. All rights reserved. Zusammenfassung Infrarotbilder liefern nuK tzliche Information zur Untersuchung von VerbrennungsvorgaK ngen. Die Flammengeometrie und -intensitaK t haK ngen vom Zustand der Verbrennnung ab und koK nnen fuK r Steuer- und UG berwachungszwecke verwendet werden. Eine Flammensegmentierung ist schwierig, da die HintergrundintensitaK t manchmal hoK her ist als die FlammenintensitaK t. Deshalb werden komplizierte Bildanalysealgorithmen benoK tigt. In dieser Arbeit werden Methoden zur Analyse von Infrarotbildern industrieller Flammen und zur Charakterisierung der Flammengeometrie beschrieben. Es wird ein Segmentierungsalgorithmus zur Abgrenzung der Flammenregion vom Hintergrund vorgeschlagen, der ein Bildformungsmodell, ein Hintergrundmodell und verfuK gbare Gestaltinformation benuK tzt. Segmentierungsalgorithmen (z.B. mittels aktiver Konturen) beruhen uK blicherweise auf der Annahme fester Objekte mit scharfen Begrenzungen. Diese Annahmen tre!en auf Flammenbilder nicht zu. Die Flamme ist inhomogen mit unscharfer Begrenzung. Um diese Schwierigkeit zu umgehen, werden zur Charakterisierung der Flammengeometrie mehrfache Konturen verwendet. Die Flammenform wird dann mittels robuster SchaK tzmethoden erhalten, wobei ein Modell des Bildformungsprozesses innerhalb des Verbrennungsraums verwendet wird. Der vorgeschlagene Algorithmus wird ausgewertet und zur UG berwachung der Flammencharakteristik im Boiler eines WaK rmekraftwerks verwendet. 2000 Elsevier Science B.V. All rights reserved.
* Corresponding author. E-mail addresses: [email protected] (J.S. Marques), [email protected] (P.M. Jorge). 0165-1684/00/$ - see front matter 2000 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 5 - 1 6 8 4 ( 0 0 ) 0 0 0 5 7 - 8
1578
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
Re2 sume2 Les images infrarouges fournissent une information utile pour l'inspection de l'eH tat de processus de combustion. La geH omeH trie de la #amme et son intensiteH deH pendant de l'eH tat de la combustion et peuvent e( tre utiliseH es a` des "ns de contro( le et de surveillance. La segmentation de la #amme est di$cile car l'intensiteH de l'arrie`re-plan est quelquefois supeH rieure a` l'intensiteH de la #amme, ce qui impose l'utilisation d'algorithmes d'analyse d'image sophistiqueH s. Cet article deH crit des meH thodes d'analyse d'images infrarouges de #ammes industrielles et de caracteH risation de la geH omeH trie des #ammes. Un algorithme de segmentation, utilisant un mode`le de formation de l'image, un mode`le de l'arrie`re-plan et l'information de forme disponible, est proposeH pour seH parer la reH gion de #amme de l'arrie`re-plan. Des algorithmes de segmentation (tels que celui des contours actifs) font en geH neH ral l'hypothe`se d'objets solides ayant des frontie`res nettes. Ceci n'est pas vrai dans le cas d'images de #ammes. Une #amme et non-homoge`ne et a une frontie`re #oue. A"n de contourner cette di$culteH des contours multiples sont employeH s pour caracteH riser la geH omeH trie de la #amme. La forme de la #amme est obtenue ensuite a` l'aide de meH thodes d'estimation robustes, sur la base d'un mode`le du processus de formation de l'image a` l'inteH rieur de la chambre de combustion. L'algorithme proposeH est eH valueH et utiliseH pour surveiller les caracteH ristiques des #ammes dans la chaudie`re d'une centrale thermo-eH lectrique. 2000 Elsevier Science B.V. All rights reserved. Keywords: Image segmentation; Active contours; Visual inspection; Bayesian methods
1. Introduction Flame images are a useful source of information to evaluate the status of combustion processes in industrial plants. Most monitoring systems used in industry measure input variables characterizing the fuel, the temperature or the oxygen prior to the combustion. These are indirect measurements since they do not attempt to evaluate the combustion output: the chemical species and their spacial concentration inside the combustion chamber, the geometry of the #ame and the global e$ciency of the combustion process. Visual information (e.g., infra-red images) is used by specialists to tune the process and to periodically monitor its status (typically, once a month). Automatic measurement of #ame characteristics and its integration in the control loop is an active research area. A prototype tested in laboratory conditions is described in [5,23,24]. However, #ame measurement in largescale industrial facilities is still an open problem. Previous work in this direction can be found in [10,26]. Optical #ame measurement requires the segmentation of #ame images to separate the #ame from the background. This is a di$cult problem where classic image segmentation methods fail (e.g., thresholding, region growing [19]). Standard active contours methods which have found a wide spread application in object tracking (e.g., see [4,15]) do
not solve this problem either. The reason is simple. These methods were developed for the segmentation of solid objects and either they assume that the object is homogeneous or that it has sharp boundaries. This is not true in the case of industrial #ame images. The #ame is nonhomogeneous and it has a fuzzy boundary since the image intensity smoothly decreases from the #ame center to the periphery. It is not easy to de"ne a crisp boundary separating the #ame (space region where combustion occurs) from the background. Furthermore, the background intensity is often higher than the #ame intensity. For example, this happens inside boilers with walls illuminated by multiple #ames and covered with high-re#ectivity materials. This paper describes an algorithm for #ame segmentation in infra-red (IR) images based on two contours: an inner contour (boundary of the high intensity region) and an outer contour estimated in a Bayesian framework. The segmented #ame is then used to compute a set of geometric parameters: the #ame center, the #ame area and the distance from the #ame to the burner. A set of tests conducted at Barreiro thermoelectric plant, are presented to study the variation of the #ame parameters under controlled changes of the process variables. An air cooled infra-red camera was used to obtain images of the #ame inside a combustion chamber of the power plant. These images were obtained during the normal operation of the power plant, subject to
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
1579
the scheduled steam and energy instantaneous production. Therefore, they illustrate the application of the proposed methods with real data under industrial operating conditions. The paper is organized as follows. Section 2 discusses the geometric characterization of the #ame, Section 3 addresses #ame boundary extraction in a Bayesian framework and describes the #ame segmentation algorithm, Section 4 presents experimental results and Section 5 concludes the paper.
2. Geometric measurement of industrial 6ames The #ame shape is closely related to the spatial distribution of the chemical species produced during the combustion process. In industrial boilers, we can only visualize limited regions of the #ame from the available inspection windows. The inspection windows are usually located at the lateral walls of the combustion chamber. Typically, they are organized in rows and columns at di!erent distances from the top wall and at di!erent heights (see Fig. 1). Since the distance from the lenses to the #ame is much smaller than the #ame length, it is only possible to monitor part of the #ame. Furthermore, special cameras are needed to operate under such high temperatures even though the camera body is left outside the boiler (only the lenses are inside).
Fig. 1. Combustion chamber.
Fig. 2. Flame image.
Fig. 2 shows the image of an industrial #ame inside the boiler of a thermo-electric plant. This image and all the others used in this work were obtained with a cooled infra-red camera at Barreiro power plant. Fig. 2 displays the view from a lateral window near the top wall showing the burner and the #ame cone (beginning of the #ame). Changes in combustion inputs modify the geometry of the #ame. Two types of parameters will be measured: the distance from the #ame to the burner and the #ame shape, characterized by its center and area (see Fig. 3). The computation of these features raises the problem of deciding where is the #ame boundary. We shall consider two boundaries to characterize the #ame: an inner boundary, associated to the high-intensity region of the #ame, and an external boundary which de"nes the space region where combustion occurs. Each of these regions is characterized by a set of geometric measurements (area, center, distance to the burner) which can be used to monitor the combustion process. The estimation of the #ame boundaries is not simple. Inside the inner boundary, the image intensity is high (it is often saturated). However, this property is not enough to discriminate it from the
1580
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
3. Flame segmentation
Fig. 3. Geometric features.
other saturated regions of the image (e.g., the top wall or the other #ames inside the boiler) as shown in Fig. 4. The estimation of the external boundary is also di$cult since the di!erence between the #ame and the background is subtle. The next section describes a Bayesian segmentation method which tries to overcome these di$culties.
We wish to segment the #ame image into three regions: an inner region, an outer region and a transition region, which are characterized as follows. In the inner region only the #ame intensity is observed (the background has no in#uence inside this region). Its computation is performed as described in Section 2. In the outer region no #ame is observed. Inside this region, the image is not in#uenced by the #ame and all we can see is the top wall of the boiler and the burners (background). Finally, the transition region has a more complex structure since the image depends on the #ame intensity and on the background. We shall assume that the image intensity smoothly decays from the saturation level to the background image when we move from the inner boundary to the external boundary of the #ame. The #ame is characterized by these two boundaries. For the sake of simplicity, the #ame boundaries are de"ned by a "nite set of parameters u ,2, u+ , v ,2, v+ , (vG *uG ) which represent the distances of 2M boundary points to a reference point C chosen by the user (see Fig. 5). The boundary points are obtained by intersecting the #ame
Fig. 4. Thresholding results: (a) original image; (b) binary image.
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
1581
compute the estimate of the unknown parameters from the a posteriori probability p(x/I). This can be done in a number of ways, e.g., by using the mean (minimum mean-square error criterion) or the mode (maximum a posteriori criterion) of the distribution as estimates of the unknown parameters [21,25]. In this paper we shall adopt the maximum a posteriori (MAP) estimate of x de"ned by x( "arg max p(I/x)p(x) V
(2)
or by x( "arg max log p(I/x)#log p(x) V
Fig. 5. Boundary representation.
boundaries with M straight lines containing C, with equally spaced directions in the interval [!p, p[ (M"120 in all the experiments). For the sake of simplicity, the unknown parameters are organized in a sequence of 2D vectors x"(x , x ,2, x+ ), (1) where xG "(uG , vG ). The #ame boundaries de"ned in this way belong to a vector space of dimension 2M. Other parametric models have been used to represent plane curves, e.g., splines [7,16], Fourier shape models [22], landmark points [6,12]. The radial model used in this paper is however more adequate since it is able to e$ciently represent the #ame geometry and it allows to incorporate the a priori knowledge about the #ame shape and the hard restrictions imposed by the image boundaries. Flame segmentation is now converted into a parameter estimation problem: given an observed image I, what are the best contours (parameter con"guration) which separate the #ame from the background according to a given model. This problem is addressed in a Bayesian context, assuming that the image I and the con"guration of the coupled contours, x, are random variables with known probability distribution. Bayesian methods
(3)
provided that p(I/x)p(x)'0 for all x. In (3), p(I/x) is the probability density function of the observed image given the #ame boundary (image model) and p(x) is the shape prior which contains the a priori knowledge about the shape parameters. Related approaches were used in [8,9]. To compute the MAP estimate, an image model and a prior distribution have to be speci"ed, using the available information about the image statistics and the shape con"guration. This is the subject of the next two sections. 3.1. Image model The observed image contains two basic components: the #ame and the background (see Fig. 2). We shall distinguish three regions in the image where each of these components have a di!erent strength: the inner region, which is in#uenced by the #ame intensity only; the outer region which depends on the background intensity only and the transition region in#uenced by both the #ame and the background. Fig. 6 shows the intensity pro"le along a radial direction of Fig. 2, as a function of the distance to the reference point. The intensity pro"le is saturated in the interval [0,40]. This interval belongs to the inner region of the #ame. Then the intensity pro"le decreases until it reaches the background level at vG "76. This point is the boundary between the transition region and the outer region. Finally, the image pro"le follows the background pro"le if the distance is increased.
1582
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
Fig. 6. Intensity pro"les: (a) observed data; (b) model (solid line) and background pro"le (dashed line).
We shall assume that the observed image I is the superposition of a deterministic image IM with a white noise image = with Gaussian distribution. Let IG , IM G and =G , be the intensity pro"les of these images along the ith direction. Then, IG "IM G #=G ,
(4)
where IM G is constant in the saturated region, it is equal to the background pro"le in the outer region and it evolves according to a straight line in the transition region, i.e.,
S,
s)uG , S!BG (vG ) IM G (s;xG )" S! (s!uG ), uG (s)vG , (5) vG !uG BG (s), s'vG , where BG is the intensity pro"le of the background in the ith direction, xG "(uG , vG ) de"ned the boundary of the transition region and s is the distance to C. Fig. 6b shows the model pro"le (solid line) with vG "76 and the background pro"le (dashed line) for a speci"c direction. A background image (see Fig. 7) was used to obtain the background pro"le. This example shows that there is a good agreement between the intensity pro"le and the model provided that the boundary parameters, uG , vG , are accurately estimated. To de"ne a probabilistic model for the observed data let us assume that the image I is well repre-
Fig. 7. Background image.
sented by the set of radial intensity pro"les, i.e., I"(I , I ,2, I+ ) and let us also assume independence of the intensity pro"les, i.e., + p(I/x)" p(IG /xG ). G
(6)
Since the ith intensity pro"le is the sum of a deterministic pro"le with Gaussian white noise, its
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
1583
density function is given by
1 p(IG /xG )a exp ! "I (s)!IM G (s;xG )" ds . 2p G
(7)
Therefore,
1 + log p(I/x)"K! "IG (s)!IM G (s;xG )" ds. (8) 2p G In practice, the intensity pro"les will be sampled at equally spaced points and the integrals will be replaced by sums of the sampled squared error. The background image is obtained by editing the average image of the combustion chamber under normal operation conditions. The chamber walls are partially ocluded by the #ame. Therefore, in the #ame region the walls are reconstructed using the available knowledge of the boiler geometry and interpolated intensity values. This task is done only once for each boiler window. 3.2. Contour model The image model is insu$cient to obtain meaningful #ame estimates. There is little information to discriminate the #ame from the background in some regions, e.g., in high-intensity regions of the top wall, illuminated by multiple #ames. To overcome this di$culty, we have to rely on the a priori information available about the #ame shape. We shall require that (i) both boundaries are smooth and (ii) the external boundary is close to an average #ame shape, de"ned by the user or estimated from a large set of images. A simple approach consists of assuming that the x, is an unidimensional Markov random "eld with Gibbs density function [2,11]
1 p(x)" exp ! < (x) , ! Z !
(9)
where Z"exp+! !
Fig. 8. Average external boundary.
are smooth and the external boundary is close to the average shape v , shown in Fig. 8. Zero-order and "rst-order cliques are used to model this information. Zero-order cliques are used to measure the distance of the #ame boundary with respect to the average shape. The potential of a zero-order clique CG "+vG , is de"ned by the Lorentzian function [3]
1 v !v G G
(10)
as shown in Fig. 9, where v G is the ith component of the average shape v and a is a scale parameter. The probability density function associated with a zero-order clique is 1 pG (vG )J 1#((vG !v G )/a)
(11)
which has longer tails than the ubiquitous Gaussian distribution. Therefore, outliers (boundary points far from the average con"guration v G ) have smaller in#uence on the estimates. This allows a robust estimation of the #ame boundary [14]. The average shape is de"ned by the user, using an
1584
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
relaxation algorithm which performs a minimization with respect to a single variable in each iteration. Let +i , i ,2, iR ,2, be a deterministic or random sequence of sites (indices), such that each site is repeated an in"nite number of times. At the tth iteration x( GR is obtained by minimizing the objective function with respect to xGR , keeping the other variables constant. Let iR "k. Then,
1 "I (s)!IM (s;xI )" ds x( I "arg minVI 2p I
(v !v ) I #log 1# I 2a
# b(uI !u I )#c(vI !v I ),
Fig. 9. Lorentzian function.
average image of the #ame under normal operating conditions. To increase the smoothness of the #ame contours, "rst-order cliques CG "+i!1, i, are used. The clique potentials are de"ned by
+ 1 vG !v G log p(x)"C! log 1# 2 a G !b(uG !uG\ )!c(vG !vG\ ).
(13)
3.3. Optimization The computation of the MAP estimates by (3) requires the optimization of a non-convex function with a large number of variables [20]. Several methods have been proposed to tackle this problem (e.g. Metropolis algorithm [17], Gibbs sampler [11] or iterated conditional modes [2]). We have chosen the iterated conditional modes (ICM) algorithm proposed by Besag in [2]. ICM is a deterministic
(14)
where x I "(u I , v I )"(x( I\ #x( I> ) and x( G , iOk, are the most recent shape estimates. In the spirit of the ICM algorithm we will optimize (14) with respect to uI "rst and with respect to vI afterwards. These are unidirectional optimization operations which can be easily performed (e.g., by sampling the objective function and searching for the minimum). The algorithm is initialized using the maximum likelihood estimate of the #ame boundary and recursion (14) is repeated until convergence is observed. Typically, convergence is obtained in less then 20 iterations in this problem.
4. Experimental results Experimental tests were carried out using a set of 3355 IR images obtained at Barreiro thermoelectric plant. The goal of these tests is twofold: to assess the performance of the #ame segmentation algorithm and to study the relationship between the #ame measurements (see Section 2) and the process variables to determine if the geometry of the #ame contains useful information for on-line monitoring of the combustion process. The images used in these tests characterize the operation of a combustion chamber under 15 different operating conditions. In each test, video sequences were obtained from four inspection windows close to the burners (see Fig. 1), i.e., the video sequences allow to characterize the operation of four burners under 15 operating conditions.
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589 Table 1 Reference variables < (mm/s)
O (%)
P (bar)
20
1.1
9.8
Table 1 shows the values of the fuel viscosity (< ), excess oxygen (O ) and atomization steam pressure (P ) used as reference. In each test, the power plant was controlled to modify one of these variables, keeping the others as invariant as possible. The regulation is not always possible since the tests were performed during the normal duty of the power plant and therefore subject to the restrictions imposed by power and steam instantaneous consumption. More information would be obtained if we had considered all possible combinations of the process variables, modi"ed with given step. A comprehensive study is done in [24] in a laboratory environment, for a process with two input variables (oxygen, gas). This methodology leads to a high number of tests which are useful but di$cult to perform in an industrial environment (here the problem becomes even harder due to the fact that a third input variable (fuel viscosity) is considered). Fig. 10 shows the estimates of the coupled boundaries obtained by the method described in this paper. The images show the #ame produced by four burners observed from close inspection windows. The #ame boundaries displayed in the "gure are in good agreement with the contours which were manually de"ned by a specialist. The importance of the prior model on the "nal estimates was con"rmed. Both the smoothness and the average shape prior play relevant roles. Without the average shape information the algorithm fails to produce meaningful estimates when the #ame is surrounded by a high-intensity background (e.g., near the burner) since there is not enough information to compute the boundary between both regions. The smoothness prior is also relevant since it avoids the ringing e!ects which would be present otherwise and helps to discriminate the #ame from the background. The prior parameters a,b,c were chosen by trial and test.
1585
The experimental results are insensitive to small changes of these parameters. In fact the same parameters were used in all the tests performed for all the windows, i.e., no adjustment was needed every time the experimental conditions or the camera position changed. A second set of tests was performed to study the relationship between the #ame parameters (distance to the burner, area and center of the #ame) and the process variables (fuel viscosity, excess oxygen and atomization steam pressure). Figs. 11a}d show the evolution of the #ame measurements as a function of the fuel viscosity. Each "gure displays the results obtained for with the inner boundary (dashed line) and with the external boundary (solid line). For the sake of simplicity the maximum of each parameter was normalized to 1. We can conclude from this "gure that the shape measurements change, specially in the case of the inner boundary, when the set point characteristics are modi"ed: when the viscosity decreases (increase of temperature), the #ame becomes closer to the wall (combustion starts earlier) and the visible area of the #ame increases. This evolution corresponds to the expected behavior of the #ame under a change of the fuel viscosity. The average values of the process variables during these experiments are shown in Table 2. The fuel viscosity increased as desired. The other variables which should have stayed constant show however small changes. These changes are hard to avoid since the production requirements vary during the tests. Experiments were also performed to evaluate the dependence of the #ame features on the other variables. It was observed in these tests that changes in the excess oxygen and, in a less extent, changes in the atomization steam pressure also produce noticeable modi"cations of the #ame measurements.
5. Conclusions The optical inspection of #ames was addressed as a tool for monitoring industrial processes. The advantage of this approach is to measure the output of the combustion process and its spatial distribution. This is a di$cult task for three main reasons:
1586
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
Fig. 10. Boundary estimates of four di!erent #ames.
(i) the process variables we wish to monitor are encoded in a complex way in the #ame geometry; (ii) we can only observe a small region of the #ame from a single vantage point, i.e., our measurements are local and (iii) the segmentation of the #ame images obtained inside a combustion chamber is a challenging image analysis problem.
This paper presents methods to estimate the shape of non-solid objects, including several novel features: E Multiple-model representation. The #ame geometry is represented by two contours which are used to represent the fuzzy boundary of the #ame. The contours are jointly estimated.
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
1587
Fig. 11. Flame measurements as a function of viscosity: (a) distance to the burner; (b) #ame center: horizontal displacement; (c) #ame center: vertical displacement; (d) #ame area (results obtained using the inner boundary (dashed line) and external boundary (solid line)). Table 2 Process variables Test
< (mm/s)
O (%)
P (bar)
1 2 3
20 27 37
1.1 1.0 1.1
9.8 10.0 9.7
E Image formation model. An image formation model is proposed in the paper to characterize the #ame image. Most works in the area of active contours do not use image formation models. They assume that the object boundary is associated with sudden intensity changes. This approach does not work in this problem since it is tailored for solid objects and not for transparent ones. E Robust estimation. Robust estimation is used to deal with the cluttered background. Non-Gaussian data distributions, with long tails, are used to reduce the in#uence of outliers. (Most active contours models are based on Gaussian assumptions. Recent works have addressed the robustness problem using complex nonlinear "ltering
methods such as the Condensation algorithm [4]. The method used in this paper is di!erent.) Flame is characterized by two contours: an inner contour bounding the high-intensity region of the #ame, and an external contour, de"ning the space region where combustion occurs. The geometric features used to characterize the #ame are easily computed from these boundaries which have to be estimated from the observed IR images. The computation of the #ame boundaries is di$cult since it cannot be achieved by simple thresholding or segmentation methods: the intensity of the inner region is similar to the intensity of the top wall and the di!erences between the #ame and the background in the vicinity of the external boundary, are subtle. In some locations, the two regions are virtually indistinguishable. A Bayesian approach is adopted in this paper to achieve a joint estimation of the #ame boundaries. An image model and a shape prior are required. The image model takes into account the information available about the background and the image formation noise. It is assumed that the #ame intensity is superimposed on a known background image. The #ame region is split into two subsets: a high-intensity region
1588
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589
containing the #ame center and a transition region between the inner region and the background. This model provides an accurate description of the observed images. However, this information is not enough to obtain meaningful shape estimates. A shape prior is needed. For the sake of simplicity, it is assumed that the boundaries are uni-dimensional Markov random "elds with clique potentials chosen in order to obtain smooth contours and, in the case of the external boundary, the prior assigns high probabilities to shapes close to the average #ame shape. This prior makes the external boundary estimates follow the average contour when there is no information available in the observations (this happens at speci"c locations where the #ame is indistinguishable from the background). Lorentzian functions are used to de"ne the clique potentials in order to allow large deviations from the average contour. Experimental tests were carried out to evaluate the #ame segmentation algorithm and to characterize the relationship between the #ame measurements and the process variables. Theses tests were performed using a database of 3355 IR images obtained at Barreiro thermoelectric plant and characterize the operation of 4 burners (#ames) in 15 operating conditions. Good segmentation results were achieved with the proposed algorithm. The estimated boundary is free of artifacts and provides good estimates of both contours. The tests performed under di!erent conditions show that #ame characteristics are modi"ed by the variation of the fuel viscosity, the excess oxygen and, in less extent, by changes of the atomization steam pressure. These changes are more apparent in the case of the inner contour estimates. As an example, it was concluded that a reduction of the viscosity reduces the distance from the #ame to the burner. The outer boundary is useful when the whole #ame is observed. In this study, only a small part of the #ame is imaged (due to physical restrictions) and the inner boundary carries more information than the outer boundary in this context. This is a useful conclusion which can be extended to the cases where only a small part of the #ame is observed. In practice, we still have to compute both boundaries since they are simultaneously estimated
by the MAP method. The geometric characteristics of the #ame extracted from the IR images can be used to detect deviations with respect to speci"c operating conditions and may also be useful to discriminate normal operating conditions from abnormal or dangerous ones.
Acknowledgements We thank the anonymous reviewers for helpful suggestions. This work was supported by CPPE under the contract CHAMA. The data-acquisition tests at Barreiro thermoelectric plant were planned and executed by the CPPE sta!. Special thanks are due to Engs. AntoH nio Gonc7 alves and Alfredo Parada of CPPE who conducted the tests, to Eng. QueiroH s dos Santos for support and helpful suggestions and to Prof. J. Miranda Lemos of IST who suggested this project and provided stimulating ideas.
References [1] Y. Aloimonos, A. Rosenfeld, Principles of computer vision, in: Handbook of Pattern Recognition and Image Processing, 1994. [2] J. Besag, On the statistical analysis of dirty picture, J. Roy. Stat. Soc. B 48 (1986) 259}302. [3] M. Black, A. Rangaranjan, The outlier process: unifying line processes and robust statistic, IEEE Proc. Comput. Vision Pattern Recognition (1994) 15}22. [4] A. Blake, M. Isard, Active Contours, Springer, Berlin, 1998. [5] H. Burkhardt, L. Oest, W. Tao, Vision-guided #ame control, SENSOR 95, Nurberg, 1995. [6] T. Cootes, C. Taylor, D. Cooper, J. Graham, Active shape models } their training and application, Comput. Vision Image Understanding 61 (1995) 38}59. [7] R. Curwen, A. Blake, Dynamic contours: real-time active splines, in: Active Vision, MIT Press, Cambridge, MA, 1992. [8] M. Figueiredo, J. Leita o, Bayesian estimation of ventricular contours in angiographic images, IEEE Trans. Med. Imaging 11 (3) (September 1992) 416}429. [9] N. Friedland, A. Rosenfeld, An integrated approach to 2D object recognition, Pattern Recognition 30 (3) (1997) 525}535. [10] B. Georgel, B. Lavaissiere, P. Jacques, J. Labarre, Flamenco: an image processing system for #ame analysis in a combustion chamber, Technical Report, EDF, 1995.
J.S. Marques, P.M. Jorge / Signal Processing 80 (2000) 1577}1589 [11] S. Geman, D. Geman, Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images, IEEE Trans Pattern Anal. Mach. Intell. 6 (6) (1984) 721}741. [12] U. Grenander, Y. Chow, D. Keenan, Hands, A Pattern Theoretical Study of Biological Shapes, Springer, Berlin, 1991. [13] B. Horn, Robot Vision, MIT Press, Cambridge, MA, 1986. [14] P. Huber, Robust Statistics, Wiley, New York, 1981. [15] M. Kass, A. Witkin, D. Terzopoulos, Snakes: active contour models, Int. J. Comput. Vision 1 (1987) 259}268. [16] S. Menet, P. Saint-Marc, G. Medioni, B-Snakes: implementation and applications to stereo, DARPA, 1990, pp. 720}726. [17] N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, E. Teller, Equation of state calculation by fast computing machines, J. Chem. Phys. 21 (1953) 1087}1091. [18] T. Poggio, V. Torre, C. Koch, Computational vision and regularization theory, Nature 317 (1985) 314}319. [19] W. Pratt, Digital Image Processing, Wiley, New York, 1991.
1589
[20] W. Press, S. Teukolsky, W. Vetterling, B. Flannery, Numerical Recipes in C, Cambridge University Press, Cambridge, 1994. [21] B. Ripley, Pattern Recognition and Neural Networks, Cambridge University Press, Cambridge, 1996. [22] L. Staib, J. Duncan, Deformable fourier models for surface "nding in 3D images, Conference On Visualisation in Biomedical Computing, SPIE, 1994, pp. 90}104. [23] W. Tao, H. Burkhardt, An e!ective image thresholding method using a fuzzy compactness measure, International Conference on Pattern Recognition, Vol. I, 1994, pp. 47}51. [24] W. Tao, H. Burkhardt, Vision-guided #ame control using fuzzy logic and neural networks (invited paper), Part. & Part. Systems Characterization 12 (1995) 87}94. [25] H. Van Trees, Detection, Estimation and Modulation Theory, Wiley, New York, 1968. [26] J. Santos-Victor, A system for the analysis and classi"cation of industrial #ames, M.Sc. Thesis, Instituto Superior Tecnico, 1991 (in Portuguese).