- Email: [email protected]
Preserving brightness in histogram equalization based contrast enhancement techniques
Digital Signal Processing 14 (2004) 413–428 www.elsevier.com/locate/dsp
Preserving brightness in histogram equalization based contrast enhancement techniques Soong-Der Chen a,∗ , Abd. Rahman Ramli b a College of Information Technology, Universiti Tenaga Nasional, Km 7, Jalan Kajang-Puchong,
43009 Kajang, Selangor D.E., Malaysia b Department of Computer and Communication Systems Engineering, Faculty of Engineering,
Universiti Putra Malaysia, 43400 Serdang, Selangor D.E., Malaysia Available online 28 July 2004
Abstract Histogram equalization (HE) has been a simple yet effective image enhancement technique. However, it tends to change the brightness of an image significantly, causing annoying artifacts and unnatural contrast enhancement. Brightness preserving bi-histogram equalization (BBHE) and dualistic sub-image histogram equalization (DSIHE) have been proposed to overcome these problems but they may still fail under certain conditions. This paper proposes a novel extension of BBHE referred to as minimum mean brightness error bi-histogram equalization (MMBEBHE). MMBEBHE has the feature of minimizing the difference between input and output image’s mean. Simulation results showed that MMBEBHE can preserve brightness better than BBHE and DSIHE. Furthermore, this paper also formulated an efficient, integer-based implementation of MMBEBHE. Nevertheless, MMBEBHE also has its limitation. Hence, this paper further proposes a generalization of BBHE referred to as recursive mean-separate histogram equalization (RMSHE). RMSHE is featured with scalable brightness preservation. Simulation results showed that RMSHE is the best compared to HE, BBHE, DSIHE, and MMBEBHE. 2004 Elsevier Inc. All rights reserved. Keywords: Bi-histogram equalization; Dualistic sub-image; Histogram equalization; Mean separate; Minimum mean brightness error; Recursive
* Corresponding author.
E-mail address: [email protected] (S.-D. Chen). 1051-2004/$ – see front matter 2004 Elsevier Inc. All rights reserved. doi:10.1016/j.dsp.2004.04.001
414
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
1. Introduction Histogram equalization (HE) has been a very popular technique for enhancing the contrast of an image [1]. It has been applied in various fields such as medical image processing [2] and sonar image processing [3]. It remaps the gray levels of an image based on the image’s gray levels cumulative density function. As a result, it stretches the dynamic range of an image and improves the image’s contrast. Besides, HE also produces an image with flat histogram where the entire gray levels have uniform probability density as shown in Fig. 1, where p(x) denotes the probability density of the gray level, x ∈ {X0 , X1 , . . . , XL−1 } and L denotes the total number of discrete gray levels of the image. This is a useful feature as it maximizes the entropy of an image [4]. As shown by Kim [5], an image’s mean brightness after HE tends to be around the middle of the gray scale regardless of the original image’s mean. Let Y denotes the output image, E(·) is the statistical expectation or mean and XG is the middle of the gray scale. Then it follows that: 1 (1.01) E(Y) = XG = (X0 + XL−1 ). 2 While this is useful in application such as face detection for reducing brightness variance [6,7], it is not desirable in consumer electronics application (digital camera, TV, etc.) where preserving the original brightness is necessary to avoid annoying artifacts and unnatural enhancement [5]. For instance, Fig. 4(a) shows an original image hands while Fig. 4(c) shows the resultant image of HE that are composed of 256 gray levels. The equalized image is much brighter compared to the input image. The overall contrast of the image is also degraded after HE. Figures 5(a) and 5(c) show the original image F16, and the resultant image of HE, respectively. Unnatural enhancement can be seen around the cloud; one would perceive a totally different visual recognition around the cloud. Observed that the contrasts around the letters and the emblem on the airplane are also degraded. This is a direct consequence of the excessive change in brightness by HE. Brightness preserving bi-histogram equalization (BBHE) has been proposed to overcome the aforementioned problems [5]. Firstly, it separates an image’s histograms into two
Fig. 1. An image’s histogram before and after HE.
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
415
Fig. 2. Histogram before and after BBHE.
based on the image’s mean, Xm ∈ {X0 , X1 , . . . , XL−1 }. The two histograms are then equalized independently as illustrated in Fig. 2. The output image’s mean of BBHE is a function of the input image’s mean as shown by Kim [5] with the following equation: 1 E(Y) = (Xm + XG ). 2
(1.02)
This implies that BBHE has the feature of preserving an image’s original brightness, which is not found in typical HE. Figures 4(d) and 5(d) clearly demonstrate the effectiveness of BBHE. Later, equal area dualistic sub-image histogram equalization (DSIHE) has been proposed by Wan et al. [4] as an extension to BBHE. DSIHE selectively separate the histogram using threshold level with cumulative probability density equal to 0.5. The DSIHE is claimed to outperform BBHE in term of preserving an image’s brightness and entropy. Nevertheless, both BBHE and DSIHE may fail to enhance an image under certain conditions. Figures 6(d)–6(e), 7(d)–7(e), 8(d)–8(e), 9(d)–9(e), and 10(d)–10(e) show the resultant images of U2, arctic hare, copter, girl, and jet after BBHE and DSIHE, respectively. Their appearances clearly indicate that BBHE and DISHE fail to preserve their original brightness well. This paper aims to present some solutions to overcome the limitations of HE, BBHE, and DISHE in preserving an image’s original brightness. In what follows, the extension of BBHE, namely—minimum mean brightness error bi-histogram equalization (MMBEBHE), together with the formulation of its efficient integer-based implementation will be presented in Section 2. Section 3 lists a few simulation results to demonstrate the effectiveness of MMBEBHE. Next, the generalization of BBHE, namely—recursive meanseparate histogram equalization (RMSHE) will be presented in Section 4 together with the mathematical analysis. Section 5 lists a few simulation results to demonstrate the effectiveness of RMSHE. Section 6 serves as the conclusion of this paper.
416
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
2. Minimum mean brightness error bi-histogram equalization (MMBEBHE) An objective measurement is proposed to rate the performance in preserving the original brightness. It is referred to as absolute mean brightness error (AMBE) and is defined as the absolute difference between the input and the output image’s mean AMBE = E(X) − E(Y). (2.01) X and Y denotes the input and output image, respectively. Lower AMBE indicates that the brightness is better preserved. MMBEBHE, which perform bi-histogram equalization with minimum AMBE is also proposed. MMBEBHE is formally defined by the following procedures: (1) Calculate the AMBE for each of the possible threshold levels. (2) Find the threshold level, XT that yield minimum AMBE. (3) Separate the input histogram into two based on the XT found in Step 2 and equalize them independently as in BBHE. Steps 2 and 3 are straightforward processes. Step 1 would require considerable amount of computation if one full BBHE were required for computing the AMBE for each of the possible threshold levels. The computation complexity would then be a function of L2 . A 16-bits/pixel image would require up to at least (216 )2 operations. This could become a major drawback of MMBEBHE in real time implementation. This paper proposes a fast implementation of MMBEBHE. Equation (2.02) is the general definition for the mean brightness of an image after bi-histogram equalization based techniques such as BBHE, DSIHE, and MMBEBHE, where p(·) denotes probability density and XT denotes the selected threshold level E(Y) = E(Y | X XT )p(X XT ) + E(Y | X > XT )p(X > XT ).
(2.02)
Let us take the differentiation on both sides of Eq. (2.02): E(Y) = E(Y | X XT )p(X XT ) + E(Y | X XT )p(X XT ) + E(Y | X > XT )p(X > XT ) + E(Y | X > XT )p(X > XT ).
(2.03)
Equation (2.03) can be rewritten into a more useful form by applying the following facts. First, it has been shown by Kim [5] that 1 E(Y | X XT ) = (XT + X0 ), 2 1 E(Y | X > XT ) = (XL−1 + XT ). 2
(2.04) (2.05)
Second, for typical representation of digital gray scale image, it can be assumed that X = XT+1 − XT = 1. Then it follows that:
(2.06)
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
E(Y | X XT ) = E(Y | X XT+1 ) − E(Y | X XT ) 1 1 1 = (XT+1 + X0 ) − (XT + X0 ) = , 2 2 2 E(Y | X > XT ) = E(Y | X > XT+1 ) − E(Y | X > XT ) 1 1 1 = (XL−1 + XT+2 ) − (XL−1 + XT+1 ) = . 2 2 2 Third, it can be easily seen that
417
(2.07)
(2.08)
p(X XT ) + p(X > XT ) = 1,
(2.09)
p(X XT ) = p(X XT+1 ) − p(X XT ) = p(XT+1 ),
(2.10)
p(X > XT ) = p(X > XT+1 ) − p(X > XT ) = −p(XT+1 ).
(2.11)
Combining Eqs. (2.07)–(2.11) and Eq. (2.03), 1 + E(Y | X XT ) − E(Y | X > XT ) p(XT+1 ) 2 1 1 1 = − (XL−1 − X0 + 1)p(XT+1 ) = 1 − Lp(XT+1 ) . 2 2 2
E(Y) =
(2.12)
Let ET+1 (Y) and ET (Y) denote the output mean when threshold level is set as XT+1 and XT , respectively. Then, Eq. (2.03) can also be used to relate mean brightness error (MBE) as follows: E(Y) = ET+1 (Y) − ET (Y) = ET+1 (Y) − E(X) − ET (Y) − E(X) = MBET+1 − MBET . (2.13) Combining (2.12) and (2.13), MBET+1 = MBET +
1 1 − Lp(XT+1 ) . 2
(2.14)
Note that MBE0 = XG − E(X).
(2.15)
Equation (2.14) involves p(XT+1 ) which is a floating point number. In fact, the scaled MBE in the form of integer number is sufficient to determine the threshold level that yield minimum AMBE. Let us denote F (Xi ) as the number of pixel with gray level Xi and N as the total number of pixel in the image. It follows that: 1 (2N)MBET+1 = (2N) MBET + 1 − Lp(XT+1 ) , 2 SMBET+1 = SMBET + N − LF (XT+1 ) , (2.16) where SMBE = (2N)MBE. The number gray levels, L is often of base 2. As such, the multiplication with L could further be reduced to a simple shift operation. Finally, a comparator is required to compute the absolute value of SMBE. From Eq. (2.16), it is clear that the complexity of the computation has been reduced from the function of L2 to L.
418
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
3. Simulation results of MMBEBHE Table 1 contains the AMBE of the test images after being enhanced by HE, BBHE, DSIHE, and MMBEBHE, respectively. Image F16 is a test image used by Kim [5] to demonstrate the effectiveness of BBHE. In this particular case, the threshold level used by BBHE and MMBEBHE are both close to zero. This indicates that the threshold level should be chosen based on the resulting AMBE and not fixed to the input mean. Table 1 also shows that HE, BBHE, and DSIHE do not guarantee minimum mean error. Simulation results of HE, BBHE, DISHE, and MMBEBHE are presented in Figs. 4, 5, 6, 7, and 8. For test image U2 (Fig. 6), there are unpleasant artifacts (bright noise) in the resultant images of HE, BBHE, and DSIHE, but these artifacts are not seen with MMBEBHE. For test image arctic hare (Fig. 7), the resultant images of HE, BBHE, and DSIHE are much darker compared to the original image and they clearly show unnatural enhancement. Result from MMBEBHE indicates that the proposed algorithm has preserved the brightness well (brighter mean brightness) and yielded a more natural enhancement. For test image copter (Fig. 8), it can be observed that the resultant image of HE, BBHE and DSIHE have experienced excessive change in brightness, causing unnatural enhancement and also contrast decrement in the main object (copter). Using MMBEBHE, the image’s original brightness is well preserved and the enhancement yielded is more natural. Test image, hands (Fig. 4) was used by both [4] and [5] to demonstrate the effectiveness of BBHE and DSIHE, respectively. Using MMBEBHE, the output image shows a darker Table 1 List of selected threshold level, XT and resulting AMBE for HE, BBHE, DSIHE, and MMBEBHE
U2 Arctic hare Copter F16 Hands
HE
BBHE
AMBE
XT
AMBE
DSIHE XT
AMBE
MMBEBHE XT
AMBE
96.7 90.5 63.4 48.7 99.5
31 239 148 176 27
13.3 24.2 18.1 0.4 17.5
23 244 155 197 0
41.5 37.9 28.0 14.6 18.3
40 224 142 173 9
6.2 13.5 3.5 0.0 15.4
Fig. 3. Histogram before and after RMSHE, r = 2.
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
419
(e) Fig. 4. (a) Original image of hands, (b) result of MMBEBHE, (c) result of HE, (d) result of BBHE, (e) result of DSIHE.
420
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
(e) Fig. 5. (a) Original image of F16, (b) result of MMBEBHE, (c) result of HE, (d) result of BBHE, (e) result of DSIHE.
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
421
(e) Fig. 6. (a) Original image of U2, (b) result of MMBEBHE, (c) result of HE, (d) result of BBHE, (e) result of DSIHE.
422
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
(e) Fig. 7. (a) Original image of arctic hare, (b) result of MMBEBHE, (c) result of HE, (d) result of BBHE, (e) result of DSIHE.
background, thus preserving the background better than BBHE. Output of DSIHE shows annoying artifacts (white spots) along the edges around the hand and head. These artifacts are not seen in the result of MMBEBHE. For image F16 (Fig. 5), the output of BBHE and MMBEBHE is very similar but better than those of HE and DSIHE, which have become much darker and show poorer contrast around the letters “F16.” The simulation results indicate that MMBEBHE performs better than HE, BBHE, and DSIHE. Nevertheless, there are images that require far more brightness preservation than MMBEBHE can provide to avoid annoying artifacts. This is clearly shown in image girl (Fig. 9(f)) and jet (Fig. 10(f)). The next section presents a generalization of bi-histogram equalization to overcome this limitation.
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
423
(e) Fig. 8. (a) Original image of copter, (b) result of MMBEBHE, (c) result of HE, (d) result of BBHE, (e) result of DSIHE.
4. Recursive mean-separate histogram equalization (RMSHE) The design of BBHE indicates that performing mean-separation before the equalization process does preserve an image’s original brightness. Mean-separation refers to separating an image into two sub-image based on the mean of the input image. In fact, this is
424
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
(e)
(f)
Fig. 9. (a) Original image girl, (b) result of RMSHE, r = 2, (c) result of HE, (d) result of BBHE, (e) result of DSIHE, (f) result of MMBEBHE.
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
(e)
(f)
425
Fig. 10. (a) Original image jet, (b) result of RMSHE, r = 2, (c) result of HE, (d) result of BBHE, (e) result of DSIHE, (f) result of MMBEBHE.
equivalent to separating an image’s histogram into two portions based on the mean of the input image. In HE, no mean-separation is performed and hence, the original brightness is not preserved at all. HE is equivalent to RMSHE level 0 (r = 0). In BBHE, the meanseparation is performed once and the original brightness is preserved to a certain extent. BBHE is equivalent to RMSHE with r = 1. RMSHE generalized BBHE by performing the mean-separation recursively to further preserve the original brightness. Supposed an image X is separated into 4 portions based on the mean of the two portions of the histogram, Xml and Xmu as shown in Fig. 3. Let us define Xml and Xmu as
426
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
Xm Xml =
X0 xp(x) dx Xm X0 p(x) dx
Xm = 2 xp(x) dx, X0
XL−1 Xmu =
xp(x) dx Xm XL−1 p(x) dx Xm
(4.01)
X L−1
=2
xp(x) dx,
(4.02)
Xm
where for simplicity, X is assumed to have a symmetric distribution around Xm [5] such that X Xm L−1 1 p(x) dx = p(x) dx = . 2 X0
(4.03)
Xm
Figure 3 shows the histogram before and after equalizing the four portions of the histogram independently. This is the result of RMSHE with r = 2. The following shows the formulation of the output mean, where X is assumed to have a symmetric distribution around Xm such that p(X Xml ) = p(Xml < X Xm ) = p(Xm < X Xmu ) = p(X > Xmu ) =
1 4
(4.04)
and E(Y) = E(Y | X Xml )p(X Xml ) + E(Y | Xml < X Xm )p(Xml < X Xm ) + E(Y | Xm < X Xmu )p(Xm < X Xmu ) + E(Y | X > Xmu )p(X > Xmu ) 1 = E(Y | X Xml ) + E(Y | Xml < X Xm ) + E(Y | Xm < X Xmu ) 4 + E(Y | X > Xmu ) . (4.05) With similar discussion by Kim [5] to obtain Eqs. (2.04) and (2.05) and the fact that (1/2)(Xml + Xmu ) = Xm (derived from Eqs. (4.01) and (4.02)), it follows that:
Xml + Xm Xm + Xmu Xmu + XL−1 1 X0 + Xml + + + E(Y) = 4 2 2 2 2 1 = {XG + 3Xm }. (4.06) 4 Performing two mean-separations helps to preserve the original brightness better, as indicated by Eq. (4.06) where the weight of the input mean, Xm has increased to three times the weight of the middle of the gray scale, XG . Based on the output mean for RMSHE with r = 0, 1, and 2 as discussed, it can be observed that the weight of the input mean increases as the number of recursive mean-separations increases. The output mean E(Y) for RMSHE with r can be generalized as follows: XG − Xm (2r − 1)Xm + XG . (4.07) = X + E(Y) = m 2r 2r
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
427
Equation (4.07) indicates that as r grows larger, E(Y) will eventually converge to the input mean, Xm . In other words, RMSHE can preserve an image’s original brightness in a scalable manner, range from 0% (output of HE) to 100% (getting back the original image).
5. Simulation results of RMSHE Simulation results using test images girl and jet are presented in Figs. 9 and 10, respectively. For the test image girl, result using HE, BBHE, DSIHE, MMBEBHE (Fig. 9(c)– 9(f)) show unpleasant artifacts in the background, decrease of contrast in the hair and unnatural enhancement of the face. All these artifacts are not seen when using RMSHE with r = 2 (Fig. 9(b)). For the test image jet, it is observed that the results of HE, BBHE, DSIHE, MMBEBHE (Fig. 10(c)–10(f)) are too dark when compared to the original image. The contrast within the body of the jet also becomes poorer. The result of RMSHE, r = 2 (Fig. 10(b)) shows that the proposed algorithm has preserved the brightness well and gives natural enhancement in most part of the image, particularly at the body of the jet.
6. Conclusions In the context of bi-histogram equalization, MMBEBHE is better than BBHE and DSIHE in preserving an image’s original brightness. Real time implementation does not become an issue as there is an efficient integer-based computation of MMBEBHE. Nevertheless, the variety of images in consumer electronics is too wide to be covered with a limited level of brightness preservation offered by bi-histogram equalization based techniques (BBHE, DSIHE, and MMBEBHE). Therefore, RMSHE turns out to be the most practical solution as it allows full range and scalable brightness preservation. Further work is suggested to look for proper mechanism than can automate the selection of the appropriate level of RMSHE, r that would give optimum output. We also suggests further work to formulate a more efficient implementation of RMSHE with reference to how quantized mean-separate HE [8] has been proposed as a cost reduced implementation for BBHE. Last but not least, this paper suggests to look into the extension of this algorithm to color image with reference to what has been proposed in [9] to extend BBHE to color image.
References [1] S.E. Umbaugh, Computer Vision and Image Processing, Prentice Hall, New Jersey, 1998, p. 209. [2] S.M. Pizer, The medical image display and analysis group at the University of North Carolina: reminiscences and philosophy, IEEE Trans. Med. Imag. 22 (1) (2003) 2–10. [3] A. Malinverno, et al., Processing of SeaMARC swath sonar data, IEEE J. Oceanic Eng. 15 (1) (1990) 563– 576. [4] Y. Wan, Q. Chen, B.-M. Zhang, Image enhancement based on equal area dualistic sub-image histogram equalization method, IEEE Trans. Consum. Electron. 45 (1) (1999) 68–75.
428
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
[5] Y.-T. Kim, Contrast enhancement using brightness preserving bi-histogram equalization, IEEE Trans. Consum. Electron. 43 (1) (1997) 1–8. [6] T.V. Pham, M. Worring, A.W.M. Smeulders, Face detection by aggregated Bayesian network classifiers, in: Elsevier Pattern Recognition Letter, vol. 23, 2002, pp. 451–461. [7] L.-L. Huang, et al., Face detection from cluttered images using a polynomial neural network, in: Elsevier Neurocomputing, vol. 51, 2003, pp. 197–211. [8] Y.-T. Kim, Method for image enhancing using quantized mean-separate histogram equalization, US patent 5,857,033, Jan. 5, 1999. [9] Y.-T. Kim, Image enhancing method and circuit using mean-separate/quantized mean separate histogram equalization and color compensation, US patent 6,049,626, Apr. 11, 2000.
Preserving brightness in histogram equalization based contrast enhancement techniques Soong-Der Chen a,∗ , Abd. Rahman Ramli b a College of Information Technology, Universiti Tenaga Nasional, Km 7, Jalan Kajang-Puchong,
43009 Kajang, Selangor D.E., Malaysia b Department of Computer and Communication Systems Engineering, Faculty of Engineering,
Universiti Putra Malaysia, 43400 Serdang, Selangor D.E., Malaysia Available online 28 July 2004
Abstract Histogram equalization (HE) has been a simple yet effective image enhancement technique. However, it tends to change the brightness of an image significantly, causing annoying artifacts and unnatural contrast enhancement. Brightness preserving bi-histogram equalization (BBHE) and dualistic sub-image histogram equalization (DSIHE) have been proposed to overcome these problems but they may still fail under certain conditions. This paper proposes a novel extension of BBHE referred to as minimum mean brightness error bi-histogram equalization (MMBEBHE). MMBEBHE has the feature of minimizing the difference between input and output image’s mean. Simulation results showed that MMBEBHE can preserve brightness better than BBHE and DSIHE. Furthermore, this paper also formulated an efficient, integer-based implementation of MMBEBHE. Nevertheless, MMBEBHE also has its limitation. Hence, this paper further proposes a generalization of BBHE referred to as recursive mean-separate histogram equalization (RMSHE). RMSHE is featured with scalable brightness preservation. Simulation results showed that RMSHE is the best compared to HE, BBHE, DSIHE, and MMBEBHE. 2004 Elsevier Inc. All rights reserved. Keywords: Bi-histogram equalization; Dualistic sub-image; Histogram equalization; Mean separate; Minimum mean brightness error; Recursive
* Corresponding author.
E-mail address: [email protected] (S.-D. Chen). 1051-2004/$ – see front matter 2004 Elsevier Inc. All rights reserved. doi:10.1016/j.dsp.2004.04.001
414
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
1. Introduction Histogram equalization (HE) has been a very popular technique for enhancing the contrast of an image [1]. It has been applied in various fields such as medical image processing [2] and sonar image processing [3]. It remaps the gray levels of an image based on the image’s gray levels cumulative density function. As a result, it stretches the dynamic range of an image and improves the image’s contrast. Besides, HE also produces an image with flat histogram where the entire gray levels have uniform probability density as shown in Fig. 1, where p(x) denotes the probability density of the gray level, x ∈ {X0 , X1 , . . . , XL−1 } and L denotes the total number of discrete gray levels of the image. This is a useful feature as it maximizes the entropy of an image [4]. As shown by Kim [5], an image’s mean brightness after HE tends to be around the middle of the gray scale regardless of the original image’s mean. Let Y denotes the output image, E(·) is the statistical expectation or mean and XG is the middle of the gray scale. Then it follows that: 1 (1.01) E(Y) = XG = (X0 + XL−1 ). 2 While this is useful in application such as face detection for reducing brightness variance [6,7], it is not desirable in consumer electronics application (digital camera, TV, etc.) where preserving the original brightness is necessary to avoid annoying artifacts and unnatural enhancement [5]. For instance, Fig. 4(a) shows an original image hands while Fig. 4(c) shows the resultant image of HE that are composed of 256 gray levels. The equalized image is much brighter compared to the input image. The overall contrast of the image is also degraded after HE. Figures 5(a) and 5(c) show the original image F16, and the resultant image of HE, respectively. Unnatural enhancement can be seen around the cloud; one would perceive a totally different visual recognition around the cloud. Observed that the contrasts around the letters and the emblem on the airplane are also degraded. This is a direct consequence of the excessive change in brightness by HE. Brightness preserving bi-histogram equalization (BBHE) has been proposed to overcome the aforementioned problems [5]. Firstly, it separates an image’s histograms into two
Fig. 1. An image’s histogram before and after HE.
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
415
Fig. 2. Histogram before and after BBHE.
based on the image’s mean, Xm ∈ {X0 , X1 , . . . , XL−1 }. The two histograms are then equalized independently as illustrated in Fig. 2. The output image’s mean of BBHE is a function of the input image’s mean as shown by Kim [5] with the following equation: 1 E(Y) = (Xm + XG ). 2
(1.02)
This implies that BBHE has the feature of preserving an image’s original brightness, which is not found in typical HE. Figures 4(d) and 5(d) clearly demonstrate the effectiveness of BBHE. Later, equal area dualistic sub-image histogram equalization (DSIHE) has been proposed by Wan et al. [4] as an extension to BBHE. DSIHE selectively separate the histogram using threshold level with cumulative probability density equal to 0.5. The DSIHE is claimed to outperform BBHE in term of preserving an image’s brightness and entropy. Nevertheless, both BBHE and DSIHE may fail to enhance an image under certain conditions. Figures 6(d)–6(e), 7(d)–7(e), 8(d)–8(e), 9(d)–9(e), and 10(d)–10(e) show the resultant images of U2, arctic hare, copter, girl, and jet after BBHE and DSIHE, respectively. Their appearances clearly indicate that BBHE and DISHE fail to preserve their original brightness well. This paper aims to present some solutions to overcome the limitations of HE, BBHE, and DISHE in preserving an image’s original brightness. In what follows, the extension of BBHE, namely—minimum mean brightness error bi-histogram equalization (MMBEBHE), together with the formulation of its efficient integer-based implementation will be presented in Section 2. Section 3 lists a few simulation results to demonstrate the effectiveness of MMBEBHE. Next, the generalization of BBHE, namely—recursive meanseparate histogram equalization (RMSHE) will be presented in Section 4 together with the mathematical analysis. Section 5 lists a few simulation results to demonstrate the effectiveness of RMSHE. Section 6 serves as the conclusion of this paper.
416
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
2. Minimum mean brightness error bi-histogram equalization (MMBEBHE) An objective measurement is proposed to rate the performance in preserving the original brightness. It is referred to as absolute mean brightness error (AMBE) and is defined as the absolute difference between the input and the output image’s mean AMBE = E(X) − E(Y). (2.01) X and Y denotes the input and output image, respectively. Lower AMBE indicates that the brightness is better preserved. MMBEBHE, which perform bi-histogram equalization with minimum AMBE is also proposed. MMBEBHE is formally defined by the following procedures: (1) Calculate the AMBE for each of the possible threshold levels. (2) Find the threshold level, XT that yield minimum AMBE. (3) Separate the input histogram into two based on the XT found in Step 2 and equalize them independently as in BBHE. Steps 2 and 3 are straightforward processes. Step 1 would require considerable amount of computation if one full BBHE were required for computing the AMBE for each of the possible threshold levels. The computation complexity would then be a function of L2 . A 16-bits/pixel image would require up to at least (216 )2 operations. This could become a major drawback of MMBEBHE in real time implementation. This paper proposes a fast implementation of MMBEBHE. Equation (2.02) is the general definition for the mean brightness of an image after bi-histogram equalization based techniques such as BBHE, DSIHE, and MMBEBHE, where p(·) denotes probability density and XT denotes the selected threshold level E(Y) = E(Y | X XT )p(X XT ) + E(Y | X > XT )p(X > XT ).
(2.02)
Let us take the differentiation on both sides of Eq. (2.02): E(Y) = E(Y | X XT )p(X XT ) + E(Y | X XT )p(X XT ) + E(Y | X > XT )p(X > XT ) + E(Y | X > XT )p(X > XT ).
(2.03)
Equation (2.03) can be rewritten into a more useful form by applying the following facts. First, it has been shown by Kim [5] that 1 E(Y | X XT ) = (XT + X0 ), 2 1 E(Y | X > XT ) = (XL−1 + XT ). 2
(2.04) (2.05)
Second, for typical representation of digital gray scale image, it can be assumed that X = XT+1 − XT = 1. Then it follows that:
(2.06)
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
E(Y | X XT ) = E(Y | X XT+1 ) − E(Y | X XT ) 1 1 1 = (XT+1 + X0 ) − (XT + X0 ) = , 2 2 2 E(Y | X > XT ) = E(Y | X > XT+1 ) − E(Y | X > XT ) 1 1 1 = (XL−1 + XT+2 ) − (XL−1 + XT+1 ) = . 2 2 2 Third, it can be easily seen that
417
(2.07)
(2.08)
p(X XT ) + p(X > XT ) = 1,
(2.09)
p(X XT ) = p(X XT+1 ) − p(X XT ) = p(XT+1 ),
(2.10)
p(X > XT ) = p(X > XT+1 ) − p(X > XT ) = −p(XT+1 ).
(2.11)
Combining Eqs. (2.07)–(2.11) and Eq. (2.03), 1 + E(Y | X XT ) − E(Y | X > XT ) p(XT+1 ) 2 1 1 1 = − (XL−1 − X0 + 1)p(XT+1 ) = 1 − Lp(XT+1 ) . 2 2 2
E(Y) =
(2.12)
Let ET+1 (Y) and ET (Y) denote the output mean when threshold level is set as XT+1 and XT , respectively. Then, Eq. (2.03) can also be used to relate mean brightness error (MBE) as follows: E(Y) = ET+1 (Y) − ET (Y) = ET+1 (Y) − E(X) − ET (Y) − E(X) = MBET+1 − MBET . (2.13) Combining (2.12) and (2.13), MBET+1 = MBET +
1 1 − Lp(XT+1 ) . 2
(2.14)
Note that MBE0 = XG − E(X).
(2.15)
Equation (2.14) involves p(XT+1 ) which is a floating point number. In fact, the scaled MBE in the form of integer number is sufficient to determine the threshold level that yield minimum AMBE. Let us denote F (Xi ) as the number of pixel with gray level Xi and N as the total number of pixel in the image. It follows that: 1 (2N)MBET+1 = (2N) MBET + 1 − Lp(XT+1 ) , 2 SMBET+1 = SMBET + N − LF (XT+1 ) , (2.16) where SMBE = (2N)MBE. The number gray levels, L is often of base 2. As such, the multiplication with L could further be reduced to a simple shift operation. Finally, a comparator is required to compute the absolute value of SMBE. From Eq. (2.16), it is clear that the complexity of the computation has been reduced from the function of L2 to L.
418
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
3. Simulation results of MMBEBHE Table 1 contains the AMBE of the test images after being enhanced by HE, BBHE, DSIHE, and MMBEBHE, respectively. Image F16 is a test image used by Kim [5] to demonstrate the effectiveness of BBHE. In this particular case, the threshold level used by BBHE and MMBEBHE are both close to zero. This indicates that the threshold level should be chosen based on the resulting AMBE and not fixed to the input mean. Table 1 also shows that HE, BBHE, and DSIHE do not guarantee minimum mean error. Simulation results of HE, BBHE, DISHE, and MMBEBHE are presented in Figs. 4, 5, 6, 7, and 8. For test image U2 (Fig. 6), there are unpleasant artifacts (bright noise) in the resultant images of HE, BBHE, and DSIHE, but these artifacts are not seen with MMBEBHE. For test image arctic hare (Fig. 7), the resultant images of HE, BBHE, and DSIHE are much darker compared to the original image and they clearly show unnatural enhancement. Result from MMBEBHE indicates that the proposed algorithm has preserved the brightness well (brighter mean brightness) and yielded a more natural enhancement. For test image copter (Fig. 8), it can be observed that the resultant image of HE, BBHE and DSIHE have experienced excessive change in brightness, causing unnatural enhancement and also contrast decrement in the main object (copter). Using MMBEBHE, the image’s original brightness is well preserved and the enhancement yielded is more natural. Test image, hands (Fig. 4) was used by both [4] and [5] to demonstrate the effectiveness of BBHE and DSIHE, respectively. Using MMBEBHE, the output image shows a darker Table 1 List of selected threshold level, XT and resulting AMBE for HE, BBHE, DSIHE, and MMBEBHE
U2 Arctic hare Copter F16 Hands
HE
BBHE
AMBE
XT
AMBE
DSIHE XT
AMBE
MMBEBHE XT
AMBE
96.7 90.5 63.4 48.7 99.5
31 239 148 176 27
13.3 24.2 18.1 0.4 17.5
23 244 155 197 0
41.5 37.9 28.0 14.6 18.3
40 224 142 173 9
6.2 13.5 3.5 0.0 15.4
Fig. 3. Histogram before and after RMSHE, r = 2.
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
419
(e) Fig. 4. (a) Original image of hands, (b) result of MMBEBHE, (c) result of HE, (d) result of BBHE, (e) result of DSIHE.
420
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
(e) Fig. 5. (a) Original image of F16, (b) result of MMBEBHE, (c) result of HE, (d) result of BBHE, (e) result of DSIHE.
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
421
(e) Fig. 6. (a) Original image of U2, (b) result of MMBEBHE, (c) result of HE, (d) result of BBHE, (e) result of DSIHE.
422
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
(e) Fig. 7. (a) Original image of arctic hare, (b) result of MMBEBHE, (c) result of HE, (d) result of BBHE, (e) result of DSIHE.
background, thus preserving the background better than BBHE. Output of DSIHE shows annoying artifacts (white spots) along the edges around the hand and head. These artifacts are not seen in the result of MMBEBHE. For image F16 (Fig. 5), the output of BBHE and MMBEBHE is very similar but better than those of HE and DSIHE, which have become much darker and show poorer contrast around the letters “F16.” The simulation results indicate that MMBEBHE performs better than HE, BBHE, and DSIHE. Nevertheless, there are images that require far more brightness preservation than MMBEBHE can provide to avoid annoying artifacts. This is clearly shown in image girl (Fig. 9(f)) and jet (Fig. 10(f)). The next section presents a generalization of bi-histogram equalization to overcome this limitation.
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
423
(e) Fig. 8. (a) Original image of copter, (b) result of MMBEBHE, (c) result of HE, (d) result of BBHE, (e) result of DSIHE.
4. Recursive mean-separate histogram equalization (RMSHE) The design of BBHE indicates that performing mean-separation before the equalization process does preserve an image’s original brightness. Mean-separation refers to separating an image into two sub-image based on the mean of the input image. In fact, this is
424
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
(e)
(f)
Fig. 9. (a) Original image girl, (b) result of RMSHE, r = 2, (c) result of HE, (d) result of BBHE, (e) result of DSIHE, (f) result of MMBEBHE.
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
(a)
(b)
(c)
(d)
(e)
(f)
425
Fig. 10. (a) Original image jet, (b) result of RMSHE, r = 2, (c) result of HE, (d) result of BBHE, (e) result of DSIHE, (f) result of MMBEBHE.
equivalent to separating an image’s histogram into two portions based on the mean of the input image. In HE, no mean-separation is performed and hence, the original brightness is not preserved at all. HE is equivalent to RMSHE level 0 (r = 0). In BBHE, the meanseparation is performed once and the original brightness is preserved to a certain extent. BBHE is equivalent to RMSHE with r = 1. RMSHE generalized BBHE by performing the mean-separation recursively to further preserve the original brightness. Supposed an image X is separated into 4 portions based on the mean of the two portions of the histogram, Xml and Xmu as shown in Fig. 3. Let us define Xml and Xmu as
426
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
Xm Xml =
X0 xp(x) dx Xm X0 p(x) dx
Xm = 2 xp(x) dx, X0
XL−1 Xmu =
xp(x) dx Xm XL−1 p(x) dx Xm
(4.01)
X L−1
=2
xp(x) dx,
(4.02)
Xm
where for simplicity, X is assumed to have a symmetric distribution around Xm [5] such that X Xm L−1 1 p(x) dx = p(x) dx = . 2 X0
(4.03)
Xm
Figure 3 shows the histogram before and after equalizing the four portions of the histogram independently. This is the result of RMSHE with r = 2. The following shows the formulation of the output mean, where X is assumed to have a symmetric distribution around Xm such that p(X Xml ) = p(Xml < X Xm ) = p(Xm < X Xmu ) = p(X > Xmu ) =
1 4
(4.04)
and E(Y) = E(Y | X Xml )p(X Xml ) + E(Y | Xml < X Xm )p(Xml < X Xm ) + E(Y | Xm < X Xmu )p(Xm < X Xmu ) + E(Y | X > Xmu )p(X > Xmu ) 1 = E(Y | X Xml ) + E(Y | Xml < X Xm ) + E(Y | Xm < X Xmu ) 4 + E(Y | X > Xmu ) . (4.05) With similar discussion by Kim [5] to obtain Eqs. (2.04) and (2.05) and the fact that (1/2)(Xml + Xmu ) = Xm (derived from Eqs. (4.01) and (4.02)), it follows that:
Xml + Xm Xm + Xmu Xmu + XL−1 1 X0 + Xml + + + E(Y) = 4 2 2 2 2 1 = {XG + 3Xm }. (4.06) 4 Performing two mean-separations helps to preserve the original brightness better, as indicated by Eq. (4.06) where the weight of the input mean, Xm has increased to three times the weight of the middle of the gray scale, XG . Based on the output mean for RMSHE with r = 0, 1, and 2 as discussed, it can be observed that the weight of the input mean increases as the number of recursive mean-separations increases. The output mean E(Y) for RMSHE with r can be generalized as follows: XG − Xm (2r − 1)Xm + XG . (4.07) = X + E(Y) = m 2r 2r
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
427
Equation (4.07) indicates that as r grows larger, E(Y) will eventually converge to the input mean, Xm . In other words, RMSHE can preserve an image’s original brightness in a scalable manner, range from 0% (output of HE) to 100% (getting back the original image).
5. Simulation results of RMSHE Simulation results using test images girl and jet are presented in Figs. 9 and 10, respectively. For the test image girl, result using HE, BBHE, DSIHE, MMBEBHE (Fig. 9(c)– 9(f)) show unpleasant artifacts in the background, decrease of contrast in the hair and unnatural enhancement of the face. All these artifacts are not seen when using RMSHE with r = 2 (Fig. 9(b)). For the test image jet, it is observed that the results of HE, BBHE, DSIHE, MMBEBHE (Fig. 10(c)–10(f)) are too dark when compared to the original image. The contrast within the body of the jet also becomes poorer. The result of RMSHE, r = 2 (Fig. 10(b)) shows that the proposed algorithm has preserved the brightness well and gives natural enhancement in most part of the image, particularly at the body of the jet.
6. Conclusions In the context of bi-histogram equalization, MMBEBHE is better than BBHE and DSIHE in preserving an image’s original brightness. Real time implementation does not become an issue as there is an efficient integer-based computation of MMBEBHE. Nevertheless, the variety of images in consumer electronics is too wide to be covered with a limited level of brightness preservation offered by bi-histogram equalization based techniques (BBHE, DSIHE, and MMBEBHE). Therefore, RMSHE turns out to be the most practical solution as it allows full range and scalable brightness preservation. Further work is suggested to look for proper mechanism than can automate the selection of the appropriate level of RMSHE, r that would give optimum output. We also suggests further work to formulate a more efficient implementation of RMSHE with reference to how quantized mean-separate HE [8] has been proposed as a cost reduced implementation for BBHE. Last but not least, this paper suggests to look into the extension of this algorithm to color image with reference to what has been proposed in [9] to extend BBHE to color image.
References [1] S.E. Umbaugh, Computer Vision and Image Processing, Prentice Hall, New Jersey, 1998, p. 209. [2] S.M. Pizer, The medical image display and analysis group at the University of North Carolina: reminiscences and philosophy, IEEE Trans. Med. Imag. 22 (1) (2003) 2–10. [3] A. Malinverno, et al., Processing of SeaMARC swath sonar data, IEEE J. Oceanic Eng. 15 (1) (1990) 563– 576. [4] Y. Wan, Q. Chen, B.-M. Zhang, Image enhancement based on equal area dualistic sub-image histogram equalization method, IEEE Trans. Consum. Electron. 45 (1) (1999) 68–75.
428
S.-D. Chen, A. Rahman Ramli / Digital Signal Processing 14 (2004) 413–428
[5] Y.-T. Kim, Contrast enhancement using brightness preserving bi-histogram equalization, IEEE Trans. Consum. Electron. 43 (1) (1997) 1–8. [6] T.V. Pham, M. Worring, A.W.M. Smeulders, Face detection by aggregated Bayesian network classifiers, in: Elsevier Pattern Recognition Letter, vol. 23, 2002, pp. 451–461. [7] L.-L. Huang, et al., Face detection from cluttered images using a polynomial neural network, in: Elsevier Neurocomputing, vol. 51, 2003, pp. 197–211. [8] Y.-T. Kim, Method for image enhancing using quantized mean-separate histogram equalization, US patent 5,857,033, Jan. 5, 1999. [9] Y.-T. Kim, Image enhancing method and circuit using mean-separate/quantized mean separate histogram equalization and color compensation, US patent 6,049,626, Apr. 11, 2000.