Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation

Computers and Electrical Engineering xxx (2015) xxx–xxx

Contents lists available at ScienceDirect

Computers and Electrical Engineering journal homepage: www.elsevier.com/locate/compeleceng

Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation q S.C.F. Lin a, C.Y. Wong a, M.A. Rahman a, G. Jiang a, S. Liu a, Ngaiming Kwok a,d,⇑, Haiyan Shi b, Ying-Hao Yu c, Tonghai Wu d,* a

School of Mechanical and Manufacturing Engineering, The University of New South Wales, Sydney 2052, Australia School of Computer Science and Technology, Shaoxing University, Shaoxing, Zhejiang 312000, China Department of Electrical Engineering, National Chung Cheng University, No. 168, University Rd, Min-Hsiung Township, Chiayi County 621, Taiwan d Key Laboratory of Modern Design and Rotor Bearing System of Ministry, Xi’an Jiaotong University, Xi’an 710049, Shaanxi, China b c

a r t i c l e

i n f o

Article history: Received 9 September 2014 Received in revised form 1 June 2015 Accepted 1 June 2015 Available online xxxx Keywords: Image contrast enhancement Histogram smoothing Brightness preservation Dynamic stretching Sub-image separation optimization

a b s t r a c t Image contrast enhancement and brightness preservation are fundamental requirements for many vision based applications. However, these are two conflicting objectives when the image is processed by histogram equalization approaches. Current available methods may not provide results simultaneously satisfying both requirements. In this work, a pipelined approach including color channel stretching, histogram averaging and re-mapping is developed. By using stretching, color information from a scene is restored. Averaging against a uniform distribution enables the output image to recover the information lost. Furthermore, histogram re-mapping reduces artifacts that often arise from the equalization procedure. The technique also employs a search process to find optimal algorithmic parameters, such that the mean brightness difference between the input and output images is minimized. The effectiveness of the proposed method was tested with a set of images captured in adverse environments and compared against available methods. High performing qualitative and quantitative results were obtained. Ó 2015 Elsevier Ltd. All rights reserved.

1. Introduction Digital imagery is becoming an indispensable source of information in a wide range of contemporary applications. For instance, it is found in hand-held consumer electronic devices [1], where auto-focusing is one of the major functionalities [2]. In engineering, there are even more demands and requirements of image processing technology. This technique can be used in the monitoring of machine operation conditions [3], in the remote sensing of the environment revealing land covers [4]. In robotics, image processing can be used for object tracking [5]. It is also useful in human computer interactions [6] such as hand gesture recognition [7]. The major demand for image enhancement is to restore a captured image from degradations arising from imperfect acquisition conditions. For example, to remove the noise imposed, to correct the color cast [8] and to sharpen the objects that appear in the image [9].

q

Reviews processed and recommended for publication to the Editor-in-Chief by Associate Editor Dr. E. Cabal-Yepez.

⇑ Corresponding authors at: School of Mechanical and Manufacturing Engineering, The University of New South Wales, Sydney 2052, Australia (N. Kwok). E-mail addresses: [email protected] (N. Kwok), [email protected] (T. Wu). http://dx.doi.org/10.1016/j.compeleceng.2015.06.001 0045-7906/Ó 2015 Elsevier Ltd. All rights reserved.

Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

2

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

One of the most demanding features in restoring an image is to improve its contrast so that it is pleasant to a human viewer and to easy computer based operations such as object identification [10]. Among these, contrast improvement using histogram equalization (HE) methods has found a lot of real-world applications for its implementation simplicity and effectiveness [11]. In particular, uniform histogram equalization (UNFHE) or conventional histogram equalization is the most common HE method employed in computer vision applications, and the histogram of the output image from this equalization follows a uniform distribution. However, it is also observed that these approaches may produce artifacts due to over-enhancement if the algorithm is not properly designed. Another concern of HE based techniques is the preservation of the output image brightness such that a true reproduction of the scene characteristics can be made available. There are a number of research works reported in literature that aimed at accomplishing the aforementioned goal. An early attempt was made to separate the input image into two sub-images using the mean intensity or brightness as the threshold in the brightness preserving bi-histogram equalization (BBHE). The two sub-images were then processed independently using histogram equalization with reference to a target uniform histogram [12]. By the design of this method, an exact match between the brightness of input and output images is not generally available. In order to reduce such discrepancy, the separation was made using the median brightness [13]. In the equal area dualistic sub-image histogram equalization method (DSIHE), the numbers of pixels classified in the lower and higher sub-images are equal. It enables a closed form solution to find the output image mean brightness, however, its preservation is still not guaranteed. Further to the requirement to satisfy mean brightness preservation, avoiding the generation of artifacts due to over-enhancement is also important. A class of methods was hence developed where the peaks of the histogram are clipped to avoid over-enhancement. An original development was suggested; where the image is first separated into two sub-images using the input image mean brightness, known as the bi-histogram equalization with a plateau limit (BHEPL) method [14]. The clipping limits were obtained from the mean values of the histogram heights of individual sub-images and then used as the target histogram in the equalization [15]. Although over-enhancement can be controlled, but due to the deviation of the target histogram from a uniform profile, it is difficult to steer the output image mean brightness to the desired value. The choice of the clipping limit is not automatic but it may be problem specific, such as driven by the exposure or viewing perception [16,17]. The selection of separation thresholds and clipping limits has been addressed in several research works. A variation named the bi-histogram equalization median plateau (BHEPLD) was reported in [18], where the clipping limit was obtained from the median values of the sub-images. In fact, this choice of plateaus does not warrant perfect brightness preservation. Recently, another choice of clipping limit was proposed as the adaptive image enhancement bi-histogram equalization (AIEBHE) [19]. In the work reported therein, the sub-images are separated by the input median brightness. The plateaus were taken from the minimum of the histogram value, mean value and the median value of the sub-image histogram. This choice of limits might fail when the median value tends to zero for images biased toward the two ends of the intensity range. A further variation was reported, known as the median-mean sub-image clipped histogram equalization (MMSICHE) [20]. In the work presented there, the input image was divided into four sub-images. The first separation was carried out using the mean intensity while the division into four sectors used the median values of the two sub-images. The clipping limits for the four obtained sub-images were the median values of individual sectors. As can be seen, this process increases implementation complexity while the preservation of output image brightness cannot be assured. Other than clipping the histogram to prevent artifacts generation, there have been alternative approaches available. For example, the input image histogram can be smoothed and used as the target distribution in the histogram equalization process called smoothed histogram equalization (SMHEQ) [21]. The smoothing was conducted using a varying weighting divisor as a sliding window across the input image histogram. The next effect was that histogram elements in the middle intensity range were flattened towards a uniform distribution thus contributed to the increase in information content. The low and high intensity elements were only modified to a lesser extend hence reducing the generation of artifacts. By its algorithmic design, this method does not particularly address the brightness preservation problem. On the other hand, this approach had indicated a direction for an automatic process to enhance the image contrast and to preserve the mean brightness by properly manipulating the smoothing process. In this work, a color image enhancement approach is proposed. The developed procedure is performed in a pipeline called averaging histogram equalization (AVHEQ). The method is formulated towards restoring images degradations due to adverse environmental conditions when they are captured. Improvements are sought in the information content, contrast, and preservation of the mean brightness in order to truly represent the features in the scene. Unlike previous approaches, this method does not need to separate the histogram into sections and it is not necessary to include hard limiting plateaus to clip histogram peaks. Furthermore, an efficient search algorithm is used to optimize the performance without any user involvement. The rest of the paper is organized as follows. In Section 2, backgrounds in histogram equalization for contrast enhancement and its variations are reviewed. The proposed approach based on the histogram averaging concept is presented in Section 3. Experiments are described in Section 4 and their results are discussed. Section 5 contains the conclusions.

2. Related work Previous works using histogram equalization for color image contrast enhancement are reviewed in this section. Those methods commonly convert the color image into its gray or monochrome equivalence to be processed and re-converted back Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

3

to the color domain as the enhanced output. These methods include the conventional equalization where a global strategy is adopted and the target histogram follows a uniform distribution. Then, methods based on separating the input image into sub-images for contrast enhancement are discussed. In addition, approaches invoking histogram clipping and histogram smoothing strategies to mitigate artifacts due to over-enhancement are described. 2.1. Conventional histogram equalization Consider an input color image in red green blue (RGB) format of sized U  V in width-by-height is given. The magnitude of each color channel is bounded within ½0 L  1 for typical 8-bit storage, where L ¼ 28 ¼ 256. The image can be described by

I ðu; v Þ ¼ fRðu; v Þ Gðu; v Þ Bðu; v Þg;

ð1Þ

where ðu; v Þ is the pixel coordinate in the image. In most approaches, the color image is converted to a gray image, for example, through the hue saturation intensity (HSI) space and the intensity I-channel I is extracted and enhanced. That is

T ðfRGBgÞ;

fHSIg

ð2Þ

where T ðÞ is the RGB to HSI transformation, see [10]. In this method, a histogram is first constructed for the input image by

hðiÞ ¼ fnðiÞg;

ð3Þ

where i is the intensity index and nðiÞ is the number of pixels having the ith intensity. A cumulative density function is further obtained from

cðiÞ ¼

i 1X nðjÞ; cðL  1Þ ¼ 1; N ¼ U  V: N j¼0

ð4Þ

The enhanced gray image is then derived from the equalization operation

Ienh ¼ Imin þ ðImax  Imin Þ  cðiÞ;

ð5Þ

where Imin and Imax are the minimum and maximum target intensities, and usually Imin ¼ 0 and Imax ¼ L  1. 2.2. Histogram separation Since the output image from conventional histogram equalization, Ienh , follows a uniform distribution, its mean intensity becomes

Im ¼

L1 X ði  po ðiÞÞ; i ¼ 0; . . . ; L  1;

ð6Þ

i¼0

where po ðiÞ ¼ ho ðiÞ=N ¼ 1=L and ho ðiÞ represents the histogram of the image Ienh . It is deviated from the input image and may be regarded as undesirable [12]. Attempts had been made to remedy this drawback by invoking the brightness preserving concepts in designing the bi-histogram based approaches. There are methods that separate the input image into two sub-images and then carry out conventional histogram equalization on each sub-image independently [12,13]. The difference is that the former work used the mean brightness as the separation threshold while the latter used the median value. Since the numbers of pixels below and above the threshold are determined by the image content, the output mean brightness is not exactly the same as the input mean brightness. Following the works with bi-histogram equalization, methods were developed that limited the peaks in the histogram by the mean value and median value of individual sub-images [14,18]. For example, the image is first divided into the low and high portions as

Ilo ¼ fIðiÞ j 0 < i 6 Im g;

ð7Þ

Ihi ¼ fIðjÞ j Imþ1 < j 6 L  1g:

ð8Þ

The output image is obtained from the aggregation of two enhanced sub-images as

Ienh ¼ Ienh;lo [ Ienh;hi ;

ð9Þ

where the enhanced sub-images Ienh;lo and Ienh;hi are separately generated by performing the conventional histogram equalization discussed in the last section on the two sub-images Ilo and Ihi . The target distributions of these enhanced sub-images are the clipped sub-histograms. Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

4

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

2.3. Histogram clipping Conventional histogram equalization methods suffer from over-enhancement effects, which frequently produce unnatural viewing artifacts. It was observed that artifacts arise when the change in intensity is too excessive during equalization. It is mostly observed at intensity levels where there are peaks in the input image histogram. In the method presented in [20], the input image is divided into four quadrants using the mean to divide into two sub-images, then the median values of each sub-image are further used to divide each into two more sub-images. The clipping limits were chosen as the median values of each of the four sub-images. A further variation was reported in [19] where the input image was divided using the mean intensity. Unlike the others, the clipping limits were chosen from the minimum of the histogram element and the mean and median values of the sub-image. That is

lð~iÞ; mð~iÞg; 0 < i 6 Im ; Phi ðjÞ ¼ minfhðjÞ; lð~jÞ; mð~jÞg; Imþ1 < j 6 L  1;

Plo ðiÞ ¼ minfhðiÞ;

ð10Þ ð11Þ

where ~i denotes the sub-image range 0 < i 6 Im , and lð~iÞ and mð~iÞ are the mean and median values of the corresponding sub-image. Variables lð~jÞ and mð~jÞ are defined accordingly for Imþ1 < j 6 L  1. The target histograms to be used in the equalization then become

 hlo ðiÞ ¼

hlo ðiÞ; hlo ðiÞ 6 Plo ðiÞ

Plo ðiÞ; otherwise;  hhi ðjÞ; hhi ðjÞ 6 Phi ðjÞ

hhi ðjÞ ¼

Phi ðjÞ; otherwise:

ð12Þ ð13Þ

With the modifications and variations originating from the bi-histogram concept, procedures are becoming available to be used in different problem domains. However, it is still non-trivial to arrive at an exact matching of the input and output image mean brightness as well as simultaneously enhancing the image contrast without producing viewing artifacts.

2.4. Histogram smoothing While approaches based on sub-image separation had gained popularity in contrast enhancement and mean brightness preservation, alternatives from a different conceptual perspective are still possible. In order to mitigate artifacts generated from equalization towards a uniform histogram, as commonly practiced in sub-image based processes, the target histogram can be obtained from smoothing the input image histogram [21]. In this method, the input image histogram is smoothed by calculating the average values for each intensity level in the histogram. The averaging is conducted by adopting the dividing factor to the intensity. The process can be described as

hðiÞ

hðiÞ ; i  wðiÞ

ð14Þ

where the range of division is a function of the intensity and is determined from

wðiÞ ¼



2ði þ 1Þ  1;

0 < i 6 L=2;

2ðL  i þ 1Þ  1; L=2 < i 6 L  1:

ð15Þ

The resultant smoothed histogram becomes flatter in the middle intensity range and replicates the input histogram at the two ends of the intensity range. The modified histogram is then used as the target distribution in the equalization process. On the other hand, since the histogram is only moderately altered at the two intensity ends the output image contains a reduced amount of artifacts, where they are most noticeable at low and high intensities.

2.5. Discussion It can be noted from the review above that conventional histogram equalization is vulnerable to produce viewing artifacts where the changes of intensities between the input and output are too large. On the other hand, this kind of approach also shifts the mean brightness of the output image. Methods adopting histogram peak clipping are able to reduce the abrupt change of intensities, but it also destroys the matching of input and output mean brightness. The histogram smoothing strategy is able to mitigate artifacts but by its design the maintenance of brightness is not specifically considered. The analysis has thus motivated a new scheme to enhance the image contrast while keeping the change in brightness to a minimum. The latter objective is tackled as an optimization problem and the details are presented below. Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

5

3. Averaging histogram equalization (AVHEQ) approach The objective of this work can be stated as increasing the image contrast while keeping the mean brightness of the output image unaltered as much as possible. For an input color image, the proposed approach is carried out in a pipeline process as depicted in Fig. 1. First, a color channel stretching operation is performed allowing the full coverage of the permitted dynamic ranges in order to convey the richest set of color information to the viewer. Then, a histogram averaging process is carried out before it is used in the equalization operation. As it is observed that an image having a uniform histogram carries the highest amount of information, the proposed method proceeds by averaging the given image histogram to a scaled uniform histogram. The purpose is to increase the information content that is obscured by the image capturing conditions. In particular, the choice of the scale is cast as an optimization problem for the best performance while removing the need for user specifications. An iterative search loop is incorporated, as shown by the dotted line in Fig. 1, where an intermediate enhanced image is compared with the original for brightness error. The error and the input image entropy form an objective function to guide the search for the optimal scale. When the brightness error is minimized and the entropy is maximized then the search terminates. Finally, a histogram re-mapping process is conducted together with the HSI to RGB conversion stage to generate the output image. 3.1. Color channel stretching The input image is first stretched on every color channel in order to correct for any color distortion arising from unfavorable capturing environment. We have

Rðu; v Þ

Rðu; v Þ  Rmin ; Rmax  Rmin

ð16Þ

where Rmin ¼ minfRðu; v Þg for all ðu; v Þ, and Rmax ¼ maxfRðu; v Þg for all ðu; v Þ. The green (G) and blue (B) channels are stretched in the similar manner. In parallel with the stretching, the input image is converted to its gray scale image and the I-channel is extracted, see Eq. (2). The mean brightness Im is calculated using Eq. (6). This mean intensity will be used as the reference for the output image brightness. 3.2. Histogram averaging An image having a uniform intensity distribution is known to carry the maximum amount of information about the scene. However, artifacts often appear as compared to the original image. An optimized combination of histogram equalized image, to a uniform distribution, with the original image would give an enhancement with reduced artifacts. The stretched color signal is converted to its gray image equivalent for use in contrast enhancement. From the converted I-channel, a histogram is constructed using the procedure given by Eq. (3). The averaging process is illustrated in Fig. 2. The distribution of the input image occupies the middle intensity range and its information is low. After stretching, the histogram expands and the information increases to a sub-maximum level. A desirable uniform distribution, the straight line in Fig. 2, is then combined in a weighted-sum manner in order to further increase the information content. After optimally weighted, the histogram is normalized such that the sum of the bins is equal to the number of pixels. The averaging process is further elaborated below. Given the histogram, it is averaged as

 hðiÞ

hðiÞ þ s  maxfhðiÞg ; 0 < s < 1; 2

ð17Þ

where maxfhðiÞg is taken over all intensity levels i ¼ 0; . . . ; L  1. It is further normalized before being used as the target distribution in the histogram equalization process. The normalization is conducted as

Fig. 1. System block diagram of the proposed approach.

Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

6

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

Fig. 2. Demonstration of the histogram averaging process.

 ^ ¼ P hðiÞ  N: hðiÞ L1 i¼0 hðiÞ

ð18Þ

With the use of the normalized histogram, an output image ^Ienh is obtained through conventional histogram equalization such that

^Ienh ðiÞ ¼ ðL  1Þ  ^cðiÞ; ^cðiÞ ¼

i X

^ hðjÞ:

ð19Þ

j¼0

Note that at this stage, the image in its gray level equivalence is not yet the final enhanced image in colors. 3.3. Histogram re-mapping The histogram equalization process is liable to produce abrupt changes in the output image intensities and this effect is undesirable. In order to cope with the drawback, an intensity re-mapping procedure is developed in this work. In most cases, the abrupt change results in the absence of several intensity levels and this can be detected from inspecting the histogram. First, intensity indices of non-empty bins are identified to give

^ j hðiÞ ^ > 0g; z ¼ fzðkÞ ¼ hðiÞ

ð20Þ

where the parameter k ¼ 1; . . . ; kmax is a variable that counts the number of non-empty bins in the processed image his^ In other words, the set z is a collection of non-empty histogram bins. The number of entries kmax is less than togram hðiÞ. the intensity levels L if non-empty bins exist. To this end, the set z is regarded as a compressed histogram where consecutive bins are non-empty. The intensities of pixels in the non-empty bins are then re-mapped to an evenly distributed intensity range spanning across the entire permissible limits in ½0 L  1. The re-mapping process can be described by

~zðkÞ

k1  ðL  1Þ: kmax  1

ð21Þ

Fig. 3 illustrates the histogram re-mapping process. It can be seen that the non-empty bins in the histogram of the image ^Ienh ðiÞ are not spaced equally before re-mapping. On the other hand, the re-mapping process allocated the non-empty bins in an evenly distributed manner after re-mapping. Therefore, abrupt changes in intensities are alleviated and the occurrences of artifacts are reduced. 3.4. Averaging threshold optimization In order to accomplish the dual objectives in enhancing the image contrast and to maintain the mean brightness, the choice of the averaging threshold s, see Eq. (17), is cast as an optimization problem and the efficient golden section search algorithm is adopted for its implementation simplicity [22]. To find the optimal averaging threshold, an objective function for optimization is defined from the information content, entropy ðHÞ, as an indication of the image contrast. Furthermore, to meet the requirement of minimum brightness change, it is used as a penalty term in the objective function. We have

J ¼

j ^Im;enh  Im j  H; Im

ð22Þ

Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

7

Fig. 3. Demonstration of the histogram re-mapping process.

where ^Im;enh is the mean brightness of the processed image ^Ienh . The variable H is the entropy given by L1 X ^ H ¼  pðiÞ log pðiÞ; pðiÞ ¼ hðiÞ=N:

ð23Þ

i¼0

The objective function J is then to be maximized. After defining the optimization objective function J , the proper averaging threshold s is determined from Algorithm 1 using golden section search. A final enhanced image Ienh is obtained using the optimal averaging threshold. Algorithm 1. Optimization of averaging threshold Input: reference brightness of input image Im Output: optimal averaging threshold s pffiffiffi Set iteration ratio q ¼ 0:5ð 5  1Þ ¼ 0:618 Set trial thresholds, s1 ; s2 ; Ds ¼ s2  s1 for s2 > s1 Set range limit a !  Carry out enhancement process to obtain J 1 , J 2 while Ds > a do if J 1 > J 2 then Set s2 ¼ s1 þ q  Ds Do enhancement to obtain J 2 end if if J 2 > J 1 then Set s1 ¼ s1 þ ð1  qÞ  Ds Do enhancement to obtain J 1 end if Re-calculate range Ds end while Return optimal averaging threshold s ¼ ðs1 þ s2 Þ=2

4. Experiments Experiments were conducted to verify the effectiveness of the proposed method both qualitatively and quantitatively. A set of 220 color images were used. These include 20 public available images from http://r0k.us/graphics/kodak/. Others are Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

8

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

images captured natural scenery under different environmental conditions such as clear day, night time and poor conditions with overcast weather. The images are delivered in JPEG files of 8-bit data format in each RGB color channel. The image sizes are 300  400 width-by-height. The qualities of the enhanced images are assessed using the brightness error, contrast, gradient and entropy, see Eq. (23). Statistics of results are also collected and evaluated from their corresponding box plots. The performance of the averaging histogram equalization AVHEQ is compared to available methods including UNFHE, BBHE, DSIHE, BHEPL, BHEPLD, AIEBHE, MMSICHE and SMHEQ. 4.1. Assessment criteria The brightness error is defined as the difference between the mean brightness of the input image and the mean brightness of the enhanced image. It can be expressed as [14],

Im;enh ¼ Im  Im;enh :

ð24Þ

A value close to zero indicates that the mean brightness of the enhanced image is preserved with respect to the input image. The contrast of the image is given by [19],

!2 1X 1X 2 O¼ Ienh ðu; v Þ  Ienh ðu; v Þ : N u;v N u;v

ð25Þ

This performance index takes into account the average intensities and their dispersions around a center pixel. A higher value represents a higher contrast. The gradient is obtained from [10],

G¼

1X ðDu2 þ Dv 2 Þ; N u;v

ð26Þ

where Du ¼ Ienh ðu; v Þ  Ienh ðu þ 1; v Þ and Dv ¼ Ienh ðu; v Þ  Ienh ðu; v þ 1Þ are the gradients of the enhanced image. A larger value signifies a sharper image as perceived by the viewer. 4.2. Qualitative analysis A sample of six test images and their outputs from different approaches are shown in Figs. 4–9. The first two are from the benchmark images. The input for Test Image 1 is shown in Fig. 4(a). The image contains a natural outdoor scene. It is observed that outputs from UNFHE, BBHE, and DSIHE are brighter than the input. For outputs obtained from BHEPL, BHEPLD, AIEBHE, MMSICHE and SMHEQ have brightness close to the input. However, viewing artifacts appear particularly in the homogeneous regions in the sky. The result from the AVHEQ method, which is shown in Fig. 4(j), contains no artifacts. Furthermore, the output image appears more appealing to a viewer with a better color tone and higher contrast. In addition, the change in brightness is not noticeable. Test Image 2, as shown in Fig. 5, is another sample from the benchmark images. Similar to the observations in Test Image 1, outputs are over-enhanced from the first three methods under comparison. It can be seen that darker regions have become darker and fine details are lost. Other approaches give acceptable enhancement effects, where the AVHEQ output is most satisfactory, as illustrated in Fig. 5(j). In particular, the color cast in the input image is corrected and the contrast is increased. The input image, Test Image 3, is shown in Fig. 6(a). It is an image captured in poor weather conditions that the contrast is low and the color is non-vivid. Fig. 6(b)–(d) are obtained from approaches based on image separation and equalization toward a uniform distribution. It can be seen that due to abrupt changes in intensity, viewing artifacts occur where several regions are too dark or too bright such that feature details are lost. Results from implementations based on histogram peak clipping are given in Fig. 6(e)–(h). It is observed that artifacts are alleviated but improvements in contrast are not noticeable indicating that a trade-off has to be adopted for these two conflicting goals. Fig. 6(i) contains the output from the smoothing approach. The contrast is improved and artifacts are not present. However, the brightness deviates from the input image. The enhanced image from the proposed method is shown in Fig. 6(j). It is noted that the visual perception is improved where the color is more vivid and contrast is increased. Furthermore, a balance between dark and bright regions is maintained to give a mean brightness close to the input image. Additional test images, Test Image 4, 5, and 6, and their respective outputs from various enhancements are given in Figs. 7–9. It is observed that results from approaches involving flat histograms, either from equalization to a uniform distribution or becoming uniform after clipping, produce artifacts. These approaches include UNFHE, BBHE, DSIHE, BHEPL, BHEPLD, AIEBHE, and MMSICHE. The undesirable artifacts are mostly found in image regions where the content is relatively monotonic. In particular, flat histograms may not be ideal target references in the equalization process. On the other hand, results from AVHEQ are satisfactory. Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

9

Fig. 4. Test image 1. Methods: (a) input, (b) UNFHE, (c) BBHE, (d) DSIHE, (e) BHEPL, (f) BHEPLD, (g) AIEBHE, (h) MMSICHE, (i) SMHEQ, (j) proposed AVHEQ.

Fig. 5. Test image 2. Methods: (a) input, (b) UNFHE, (c) BBHE, (d) DSIHE, (e) BHEPL, (f) BHEPLD, (g) AIEBHE, (h) MMSICHE, (i) SMHEQ, (j) proposed AVHEQ.

Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

10

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

Fig. 6. Test image 3. Methods: (a) input, (b) UNFHE, (c) BBHE, (d) DSIHE, (e) BHEPL, (f) BHEPLD, (g) AIEBHE, (h) MMSICHE, (i) SMHEQ, (j) proposed AVHEQ.

Fig. 7. Test image 4. Methods: (a) input, (b) UNFHE, (c) BBHE, (d) DSIHE, (e) BHEPL, (f) BHEPLD, (g) AIEBHE, (h) MMSICHE, (i) SMHEQ, (j) proposed AVHEQ.

Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

11

Fig. 8. Test image 5. Methods: (a) input, (b) UNFHE, (c) BBHE, (d) DSIHE, (e) BHEPL, (f) BHEPLD, (g) AIEBHE, (h) MMSICHE, (i) SMHEQ, (j) proposed AVHEQ.

Fig. 9. Test image 6. Methods: (a) input, (b) UNFHE, (c) BBHE, (d) DSIHE, (e) BHEPL, (f) BHEPLD, (g) AIEBHE, (h) MMSICHE, (i) SMHEQ, (j) proposed AVHEQ.

Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

12

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

4.3. Quantitative analysis Statistics of resultant images are collected and used in the quantitative analysis. Assessment criteria are depicted in box plots to illustrate the performance of test methods across the 220 test images. Fig. 10 indicates the measures on mean brightness errors between the input and enhanced images. Comparisons are made to the left-most box, that is, the input image having zero error. Methods under comparison give various degrees of brightness error. The largest spread comes from the uniform histogram equalization as expected since there is no mechanism to preserve the brightness. Approaches based on histogram clipping and the smoothing method also result in wide spreads. It can be seen that clipping is not contributing to brightness preservation while the smoothing approach is not designed for this objective. The proposed method produces the smallest error spread despite the small number of outliers. Furthermore, the average value of the brightness error 0.008 is the smallest among all test methods. Statistics of the information content or entropy criterion are presented in Fig. 11. It is interesting to observe that due to the histogram equalization operation, resultant entropies except for the proposed approach are reduced. These phenomena can be attributed to the fact that the permitted intensity levels have not been fully utilized. On the other hand, because of the color channel stretching and histogram re-mapping processes, the proposed method is able to use the allowed intensity range more effectively and provides a higher average entropy value at 7.520 against the input images at 7.321. Furthermore, the increase in entropy from the developed method is consistent over the test images as indicated by a compact box plot with close 1st and 3rd quartiles. Fig. 12 shows the box plots of the performance measure with respect to image contrast. As expected, the uniform histogram equalization method gives the highest contrast. However, as observed from the quantitative analysis, the output images often contain undesirable viewing artifacts. On the other hand, the smoothing approach gives the next high contrast value but the brightness is not preserved. The output contrast from the proposed method gives an average reading of 0.536, as compared to the input at 0.507, which meets our expectation comparing with other methods. A slight trade-off is resulted from the improvement in brightness preservation and mitigation of artifacts. The performance metric on gradient is shown in the box plots depicted in Fig. 13. Methods using histogram equalization without clipping produce high gradient measures. As seen from the quantitative analysis, the results often contain unwanted artifacts. Approaches employing histogram clipping contain lower gradient values as compromises are made for artifacts reduction. The smoothing based method gives higher gradient but brightness is not preserved. The proposed approach produces an slight increase in gradient at 0.036 against the input at 0.031. Moreover, artifacts are alleviated and the brightness is preserved in this method. Further to the box plots, hypothetical tests were carried out to evaluate the AVHEQ performance. In the evaluation, a t-test at 0.05 significance level was adopted for the following hypotheses.

Ho : input and output result mean values are equal: H1 : input and output result mean values are not equal: The performance metrics compared are the brightness error, entropy, contrast, and gradient for the set of 220 test images. Results are summarized in Table 1. From the brightness error test hypothesis results, we do not reject the null hypothesis for BBHE, DSHE, MMSICHE and AVHEQ. It is also observed that the p-value of the AVHEQ approach is the smallest at 0.285. Thus, the decision not to reject the null hypothesis is more certain for AVHEQ that the output image brightness is not deviated from the input.

Fig. 10. Statistics of test results – brightness error.

Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

13

Fig. 11. Statistics of test results – entropy.

Fig. 12. Statistics of test results – contrast.

Fig. 13. Statistics of test results – gradient.

For the hypothesis test on entropy, all null hypotheses of all tested approaches are not accepted and the p-values are all zero. It is evident that the entropies are all different from the input whereas the entropy of the AVHEQ approach is the highest at 7.520.

Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

14

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx Table 1 Hypothetical test results. Method

Mean

Hypo

p-Value

Brightness error Input UNFHE BBHE DSIHE BHEPL BHEPLD AIEBHE MMSICHE SMHEQ AVHEQ

0.000 0.027 0.007 0.001 0.027 0.025 0.042 0.005 0.028 0.008

N/A 1 0 0 1 1 1 0 1 0

N/A 0.000 0.316 0.926 0.000 0.001 0.000 0.371 0.000 0.285

Entropy Input UNFHE BBHE DSIHE BHEPL BHEPLD AIEBHE MMSICHE SMHEQ AVHEQ

7.321 7.166 7.180 7.181 7.104 7.103 7.022 7.113 7.156 7.520

N/A 1 1 1 1 1 1 1 1 1

N/A 0 0 0 0 0 0 0 0 0

Contrast Input UNFHE BBHE DSIHE BHEPL BHEPLD AIEBHE MMSICHE SMHEQ AVHEQ

0.507 0.572 0.514 0.527 0.454 0.459 0.431 0.490 0.564 0.536

N/A 1 1 1 1 1 1 1 1 0

N/A 0.000 0.560 0.113 0.000 0.000 0.000 0.146 0.000 0.051

Gradient Input UNFHE BBHE DSIHE BHEPL BHEPLD AIEBHE MMSICHE SMHEQ AVHEQ

0.031 0.045 0.043 0.044 0.032 0.033 0.032 0.035 0.038 0.036

N/A 1 1 1 0 0 0 1 1 1

N/A 0.000 0.000 0.000 0.341 0.215 0.248 0.004 0.000 0.000

The hypothetical test result for the contrast criterion shows that the SMHEQ approach performs well where the null hypothesis is rejected. However, the mean contrast of the AVHEQ approach is higher than six other approaches in the comparison as observed from the box plot. For the gradient criterion, the null hypothesis is not accepted while the p-value tends to zero. This indicates a higher contrast was provided by the AVHEQ approach. Moreover, its mean value is larger than the input image gradients.

5. Conclusion In this paper, a new procedure for color image contrast enhancement and preservation of mean brightness had been presented. This method is able to release the need to separate the input image into sub-images and equalized independently. It also does not need to clip histograms to prevent artifacts. Instead, a histogram averaging strategy was adopted to produce the target distribution used in the histogram equalization process. The choice of the averaging threshold was optimized by the efficient golden section search algorithm. Furthermore, an intensity re-mapping stage was included to alleviate the generation of artifacts. Together with the color channel stretching pre-processing, the proposed approach is able to produce contrast enhanced images that are more desirable than current available methods in terms of brightness preservation, increased information content, object gradient sharpness and global contrast. Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001

S.C.F. Lin et al. / Computers and Electrical Engineering xxx (2015) xxx–xxx

15

References [1] Lin Y-J, Shen C-T, Lin C-C, Yen H-C. A fast image fusion algorithm for image stabilization on hand-held consumer electronics. In: Consumer electronics, 2009. ICCE ’09. Digest of technical papers international conference on; 2009. p. 1–2. [2] Yao Y, Abidi B, Doggaz N, Abidi M. Evaluation of sharpness measures and search algorithms for the auto focusing of high-magnification images. In: Defense and security symposium, international society for optics and photonics; 2006. p. 62460G–62460G. [3] Wu T, Wu H, Du Y, Kwok N, Peng Z. Imaged wear debris separation for on-line monitoring using gray level and integrated morphological features. Wear 2014;316(1-2):19–29. [4] Zhang G, Jia X, Kwok NM. Super pixel based remote sensing image classification with histogram descriptors on spectral and spatial data. In: International geoscience and remote sensing symposium (IGARSS); 2012. p. 4335–4338. [5] Yu Y-H, Kwok NM, Ha QP. Color tracking for multiple robot control using a system-on-programmable-chip. Automat Construct 2011;20(6):669–76. [6] Su F, Fang G, Kwok NM. Shadow removal using background reconstruction. In: 2012 5th International congress on image and signal processing, CISP 2012; 2012. p. 154–8. [7] Chaudhary A, Raheja JL. Bent fingers angle calculation using supervised ann to control electro-mechanical robotic hand. Comput Electric Eng 2013;39(2):560–70. [8] Kwok NM, Shi HY, Ha QP, Fang G, Chen SY, Jia X. Simultaneous image color correction and enhancement using particle swarm optimization. Eng Appl Artif Intell 2013;26(10):2356–71. [9] Kwok NM, Shi HY, Fang G, Ha QP. Intensity-based gain adaptive unsharp masking for image contrast enhancement. In: 2012 5th International congress on image and signal processing, CISP 2012; 2012. p. 529–33. [10] Gonzalez RC, Woods RE. Digital image processing. 3rd ed. Upper Saddle River, NJ (USA): Prentice-Hall, Inc.; 2006. [11] Wang C, Peng J, Ye Z. Flattest histogram specification with accurate brightness preservation. Image Process, IET 2008;2(5):249–62. [12] Kim Y-T. Contrast enhancement using brightness preserving bi-histogram equalization. IEEE Trans Consumer Electron 1997;43(1):1–8. [13] Wang Y, Chen Q, Zhang B. Image enhancement based on equal area dualistic sub-image histogram equalization method. IEEE Trans Consumer Electron 1999;45(1):68–75. [14] Ooi CH, Kong NSP, Ibrahim H. Bi-histogram equalization with a plateau limit for digital image enhancement. IEEE Trans Consumer Electron 2009;55(4):2072–80. [15] Liang K, Ma Y, Xie Y, Zhou B, Wang R. A new adaptive contrast enhancement algorithm for infrared images based on double plateaus histogram equalization. Infrared Phys Technol 2012;55(4):309–15. [16] Singh K, Kapoor R. Image enhancement using exposure based sub image histogram equalization. Pattern Recognit Lett 2014;36:10–4. [17] Zhang H, Li Y, Chen H, Yuan D, Sun M. Perceptual contrast enhancement with dynamic range adjustment. Optik-Int J Light Electron Opt 2013;124(23):5906–13. [18] Ooi CH, Isa NAM. Adaptive contrast enhancement methods with brightness preserving. IEEE Trans Consumer Electron 2010;56(4):2543–51. [19] Tang JR, Isa NAM. Adaptive image enhancement based on bi-histogram equalization with a clipping limit. Comput Electric Eng 2014;40(8):86–103. [20] Singh K, Kapoor R. Image enhancement via median-mean based sub-image-clipped histogram equalization. Optik-Int J Light Electron Opt 2014;125(17):4646–51. [21] Kwok NM, Jia X, Wang D, Chen SY, Fang G, Ha QP. Visual impact enhancement via image histogram smoothing and continuous intensity relocation. Comput Electric Eng 2011;37(5):681–94. [22] Chong E, Zak S. An introduction to optimization. Wiley series in discrete mathematics and optimization. Wiley; 2013. S.C.F. Lin received his Mechatronic Engineering degree in Mechanical and Manufacturing Engineering from the University of New South Wales, Australia, in 2010. He is currently a Ph.D. student in the same university. His major research interests include image processing, robotics and control system. C.Y. Wong acquired his Mechatronic Engineering degree in Mechanical and Manufacturing Engineering from the University of New South Wales, Australia, in 2010. He is now a Ph.D. student in the same university. His major research interests are image processing, robotics and control system. M.A. Rahman received his B.Sc degree in Electrical and Electronic Engineering from Bangladesh University of Engineering and Technology, Bangladesh in 2007. He had worked as professional engineer in Ericsson before he started his Ph.D. in the University of New South Wales, Sydney, Australia in 2013. His major research interests include machine vision, robotics and control system. G. Jiang received Bachelor of Mechatronic Engineering from Xiamen University, China in 2009 and Master Degree from University of New South Wales in 2011. He is now a PhD student in the University of New South Wales. His research interests include Image processing and automatic control. Shilong Liu received his Master degree in Mechatronic Engineering at Beijing Institute of Technology, China, in 2014. He is now a PhD student in Mechanical and Manufacturing Engineering at the University of New South Wales, Australia. His main research interests are image processing, computer vision and applied mathematics. Ngaiming Kwok received the MPhil degree from Hong Kong Polytechnic University, China, in 1997 and PhD degree from University of Technology Sydney, Australia, in 2007. He is now a lecturer with the University of New South Wales, Australia. His research interests include image processing, intelligent computation and automatic control. Haiyan Shi obtained the MSc degree and the PhD degree in Control theory and control engineering from Zhejiang University of Technology, Hangzhou, China in 2010 and 2013 respectively. She is with the School of Computer Science and Engineering, Shaoxing University, since 2000. Her research interests include wireless sensor networks, image processing and intelligent computation. Ying-Hao Yu obtained his Ph.D. degree in Mechanical and Mechatronic Systems from University of Technology Sydney, Australia. He is currently an Assistant Professor in Department of Electrical Engineering, National Chung Cheng University, Taiwan. His research interests include FPGA designs, ubiquitous intelligence, machine vision and learning, and robotic control. Tonghai Wu is currently Associate Professor in Key Laboratory of Education Ministry for Modern Design and Rotor-Bearing System, Xian Jiaotong University, China. He obtained his PhD degree from Xian Jiaotong University, China, in 2006. His research interests include tribology system and image based intelligent monitoring.

Please cite this article in press as: Lin SCF et al. Image enhancement using the averaging histogram equalization (AVHEQ) approach for contrast improvement and brightness preservation. Comput Electr Eng (2015), http://dx.doi.org/10.1016/j.compeleceng.2015.06.001