Feature based fuzzy inference system for segmentation of low-contrast infrared ship images

Applied Soft Computing 46 (2016) 128–142

Contents lists available at ScienceDirect

Applied Soft Computing journal homepage: www.elsevier.com/locate/asoc

Feature based fuzzy inference system for segmentation of low-contrast infrared ship images Xiangzhi Bai a,b,∗ , Miaoming Liu a , Tao Wang a , Zhiguo Chen a , Peng Wang a , Yu Zhang a a b

Image Processing Centre, Beijing University of Aeronautics and Astronautics, Beijing 100191, China State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100191, China

a r t i c l e

i n f o

Article history: Received 17 December 2015 Received in revised form 29 April 2016 Accepted 4 May 2016 Available online 6 May 2016 MSC: 03B52 68U10 Keywords: Low-contrast infrared ship image Intensity feature Spatial feature Mathematical morphology Fuzzy inference system

a b s t r a c t Segmentation of infrared ship target is important for sea surveillance system. However, as a result of the deficiencies of infrared images, the segmentation of infrared ship image becomes a challenge. For the purpose of addressing this problem, a feature based infrared ship image segmentation method utilizing the fuzzy inference system is proposed. Firstly, the intensity feature is extracted by applying unimodal threshold, which could preserve the low-contrast pixels in the infrared images. Secondly, the local spatial feature is extracted by employing saliency detection, region growing and morphology processing, which could express the shape of the target. Thirdly, the global spatial feature is extracted by utilizing partial region growing and weighted distance transformation, which could suppress the background. Then these features are fuzzified using accommodative ways and prior knowledge. And in light of the fuzzy rules based upon expert knowledge, these fuzzified features are integrated in fuzzy inference system. Finally, the complete target could be directly segmented from the output of the fuzzy inference system. Experimental results illustrate that the proposed method could effectively extract more intact targets from the low-contrast infrared ship images. Additionally, the proposed method outperforms some existed segmentation methods. © 2016 Elsevier B.V. All rights reserved.

1. Introduction Infrared thermography technology could transform the infrared radiation emitted from objects to imaging sensor [1–3]. The emitted radiant intensity mainly depends on the temperature, the emissivity and the infrared wavelength interval [4]. In the condition of inadequate light, infrared imaging systems could effectively detect the target. Therefore, they are widely applied in secure surveillance, military reconnaissance, missile guidance and other fields. Segmentation of ship target in infrared image is a significant step for automatic target recognition in the surveillance system [5–8]. As a result of poor imaging circumstances in sea surface, the infrared image usually has low-contrast and may be affected by sea clutters and noises [9]. And sometimes the infrared ship target even submerges in the sea background after imaging by infrared thermal imaging system. Hence, segmentation of the ship target in low-quality infrared images becomes a challenge.

∗ Corresponding author at: Image Processing Centre, Beijing University of Aeronautics and Astronautics, Beijing 100191, China. E-mail address: [email protected] (X. Bai). http://dx.doi.org/10.1016/j.asoc.2016.05.004 1568-4946/© 2016 Elsevier B.V. All rights reserved.

A large amount of methods have been proposed to settle the problem of image segmentation, such as threshold based methods [10–13], clustering based methods [14–16], active contour based methods [17], and so on. The Otsu’s method [10], entropy based method [11] and minimum error thresholding [13] are popular thresholding strategies for segmentation, which manage to look for an optimal segmenting threshold through the statistical information of an image. Whereas the intensity difference between the ship target and background in the infrared ship images with low contrast is unobvious. And, generally, there is no optimal threshold since the infrared ship target is small comparing with the background in most cases. In addition, the spatial information in infrared ship images is not applied in the threshold based methods. Besides, the Otsu’s method usually under-segments the small target and the entropy based method is easy to be interfered by the intricate ambient noises. These make the segmentation result unsatisfying. And the minimum error thresholding is sensitive to the noises and the ratio of area between the target and background. Since the low-contrast infrared images are mostly very complex, the good segmentation results cannot be obtained when only using a threshold. In a word, the Otsu’s method [10], entropy based method [11] and minimum error thresholding [13] would be affected by the noises and cannot obtain satisfying

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142

segmentation results for low-contrast infrared images. The 2D maximum entropy method [12] considers both the distribution of gray information and the spatial neighbor information. Nevertheless, it is sensitive to the contrast of the ship target and background regions. Similar to the entropy method, it is difficult for the 2D maximum entropy method to find a proper threshold when processing the low-contrast images. In addition, this method is sensitive to the ratio of the pixel numbers between the target and background. If the ship target is much smaller than the background, this method may regards the target as background. As a result, the 2D maximum entropy method [12] may fail to extract the target from the low-contrast infrared ship images. As we all know, the mean shift [14] and fuzzy c-means method (FCM) [15] are clustering based methods for image segmentation. Nevertheless, these methods are easily affected by noises, thus the segmentation results of these methods might produce bad effects because of the complexities of the infrared ship images. Moreover, these methods might wrongly regard the background as ship target when the spatial information is not appropriately used. Consequently, the segmentation results of mean shift [14] and fuzzy c-means method (FCM) [15] may not be satisfying due to noises and low contrast. The spatial fuzzy c-means method (SFCM) [16] integrates spatial information into the membership function on the purpose of clustering. However, SFCM is also sensitive to the contrast and the ratio of pixel number between the target and background. The Chan–Vese model [17] is accommodative to the topology structure of segmented target and is frequently-used in image segmentation, which is a method grounded on active contour. But it is difficult to extract the topology structure of the target, because both the target and background regions are inhomogeneous in the infrared ship image with low contrast and there is no obvious difference between the intensities of them. The level set method [18] is robust to noises and independent to the initial contour. Moreover, the level set method obtains good results in segmenting the images with inhomogeneous intensity. Nevertheless, the ship target may not be segmented from the infrared image when the target is small or the background is complex in the low-contrast infrared images. The marker based watershed method [19–22] uses the makers generated to acquire the contour between the target and background, which could produce good segmentation results. However, the marker based watershed may not segment some detail parts of ship target from the infrared ship images with low contrast, such as the masts. To extract the ship target, an iterative technique is taken to efface background and enhance the infrared ship target [4]. Nevertheless, since the target is not very bright at most cases and the contrast is low, some of the target regions might be erased with the background. Consequently, the iterative method may not acquire the complete target from the low-contrast infrared images. Recently, an improved fuzzy c-means method grounded on the spatial information has been proposed [23,24], which utilized the non-local space information and refined the local spatial constraint through Markov Random Field with spatial shape information from the contour of target. Also, the intact ship target may not be extracted from the infrared ship image when the target is large or the contrast is low. In a word, these methods may not produce satisfying results for the segmentation of low-contrast infrared ship image or images with noises. Since there are lots of defects in the infrared ship image, a feature based infrared ship image segmentation method utilizing fuzzy inference system is proposed for ship target segmentation. The fuzzy inference system plays an important part in lots of theories and application fields, such as pattern recognition, data compression, system identification, control, decision making, supervision and some other fields [25–29]. There are several advantages for fuzzy inference system [30–35]: first, fuzzy inference system could represent expert knowledge; second, it is convenient to understand

129

and apply fuzzy rules to many fields since these rules are transparent and interpretable; third, fuzzy inference system could export a target degree to address general complex problems. In infrared ship image, the priori knowledge could be extracted to represent the ship target. In addition, the fuzzy rules could be designed based on the priori knowledge and the ship target could be extracted from the target degree map. Therefore, the fuzzy inference system could handle the segmentation of low-contrast infrared ship image. This paper is a substantial extension of our conference paper, which has been presented in an international conference [36]. The mainly extensions are as follows. Firstly, a new global spatial feature has been added as the third input of the fuzzy inference system. Secondly, 27 rules have been designed to express the expert knowledge. Thirdly, the experiments have been conducted on a completer dataset, which contains 80 infrared ship images with low contrast. Fourthly, more analyses about the method and the experimental results have been displayed. Fifthly, more image segmentation methods have been employed for comparisons, including 2D maximum entropy based method (2D Entropy) [12], minimum error thresholding (MinError) [13], spatial fuzzy c-means method (SFCM) [16], level set method (Level Set) [18], marker based watershed method (Watershed) [19–22] and Markov Random Filed constraint SFCM method (MRF-SFCM) [23,24]. To make the best of information in infrared ship images, the intensity feature, local spatial feature and global spatial feature are constructed. Besides, fuzzy inference system is capable of effectively dealing with the uncertainty in an image and easily expressing the expert knowledge with fuzzy rules. So we utilize the fuzzy inference system to integrate the features and segment the target. Firstly, the intensity feature is selected as one input of the fuzzy inference system, which is a fundamental feature of target in the infrared images. Secondly, two spatial features are extracted to signify the spatial information of infrared ship image. The local spatial feature is extracted to determine the contour of the infrared ship, while the global spatial feature could be used to suppress the background. Three steps are taken to construct the local spatial feature: saliency detection, region growing and morphology processing. And a weighted distance transformation map is used to construct the global spatial feature. Thirdly, these features are accommodatively fuzzified with statistical information of infrared ship image and prior knowledge. Then, according to expert knowledge, the fuzzy rules are defined to integrate with the three fuzzified features. Finally, the ship target is extracted from the target degree map from the fuzzy inference system, which achieves the segmentation of the ship target. Experimental results indicate that the proposed method is effective to extract the intact targets from the infrared ship images with low contrast and outperforms some existed segmentation methods. The pre-processing and post-processing procedures have contributed to the segmentation of infrared ship images. However, as far as we known, the pre-processing and post-processing steps are challenging tasks for the infrared ship images due to the existence of low contrast and noises. Actually, to obtain better segmentation results, we directly considered the low contrast and noises of the infrared images when designing the feature based fuzzy inference system for segmentation of the infrared images. Thus, there is no other pre-processing or post-processing steps in our algorithm. The novelty of the paper is the constructions of the three features. Firstly, the intensity feature is constructed using the unimodal threshold, which is consistent with the intensity distribution in the infrared images. Thus, the intensity feature segments the target from the sea background and helps to preserve the lowgray pixel in the low-contrast infrared images. Secondly, the local spatial feature is designed by employing saliency detection, region growing and morphology processing. The local spatial feature could signify the shape information of the target due to the application of

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142

Input Image Intensity Feature

Local Spatial Feature

Global Spatial Feature

Frequency

130

Background

Potential target region

Ppeak

Fuzzification Fuzzy Inference System

Fuzzy Inference Engine

Fuzzy Rules

Defuzzification Target Degree Segmentation Result Fig. 1. Flowchart of proposed method.

morphology processing, which helps to reduce the effects of noises. Thirdly, the global spatial feature is constructed by a weighted distance transformation map, which could suppress the background. The combination of these three features in fuzzy inference system could reduce the effect of noises and obtain satisfying segmentation results in most cases for the segmentation of infrared images with low contrast. The reminder of this paper is organized as follows. Section 2 depicts the proposed method in details. Section 3 demonstrates the main experiment results and discussions comparing with some existed segmentation methods. Finally, the conclusions are given in Section 4. 2. The proposed method The infrared ship images are gray-scale image and they do not contain the multichannel color information or other information. It is tough to make the best of the limited information for infrared ship image segmentation. Thus, it is necessary to construct features those could represent the ship target in infrared image from the limited information. There are relationships between the feature extraction and the construction of fuzzy inference system knowledge base. If we extract too many features, the rules of fuzzy inference system will increase dramatically, which makes the problem much more complex. Besides, if the features do not contain the information which is conformed with human understanding, the design of the fuzzy rules will become a conundrum. Thus, we extract the intensity feature, local spatial feature and global spatial feature based on the characteristics of infrared ship images. These three features make use of the information of the images and express the actual meaning of the ship target. What is more, it is convenient to design the fuzzy inference system rules using these three reduced features. Fig. 1 depicts the flowchart of our method, which includes these procedures: feature extraction, fuzzification, design of fuzzy rules, fuzzy inference, defuzzification and segmentation. The specific procedures are described as follows. 2.1. Intensity feature The objects having higher temperature or emitting more heat are mapped as regions with higher intensity in infrared images [6]. In our image dataset, the ship target region has higher intensity than the surrounding sea background in infrared images. This is because the ship targets which transmits more heat are mapped

PBVD

xp

xTh

Pend

Intensity

Fig. 2. The sketch of unimodal threshold.

as areas with higher intensity in infrared images of our dataset. However, because of heat exchanging between the ship and sea surface, the contour of the ship target is ambiguous in most situations, which would make the image segmentation complicate. In a word, it is pivotal for infrared ship image segmentation to reasonably utilize the intensity feature. What is more, the traits of the intensity distribution in infrared images are analyzed. Generally, compared to the background, the ship target is smaller. Also, the ship target has higher intensity than the sea background. Thus, the distribution of the histogram for infrared ship image would be unimodal. Then, the background is arranged as a distinct peak at the low gray level while the ship target lies on high gray level. The valley of the histogram stands for the transition region from the background to the ship target. Thus, we can utilize the unimodal threshold [37] to construct the intensity feature. Fig. 2 shows the sketch of unimodal threshold, which is a method to compute the unimodal threshold. More specifically, the unimodal threshold is used to value the membership functions. These functions are used to convert the intensity feature to fuzzy sets. As demonstrated in Fig. 2, we draw a line from the peak point Ppeak to the end point Pend in the histogram. And the point PBVD whose maximum vertical distance to the line are searched in the curve. Then, the intensity value xTh of the point PBVD is regarded as the unimodal threshold. The unimodal threshold could segment the infrared ship target if the area ratio of the ship target and background is not small and the background is not very complex. Otherwise, large amount of background would be introduced into the segmentation results. Hence, it is necessary to construct the intensity feature. The infrared ship images may appear inhomogeneous as a result of sea clutters and the inordinate attenuation when transmitting the images. Thus, there may exist some singular points which do not change along with the distribution trend of the histogram in the gray level histogram as presented in Fig. 3. We classify the singular points into two types. The first type of the singular point whose grayscale frequency is close to zero is located at the bottom of the histogram, which interrupts the distribution curve as shown in Fig. 3 (green diamond-shape point). Another type of the singular point is a point whose grayscale frequency is much less than its neighbor, which will cause obvious pits in the histogram as shown in Fig. 3 (red square-shape point). These two types of singular points are caused by the infrared ship imaging system, and they would have an effect on the performance of segmentation algorithm. When searching the threshold by unimodal threshold iteratively, the distance from the singular point to the line may be larger than the distance from the actual valley point PBVD behind the main peak to the line. Thus a local extrema would be obtained as the unimodal threshold. And the unimodal threshold is smaller than the actual threshold we need. As a result, a large amount of background will be considered as the ship target. In other words, the ship target would be submerged in the background.

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142

131

start obtain the histogram find the peak point Ppeak

draw a line from the peak point Ppeak to the end point Pend all points have been traversed

Fig. 3. The singular points in the gray level histogram. (For interpretation of the references to color in the text, the reader is referred to the web version of the article.)

To handle this problem, we eliminate the singular points to improve the unimodal threshold and avoid taking the local extrema as the unimodal threshold. For the first type of singular points, if its ordinate is zero, then we skip to the next point. As for the second type, the differences |LB − LA| and |LB − LC| are calculated. Point A and point C are the neighbors of point B as shown in Fig. 3. LA, LB and LC are the vertical distances of point A, B and C to the line, respectively. If |LB − LA| and |LB − LC| are both larger than the threshold VDTh , then skip this point. Through a lot of experiments, the optimal value of VDTh could be 15. The flowchart of the improved unimodal threshold is described in Fig. 4.

Due to numerous sea clutters and noises in the infrared image, the target could not be extracted accurately using only the intensity feature. Hence, the local spatial feature is extracted to signify the contour of the ship target in infrared image. Three procedures are employed to construct the local spatial feature: (1) salient region detection; (2) ship target region growing; (3) morphology processing. The schematic diagram of the local spatial feature construction is depicted in Fig. 5. The specific procedure is illustrated as follows. 2.2.1. Salient region detection Saliency is a crucial feature of images, which represents the visual attention of human [38–41]. In an infrared image, the intensity value of the ship target is higher than that of surrounding background, so the ship target usually has higher saliency than its surrounding background. Thus, we could approximatively find the location of the ship target through the most salient region of the infrared image. It is known that the graph-based visual saliency (GBVS) [39] could effectively calculate the salient region in the image. Therefore, GBVS are employed to search for the most salient regions, which indexes the position of the target. The specific procedure is performed as follows. First of all, we obtain the GBVS feature map [39]. Then, we apply a threshold to get the most salient region from infrared ship image. To exclude the sea background, the threshold value should be great enough. In this paper, we choose the top 10% salient pixels to acquire the approximate location of the ship target. The most salient region indexes the possible position of the target in infrared ship image. The expression is written as BWsaliency (x, y) =

No its ordinate is 0

Yes

No compute vertical distance skip to the next point

all points have been traversed?

eliminate the second type of the singular point

Yes

No vertical distance differences Yes are larger than VD

No renew PBVD skip to next point

2.2. Local spatial feature



eliminate the first type of the singular point

Yes

1,

Isaliency (x, y) ≥ 0.9 × Imax

s

0,

Isaliency (x, y) < 0.9 × Imax

s

,

take the abscissa of PBVD as unimodal threshold xTh

end Fig. 4. The flowchart of the improved unimodal threshold.

where Isaliency denotes the saliency map of infrared ship image, Imax s signifies the maximum value of Isaliency , BWsaliency represents the most salient region in infrared ship image. 2.2.2. Ship target region growing After acquiring the most salient region in infrared image, the location of target could be approximately determined. However,

(1) Fig. 5. The flowchart of constructing local spatial feature.

132

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142

it is still difficult to depict the local spatial feature of the ship target. Thus, we employ the acquired salient region and region growing technique [42] to obtain a comparatively complete ship target region BWship−region . Through region growing, the neighborhood pixels of the seed point are used as seed points iteratively and the seed points are regarded as pixels in the ship target region. Thus, the pixels pertaining to the target can be sought out. The explicit calculation steps of the comparatively intact ship target region BWship−region are described as followed. Step 1: Adopt the center of BWsaliency as the seed point. As BWsaliency approximately represents the location of the ship target, the center of this region could be regarded as the seed point for the region growing. Step 2: Check the 8-connected neighborhood pixels of the seed point, if the gray value of the pixel is greater than xTh , this neighbor pixel is regarded as the seed point for further growing. The unimodal threshold xTh could classify the ship image into target and sea background. If the gray value of the pixel is greater than xTh , then this pixel pertains to ship target region with high possibility. Through this way, the pixels pertaining to the target can be sought out. Step 3: Continue checking the 8-connected neighborhood pixels of the newly acquired seed points, until there are no new seed pixels being sought out. After this iterative procedure, the comparatively complete ship target region BWship−region could be acquired. 2.2.3. Morphology processing Since the target in the infrared ship image is not bright, the contour of the ship target is hard to be determined. The ideal local spatial feature represents the membership degrees of the pixels. In other words, if the pixel is located in the ship target region, the value of the local spatial feature (LSF) would be 1; if the pixel is located in the background, the value of the LSF would be 0; if the pixel is located in transition region between the background and ship target, the value of the LSF would range in [0, 1]. Larger value of the LSF indicates the pixel is closer to the ship target. However, there are two difficulties to construct such LSF: (1) the selection of the distance metric; (2) the definition and implement of the transition region from 0 to 1. On the one hand, if we choose the distance from the pixel to the center of BWship−region as LSF directly, the LSF would become a group of concentric circles. As a result, the constructed LSF would not contain the accurate topological shape feature in BWship−region . On the other hand, the heat exchange between the ship target and sea background in the transition region makes it harder for locating the contour of ship target. Therefore, defining the transition value according to the distance without considering the characteristics of ship target is infeasible for constructing the LSF. To depict the LSF more adequately, morphological techniques [43–45] are employed, which have been successfully applied in segmentation [46,47,19,48–50]. The multi-scale rectangle structuring elements Bi = B1 ⊕ B1 ⊕ · · ·B1 ,







i = 1. . .n,

(2)

i

are utilizing to process the ship target region BWship−region . And the explicit calculations are formulated as C1 = BWship−region ⊕ B1 − BWship−region ,

(3)

Ci = BWship−region ⊕ Bi − BWship−region ⊕ Bi−1 , ILS = BWship−region +

n   i=1

1−

i n



i = 2. . .n,

(4)



× Ci ,

(5)

Fig. 6. A basic example of the morphology processing. (a) A cropped image. (b) The ship target region BWship−region . (c) The region C1 and region C2 . (d) The final local spatial feature of ship target ILS .

where ⊕ is the morphological dilation operation; B1 is rectangle structuring element; Bi is the morphological structuring element, which is i times of dilation of B1 ; Ci is a group of ring around BWship−region ; (1 − (i/n)) is the value of Ci ; ILS is the final LSF of ship target. ILS is composed by BWship−region and a group of ring Ci around BWship−region . The detail procedures are described as follows: Step 1: Subtract BWship−region from BWship−region dilated by B1 , the region C1 nearing the possible ship target are extracted. Step 2: Subtract BWship−region dilated by Bi−1 from BWship−region dilated by Bi , the region Ci are extracted. Step 3: Continue to compute the next scale iteratively using Step 2, and then a group of Ci at the scale i nearing the possible target are obtained. These Ci would retain the shape features of ship target. Step 4: Assign the final LSF with the corresponding damped value. In other words, the value in the ship target BWship−region is 1, and the value in the region Ci is 1 − i/n. After these three steps, the constructed LSF could anastomose with the transition region and retain the topological shape information of ship target, which could solve the problem of distance metric selection. As is depicted in Fig. 5, the range of final LSF is [0, 1]. The values of pixels outside the ship target region in the LSF have an inverse correlation with the distance from the pixels to target region. Therefore, the LSF could indicate the local spatial relations of pixels with the target region. To clearly explain the procedure of the morphology processing, a basic example is demonstrated in Fig. 6. Fig. 6(a) presents a part of the infrared image. After the region growing, we obtain the ship target region BWship−region as presented in Fig. 6(b). Then, the ship target region BWship−region is subtracted from BWship−region dilated by B1 and the region C1 could be obtained. That is, the region C1 is the difference between BWship−region dilated by B1 and BWship−region . After that, the ship target region BWship−region dilated by Bi−1 is subtracted from BWship−region dilated by Bi . Similarly, the region Ci is iteratively produced ranging i from 2 to n. The value of ship target region BWship−region is 1, and the value of the region Ci is (1 − (i/n)). Fig. 6(c) shows the region C1 and region C2 . Finally, as shown in Fig. 6, the final local spatial feature of ship target ILS is produced, which is composed by BWship−region and a group of ring Ci . Generally, the height of ship target is smaller than its width. According to this prior knowledge, the rectangle structuring

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142

133

2.3.2. Weighted distance transformation After obtaining the partial ship target region growing, a global spatial feature could be constructed. Euclidean distance transformation of the ship target BWpartial−ship is computed as follows: D = norm(dist(BWpartial−ship )),

(7)

where partial ship target BWpartial−ship and distance transform map D are matrixes; dist(•) is the Euclidean distance transformation [51,52], and norm(•) is the normalization, which is dividing dist(BWpartial−ship ) by the smallest negative number. Then, the distance transform map D is filtered by a smoothed Heaviside function [53] Hε (x) =



1 2 1 + arctan 2 

x  ε

(8)

with ε = 0.5 in our experiment. The formula of this procedure can be written as DH = norm(Hε (−D)).

Fig. 7. The flowchart of constructing global spatial feature.

element is employed in morphology processing. The height B Height and width B Width of the structuring element B1 are 3 and 5, respectively. The scale n is 3, which represents the range of ambiguous contour of the ship target. 2.3. Global spatial feature In infrared ship image, there would be complex background that affects the segmentation results. And there are some backgrounds whose gray values are very similar to the gray value of the ship target. In the region growing result with unimodal threshold, these backgrounds may be wrongly classified as ship target. To extract the infrared ship target and suppress these backgrounds, a global spatial feature is constructed as followed: (1) obtain partial ship target by region growing; (2) weighted distance transformation. The distance transformation [51,52] is performed in the partial ship target with a weight. The step of the global spatial feature construction is depicted in Fig. 7. The step is detailed as follows. 2.3.1. Obtain partial ship target by region growing To avoid the ship target being submerged in the background, part of the ship target region growing results in the last subsection is used to wipe off the background. Based on the step of local spatial feature construction, a threshold, which is higher than the unimodal threshold xTh , is defined for region growing. The threshold xThH to get the partial ship target BWpartial−ship is defined as

(9)

The normalization is dividing Hε (−D) by the smallest negative number. After normalization, we could make sure that D and the smoothed distance transform map DH range from 0 to 1. However, a simple distance transformation cannot represent the relationship between the ship target and the complex background. With an unweighted distance transformation, the output of fuzzy inference system may yield a bad result and the infrared ship target would not be extracted. Thus, a weight is added to the distance transformation. In GBVS saliency map, the ship target is more salient than the background. Hence, we define the weight as ω = norm(Isaliency (x, y)),

(10)

where norm(•) denotes the normalization; Isaliency (x, y) represents the saliency map of infrared ship. The saliency map Isaliency (x, y) are not even especially at the ship target. Sometimes, the values of the pixels in Isaliency (x, y) at the ship target are close to 0, which makes the fuzzy inference system fails to work. Thus, a small constant is added to the saliency map Isaliency (x, y) to ensure all weights are larger than 0. To obtain better segmented results, the small constant is set to 0.2. Thus, the weighted distance transformation can be defined as IGS = w × DH,

(11)

where IGS is the global spatial feature, which is one input of the fuzzy inference system. The weight w and the smoothed distance transform map DH range from 0 to 1. Thus, the value of IGS is between 0 and 1. 2.4. Fuzzy inference system

(6)

In this subsection, as shown in Fig. 8, three features, including intensity feature, local spatial feature and global spatial feature, are inputted into the Mamdani [54–56] fuzzy inference system to acquire the target degree map. The fuzzy inference system is composed of the following three procedures: (1) fuzzification; (2) design of fuzzy rules; (3) defuzzification.

where 0 <  < 1. Using the threshold xThH for region growing, the partial ship target BWpartial−ship could be produced. If  is close to 0, the obtained partial ship target would still contain part of the background. If  is close to 1, xThH must be close to 255. Then the obtained partial ship target would be very small and may degenerate into a point. In this case, the partial ship target would not be obtained. In our experiment,  is set as the first quartile, that is  = 0.25. With this value, a partial ship target will be obtained to construct the global spatial features as an element.

2.4.1. Fuzzification Fuzzification converts the features inputted into fuzzy sets. The selections of membership functions impact the performance of fuzzy inference system. Then, we adopt proper membership functions for the constructed features. There are three descriptions in each fuzzy input, including low (L), medium (M) and high (H). Firstly, according to the intensity distribution of infrared ship image, the parameters in the membership functions of the intensity feature are accommodatively decided.

xThH = xTh +  × (255 − xTh ),

134

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142

Intensity Feature

1

0.5

L

H

M

0 0

100

Target Degree

200

1

Local Spatial Feature

1

L

0.5

M

0 0

FIS Mamdani 27 Rules

H

0.5

0 0

1

L

M

0 0

M

H

1

Global Spatial Feature

0.5

L

0.5

0.2

0.4

0.6

0.8

1

H

0.5

1

Fig. 8. Fuzzy inference system (FIS).

Though the contrast of the infrared image is low and the contour of the target is hard to be identified, the ship target and background could also be coarsely splitted by the unimodal threshold. Therefore, the parameters in the intensity feature membership functions could be acquired through the unimodal threshold. Then the frequently-used triangular and trapezoidal functions [57] are chosen as the membership functions to map the intensity feature to fuzzy set. The values of the set range from 0 to 1. More specifically, the membership functions and parameter settings are described as follows:

g

Low (x)

=

⎧ 1 x≤a ⎪ ⎪ ⎨ x−b a−b ⎪ ⎪ ⎩ 0

g

Medium (x)

=

a≤x≤b ,

x≥b

⎧ x−a a≤x≤b ⎪ ⎪ ⎨ b−a 0

⎪ ⎪ ⎩ x−c

b−c

g

High (x)

=

(12)

(13)

x ≤ a or x ≥ c , b≤x≤c

⎧ 0 x≤b ⎪ ⎪ ⎨ x−b c−b ⎪ ⎪ ⎩ 1

b≤x≤c ,

(14)

x≥c

where x denotes the intensity value whose range is in [0, 255]; xp represents the most frequent intensity in the histogram of the infrared image; xTh denotes the unimodal threshold; xwidth = xTh − xp is the width between the unimodal threshold and the most frequent intensity; a, b and c signify the parameters in the membership functions of the intensity feature, which are respectively determined by



a=

xp ,

0 < xp < xTh

xTh − xwidth /2,

else

,

(15)

b = xTh ,



c=

(16)

xTh + xwidth ,

xTh < xTh + xwidth < 255

xTh + xwidth /3,

else

;

(17)

g Low (x), g Medium (x), and g High (x) denote the membership functions of intensity feature, which are employed to fuzzify the

intensity feature. The diagrams of the intensity feature membership functions are depicted in Fig. 8. Secondly, trapezoidal functions [57] are chosen as the membership functions of the two spatial features respectively, and the parameters are determined based on prior knowledge. These membership functions are used to map the local spatial feature and global spatial feature to fuzzy sets. Through salient region detection and region growing, the ship target BWship−region can be obtained. Then the ship target region BWship−region is dilated using the multi-scale rectangle structuring elements Bi and the final local spatial feature ILS of the ship target is produced. The values of ILS within BWship−region are 1, while the values of ILS within Ci are (1 − (i/n)) as shown in Fig. 5. If a pixel around the ship target has a higher possibility belonging to the ship target region, the value of it in the final local spatial feature ILS would be larger. This means that local spatial feature ILS has a degree around the ship target and represents the local spatial feature of infrared ship. The values of the local spatial feature range from 0 to 1. Likewise, the global spatial feature IGS of ship target has a degree ranging from the ship target to the background. And the value of IGS is ranging from 0 to 1 as presented in Fig. 7. The membership functions of the two spatial features are depicted in Fig. 8. Through fuzzification, the intensity feature membership g , the local spatial membership ls and the global spatial membership gs are acquired. 2.4.2. Fuzzy rules In fuzzy inference system, the fuzzy rules are employed to express the expert knowledge. The structure of these fuzzy rules is IF X THEN Y, where X and Y are labels of fuzzy sets [58]. According to the descriptions of three features, proper fuzzy IF-THEN rules are designed to model these features for the segmentation of infrared ship images. All these three fuzzy inputs have three descriptions, thus there are 27 combinations in total. The output of target degree is denoted as y, and the fuzzy rules designed are presented in Table 1. 2.4.3. Defuzzification In this subsection, the centroid method [56,59] is employed to transform fuzzy set y into target degree map IR . And the approach for calculating is expressed by

 y (y)y dy , IR =  y (y) dy

(18)

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142 Table 1 Fuzzy rules.

135

The infrared ship image dataset is provided by Liu et al. [4], which contains 80 ship images in low contrast. There are different ships and different backgrounds in these infrared ship images. The sizes of the images range from 316 × 252 to 688 × 544 pixels. The experiments are conducted in Windows 7 with Inter(R) Core(TM) i5-3470 CPU with 3.20 GHz and 4.00 GB RAM.

daytime. The weather condition and the background in the videos change along with time. The weather of all the 80 infrared ship images includes sunny, cloudy and foggy. The clutter background includes sea, sky, dock and land. In a word, the infrared images of the dataset using for experiments contain various ship targets with various backgrounds. Some experimental results are presented in Fig. 9. Fig. 9(a) presents an infrared image with the targets at the seasky line, which makes it more difficult for segmentation. However, the proposed method still outputs a satisfactory result as depicted in Fig. 9(f). Overall, the proposed method is capable of segmenting the ship targets at the sea-sky line. Fig. 9(b) presents an infrared image with rich details. We can found that the contrast between the sea surface and the mast is low, which makes the ship segmentation difficult. Moreover, the gray values of the mast and the dock are similar, which is likely to segment the dock as the ship target. Fig. 9(g) presents the segmentation result of Fig. 9(b). Combined the intensity feature, the local spatial feature and the global feature, the fuzzy inference system could output a completer ship target. From Fig. 9(g), we can find that the details of the mast are completely preserved and the backgrounds are removed. Through the segmentation result, a conclusion could be driven that our method is effective in segmenting the infrared ship images which contain rich details. Fig. 9(c) shows a ship target similar to the ship target in Fig. 9(b) obtained at different times. The intensities of the sea and the sky are uneven, which makes the segmentation task difficult. However, the proposed method segments the intact ship target from the clutter background as shown in Fig. 9(h). In all, the proposed method is able to segment similar ship targets taken at different times. Fig. 9(d) shows an infrared image affected by poor weather. Fig. 9(e) shows the ship target in Fig. 9(d) obtained at different times. Fig. 9(i) and (j) presents the segmentation results of Fig. 9(d) and (e). In Fig. 9(d) and (e), some parts of the ship targets are in low contrast, which is likely to be segmented as the background. By using the intensity feature, the local spatial feature and the global spatial feature, the fuzzy inference system could output an intact ship target. In a word, the proposed method is capable of dealing with the segmentation of the ship target with low contrast. The experimental results tested on all the 80 infrared images illustrate that our method could correctly segment different ship targets in different situations for most of the infrared images, which means that our method is effective.

3.1. Infrared ship image segmentation results

3.2. Baseline experiments

Experiments are conducted on the infrared ship image dataset which contains 80 infrared ship images. The infrared ship images in our dataset are extracted from the infrared videos of different ships obtained at different times. Most of the videos are taken in the

In order to determine the role of each individual feature and different combination of features in the segmentation results, six baseline experiments are performed. The experimental results are depicted in Figs. 10–15.

Rule number

IF g

AND ls

AND gs

THEN y

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

L L L L L L L L L M M M M M M M M M H H H H H H H H H

L L L M M M H H H L L L M M M H H H L L L M M M H H H

L M H L M H L M H L M H L M H L M H L M H L M H L M H

L L M L M M L M H L M M L M H L H H L M M M H H M H H

where y (y) denotes the output fuzzy set, y signifies the output of target degree, and IR represents the target degree map. Greater pixel value in IR indicates higher probability of the ship target. Fig. 8 depicts the membership function of the output target degree map. Since the ship target region is different from the background in IR , a threshold is simply set as 0.7 to acquire the final segmented target ISeg . 3. Experiment results and discussions

Fig. 9. (a–e) The original images. (f–j) The segmentation results of images (a–e).

136

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142

Fig. 10. (a and e) The original images. (b and f) The segmentation results of fuzzy inference system with the intensity feature. (c and g) The segmentation results of the proposed method. (d and h) The ground truth image.

Fig. 11. (a and e) The original images. (b and f) The segmentation results of fuzzy inference system with the local spatial feature. (c and g) The segmentation results of the proposed method. (d and h) The ground truth image. The labeled regions are enlarged as presented in bottom right corner of each image.

Fig. 10(b) and (f) depicts the segmentation results of fuzzy inference system with the intensity feature. From Fig. 10(b) and (f), it could be found that the ship target could be extracted from the sea background. However, there are large amount of dock background or sky background in the segmentation results. On the contrary, the segmentation results of the proposed method do not contain any background as depicted in Fig. 10(c) and (g). This means that the intensity feature benefits the segmentation of the target from the sea background but cannot suppress the other background.

Fig. 11(b) and (f) presents the segmentation results of fuzzy inference system with the local spatial feature. From Fig. 11(b), we can found that the segmentation result of fuzzy inference system with the local spatial feature cannot suppress some background. As depicted in Fig. 11(c), the segmentation result of our method is more similar to the ground truth in Fig. 11(d). Compared with the segmentation result of fuzzy inference system with the local spatial feature in Fig. 11(f), the segmentation result of our method in Fig. 11(g) is completer and more similar to the ground truth image

Fig. 12. (a and e) The original images. (b and f) The segmentation results of fuzzy inference system with the global spatial feature. (c and g) The segmentation results of the proposed method. (d and h) The ground truth image.

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142

137

Fig. 13. (a and e) The original images. (b and f) The segmentation results of fuzzy inference system with the intensity feature and the local spatial feature. (c and g) The segmentation results of the proposed method. (d and h) The ground truth image. The labeled regions are enlarged as presented in bottom right corner of each image.

Fig. 14. (a and e) The original images. (b and f) The segmentation results of fuzzy inference system with the intensity feature and the global spatial feature. (c and g) The segmentation results of the proposed method. (d and h) The ground truth image. The labeled regions are enlarged as presented in top right corner of each image.

in Fig. 11(h). These means that the combination of three features could help to produce completer ship target and suppress more background. As shown in Fig. 12(b) and (f), there are some strange shapes of regions in the segmentation results of fuzzy inference system when only using the global spatial feature. And the global spatial feature could suppress the background around the ship target. However, the segmentation results of the proposed method obtain

completer ship target as presented in Fig. 12(c) and (g). This implies that the target cannot be extracted from the background when only the global spatial feature is applied. The segmentation results of fuzzy inference system using the intensity feature and the local spatial feature are presented in Fig. 13(b) and (f). Fig. 13(c) and (g) presents the segmentation results of the proposed method. In Fig. 13(a), part of the infrared ship target has low contrast. And the ship target is sheltered from

Fig. 15. (a and e) The original images. (b and f) The segmentation results of fuzzy inference system with the local and global spatial feature. (c and g) The segmentation results of the proposed method. (d and h) The ground truth image. The labeled regions are enlarged as presented in top left corner of each image.

138

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142

some plants on the shore, which make the segmentation task difficult. From Fig. 13(b) and (c), it is observed that the background cannot be suppressed without the global spatial feature. In Fig. 13(e), the contrast of ship and background is very low. From Fig. 13(g), we can found that the proposed method could suppress more background due to the employment of the global spatial feature. In a word, the global spatial feature could suppress background in the segmentation of the infrared ship images. Fig. 14(b) and (f) presents the segmentation results of fuzzy inference system with the intensity feature and the global spatial feature. Compared with the segmentation results of the proposed method in Fig. 14(c) and (g), it can be found that the contour of the ship target are not extracted well in the segmentation results of fuzzy inference system when using the intensity feature and the global spatial feature due to the lack of the local spatial feature. This means that the local spatial feature helps to determine the contour of the ship target and reduce the effect of noises. The segmentation results of fuzzy inference system using the local spatial feature and the global spatial feature are depicted in Fig. 15(b) and (f). The segmentation results of the proposed method obtain completer ship target than those of fuzzy inference system using the two constructed spatial feature as presented in Fig. 15(c) and (g). This means that the intensity feature helps to preserve the low-intensity pixels in the ship target. In a conclusion, the combination of the intensity feature, the local spatial feature and the global spatial feature could obtain better results. The intensity feature could distinguish the target from the sea background and preserve some low-gray pixels in the ship target. The local spatial feature helps to determine the contour of the target and reduce the effect of noises. The global spatial feature could suppress the background in the infrared images. The above experiments verified that it was effective to combine these three features as the inputs of the fuzzy inference system. 3.3. Comparison results To certify the effectiveness of the proposed method, twelve existed image segmentation methods are employed for comparisons in both qualitative and quantitative ways, including Otsu’s method [10], Entropy based method [11], 2D maximum entropy based method (2D Entropy) [12], mean shift [14], minimum error thresholding (MinError) [13], fuzzy c-means method (FCM) [15], spatial fuzzy c-means method (SFCM) [16], Chan–Vese model [17], level set method (Level Set) [18], the marker based watershed method (Watershed) [19–22], iterative method [4] and Markov Random Filed constraint SFCM method (MRF-SFCM) [23,24]. To be fair for the comparison, the control parameters of each comparison method are chosen by experience and experiments to get the possibly best results in the comparison experiments [4]. For the level set method, the initial contour is valued by the contour of the most salient region BWsaliency in the infrared ship image. The comparison methods are widely used segmentation methods and could produce good results in most instances. However, because there are noises, sea clutters and complex background in the low quality infrared images, segmentation of ship target becomes a challenge. Many methods have been proposed to address the problem of segmentation, such as threshold based methods [10–13], clustering based methods [14–16], active contour based methods [17], and so on. However, most of them do not consider the characteristics of infrared ship images and may not be effective for segmenting infrared ship target. The Level Set is robust to noises and independent to the initial contour. Moreover, the Level Set is effective in segmentation of the images with inhomogeneous intensity. Also, the Watershed uses the makers generated to obtain the boundary between target and background, which could produce good segmentation results. The iterative method and

Fig. 16. Comparison results.

MRF-SFCM are important segmentation methods for infrared ship images, which consider the characteristics of infrared ship images and obtain good results. These four methods could be considered as robust methods. Thus, the comparisons with these twelve representative methods would be meaningful and could better illustrate the effectiveness of the proposed method. 3.3.1. Qualitative evaluation Figs. 16 and 18 show the segmentation results of all the twelve methods and the ground truths. And the enlarged figures of Figs. 16 and 18 are presented in Figs. 17 and 19, respectively.

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142

139

Fig. 17. The enlarged comparison results of Fig. 16.

Fig. 18. Comparison results.

The Otsu’s method is a threshold technique. As presented in the segmentation results of Otsu’s method, lots of sea clutters are incorrectly regarded as the target because of intensity similarity, which has a bad effect in the segmentation results. The second row of Figs. 17 and 19 presents the explicit results. Entropy based method and 2D Entropy are also threshold based segmentation methods. As presented in the second column of Fig. 16, the Entropy based method could get a satisfying result. Also, in the third column of Fig. 18, the 2D Entropy method could segment the target from the background. Nevertheless, as shown in the first and third column in Fig. 16, the Entropy based method and 2D Entropy

cannot segment the target from the complex background. If the background is complex, the optimal threshold may not be found by the Entropy based method and 2D Entropy. The MinError is another threshold based segmentation method. As presented in the second and last column of Fig. 16, it could yield good results when the ship target is not too smaller comparing with the background. However, as presented in the first, second and third column of Fig. 18, the MinError cannot segment the ship target from the background. This is because the MinError is sensitive to the noises and the threshold may not be proper. In a word, although these four methods could

140

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142

The Level Set is independent to the initial contour and produce satisfying result in the last column of Fig. 16. However, if the background is complex and the ship target is small, the Level Set may fail to extract the target as shown in the first and third column of Fig. 16. This is because the shape of ship target and the local inhomogeneity affect the solution of the partial differential equation to some extent. As presented in the second and last column of Fig. 16, the Watershed cannot produce intact ship target in the results due to the inhomogeneity. From the last column of Fig. 16, it can be found that the Watershed cannot segment some detail parts of the ship target, such as masts. This is difficult to construct perfect marker and some of the detail parts may be regarded as the background. Finally, the proposed method, the iterative method and MRFSFCM acquire better segmentation results than other methods. This is because these three methods consider more characteristics of the target in infrared images. The iterative method is effective for segmenting infrared ship target. The iterative method extracts the target by iteratively wiping the background. However, as presented in the fourth column of Fig. 16, part of the target might also be effaced due to low contrast. The MRF-SFCM refines the local spatial constraint by Markov Random Field and gets satisfactory results in most cases. However, it cannot segment the ship target from miscellaneous backgrounds as shown in the last column of Fig. 18. In addition, part of the ship target cannot be extracted by MRF-SFCM as depicted in the last column in Fig. 16. This is because the contour of the ship target is ambiguous in low-contrast infrared images and the suppression of noises works well mainly in the neighbor of the pixels. Thus, if the ship target is large, some pixels pertaining to the ship target may be regarded as background and would not be extracted. The proposed method utilizes the intensity feature, local spatial feature and global spatial feature of the infrared images to signify the segmented target. More specifically, the intensity feature distinguishes the ship target from the sea background and preserves some low-gray pixels in the ship target, which contributes to obtain the intact ship target as presented in the second column of Fig. 17. The local spatial feature could help to determine the contour of the ship target and reduce the effect of noises. Moreover, the global spatial feature could suppress the background in the infrared images. And due to the application of fuzzy inference system, the intensity feature and two spatial features are appropriately integrated to segment the complete target. Besides, the experimental results illustrate that our method could extract intact ship targets from the infrared images. Furthermore, compared all the segmentation results with the ground truths, the segmentation results of the proposed method are the closest to the ground truths in most cases. Overall, the proposed method outperforms the other twelve methods.

Fig. 19. The enlarged comparison results of Fig. 18.

give satisfactory segmentation results between times, the results segmented may be affected by the noises and sea clutters. Mean shift, FCM and SFCM methods are segmentation methods grounded on clustering. They cannot obtain good results in segmenting infrared ship images with low contrast. Since the intensity difference between the target and the background is not obvious, these methods ineluctably lose some parts of the target or may introduce some backgrounds in their segmentation results as shown in Fig. 18. The Chan–Vese model extracts objects through seeking boundaries of homogeneous areas. As presented in Fig. 18, the segmented results of Chan–Vese model are greatly polluted by sea clutters because of their local inhomogeneity.

3.3.2. Quantitative evaluation Quantitative evaluations are implemented to further validate the effectiveness of the proposed method, which are effective ways to measure the performance of segmentation methods. The common used quantitative evaluations include the misclassification error (ME) [4,60] and relative foreground area error (RAE) [4,61]. ME, representing the percentage of the foreground wrongly classified as background and the background wrongly classified as foreground, is formulated as [60] ME = 1 −

|B0 ∩ BT | + |F0 ∩ FT | , |B0 | + |F0 |

(19)

where BT and FT are the background and the ship target in segmented image, respectively; B0 and F0 signify the background and ship target in the ground truth provided by Liu et al. [4], respectively; | • | signifies the cardinality of a set. ME is ranging from

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142 Table 2 Quantitative comparison results. Methods Otsu Entropy 2D Entropy MinError Mean shift FCM SFCM Chan–Vese Level Set Watershed Iterative method MRF-SFCM Proposed method

ME

RAE

0.3629 0.3054 0.2294 0.4878 0.4373 0.5951 0.5025 0.4888 0.2268 0.2821 0.0789 0.0227 0.0183

0.8737 0.7428 0.6741 0.7068 0.8080 0.9273 0.9416 0.9575 0.7643 0.6886 0.6093 0.4180 0.3734

The smallest values are shown in bold.

0 to 1. Smaller value indicates better performance of the segmentation method. Another quantitative evaluation is RAE, which represents the area error of ship target relative to the ground truth provided by Liu et al. [4]. RAE is defined as [61]

RAE =

⎧ A − AT ⎪ , AT < A0 ⎨ 0 A0

⎪ ⎩ AT − A0 , else

(20)

,

AT

where AT denotes the area of ship target in the segmented result; A0 represents the area of ship target in the ground truth. The range of RAE is [0, 1]. Smaller value implies better result. We conduct the quantitative evaluation using the ground truths provided by Liu et al. [4]. And the comparison results are presented in Table 2. In Table 2, it is perceived that our method obtains smaller average values for ME and RAE than the other twelve methods, which means that the proposed method could segment most of the target from the complex background with lower misclassification error and lower relative foreground area error. That is to say, our method could acquire completer and more accurate segmentation results than the other twelve methods. The average running times of all methods is presented in Table 3. The Otsu’s method, Entropy, 2D Entropy, MinError and mean shift are non-iteration methods. And these methods do not cost a lot of time for calculations. Level Set uses lattice Boltzmann method to work out level set equation and consumes less time. Watershed employs a queue of pixels to speed up the flooding procedure of water in an image [62], so it would not be too time-consuming. The maximum iteration number in the iterative method is 6, so it would not cost lots of time. MRF-SFCM only copes with the possible target region, which makes it faster than FCM, SFCM and Chan–Vese. In Table 3, our method is faster than the Chan–Vese method and is comparable with FCM and SFCM, Table 3 The comparison results of running times(s). Methods

Average time(s)

Otsu Entropy 2D Entropy MinError Mean shift FCM SFCM Chan–Vese Level Set Watershed Iterative method MRF-SFCM Proposed method

0.0597 0.1034 0.1690 0.3457 0.0640 4.7717 4.8331 14.3774 1.1652 0.1117 0.7362 1.2034 5.8187

141

which means that the computation complexity of the proposed method is lower than that of the Chan–Vese. Consequently, the proposed method is comparable with the other twelve methods in segmenting low-contrast infrared images, which verifies the effectiveness of the proposed method. 4. Conclusions This paper proposes an infrared ship image segmentation method utilizing the feature based fuzzy inference system. To extract the target from infrared image with low-contrast, three features are constructed, including intensity feature, local spatial feature and global spatial feature. Then, utilizing the prior knowledge, the fuzzy inference system combine the three features to acquire the final target degree map. Finally, the target could be extracted from the output of fuzzy inference system. Experimental results demonstrate that the proposed method could acquire a intact target from infrared ship image with low-contrast, and has superiority over some other existed methods. Acknowledgements We thank the editor and anonymous reviewers for their valuable comments and suggestions which are favorable in promoting the quality of the paper. Part of this paper has been presented in an international conference [36]. Part of this work has been supported by the National Natural Science Foundation of China (61271023), Program for New Century Excellent Talents in Universities (NCET-13-0020), Fundamental Research Funds for the Central Universities (YWF-15-YHXY-022, YWF-14-YHXY-029, YWF-13-TRSC-028, YWF-16-BJ-Y-28) and State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (BUAA-VR16ZZ-08). References [1] X. Zhao, Z. He, S. Zhang, D. Liang, Robust pedestrian detection in thermal infrared imagery using a shape distribution histogram feature and modified sparse representation classification, Pattern Recognit. 48 (6) (2015) 1947–1960, http://dx.doi.org/10.1016/j.patcog.2014.12.013. [2] S. Zhenfeng, L. Jun, C. Qimin, Fusion of infrared and visible images based on focus measure operators in the curvelet domain, Appl. Opt. 51 (12) (2012) 1910–1921, http://dx.doi.org/10.1364/AO.51.001910. [3] J.W. Davis, V. Sharma, Background-subtraction in thermal imagery using contour saliency, Int. J. Comput. Vis. 71 (2) (2007) 161–181, http://dx.doi.org/ 10.1007/s11263-006-4121-7. [4] Z. Liu, F. Zhou, X. Chen, X. Bai, C. Sun, Iterative infrared ship target segmentation based on multiple features, Pattern Recognit. 47 (9) (2014) 2839–2852, http://dx.doi.org/10.1016/j.patcog.2014.03.005. [5] J. Jakubowicz, S. Lefebvre, F. Maire, E. Moulines, Detecting aircraft with a low-resolution infrared sensor, IEEE Trans. Image Process. 21 (6) (2012) 3034–3041, http://dx.doi.org/10.1109/TIP.2012.2186307. [6] J. Wu, S. Mao, X. Wang, T. Zhang, Ship target detection and tracking in cluttered infrared imagery, Opt. Eng. 50 (5) (2011) 057207, http://dx.doi.org/ 10.1117/1.3578402. [7] X. Bai, F. Zhou, B. Xue, Infrared image enhancement through contrast enhancement by using multiscale new top-hat transform, Infrared Phys. Technol. 54 (2) (2011) 61–69, http://dx.doi.org/10.1016/j.infrared.2010.12. 001. [8] X. Wang, G. Lv, L. Xu, Infrared dim target detection based on visual attention, Infrared Phys. Technol. 55 (6) (2012) 513–521, http://dx.doi.org/10.1016/j. infrared.2012.08.004. [9] W. Zhao, Z. Xu, J. Zhao, F. Zhao, X. Han, Infrared image detail enhancement based on the gradient field specification, Appl. Opt. 53 (19) (2014) 4141–4149, http://dx.doi.org/10.1364/AO.53.004141. [10] N. Otsu, A threshold selection method from gray-level histograms, IEEE Trans. Syst. Man Cybern. 9 (1) (1979) 62–66, http://dx.doi.org/10.1109/TSMC.1979. 4310076. [11] J.N. Kapur, P.K. Sahoo, A.K.C. Wong, A new method for gray-level picture thresholding using the entropy of the histogram, Comput. Vis. Graph. Image Process. 29 (3) (1985) 273–285, http://dx.doi.org/10.1016/0734189X(85)90125-2. [12] F. Du, W. Shi, L. Chen, Y. Dent, Z. Zhu, Infrared image segmentation with 2-D maximum entropy method based on particle swarm optimization (PSO),

142

[13] [14]

[15]

[16]

[17] [18]

[19]

[20]

[21]

[22] [23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31] [32]

[33]

[34]

[35]

[36]

[37]

X. Bai et al. / Applied Soft Computing 46 (2016) 128–142 Pattern Recognit. Lett. 26 (5) (2005) 597–603, http://dx.doi.org/10.1016/j. patrec.2004.11.002. J. Kittler, J. Illingworth, Minimum error thresholding, Pattern Recognit. 19 (1) (1986) 41–47, http://dx.doi.org/10.1016/0031-3203(86)90030-0. D. Comaniciu, P. Meer, Mean shift: a robust approach toward feature space analysis, IEEE Trans. Pattern Anal. Mach. Intell. 24 (5) (2002) 603–619, http:// dx.doi.org/10.1109/34.1000236. S. Araki, H. Nomura, N. Wakami, Segmentation of thermal images using the fuzzy c-means algorithm, in: Proc. IEEE 2nd Int. Conf. Fuzzy Syst., vol. 2, 1993, pp. 719–724, http://dx.doi.org/10.1109/FUZZY.1993.327400. K.S. Chuang, H.L. Tzeng, S. Chen, J. Wu, T.J. Chen, Fuzzy c-means clustering with spatial information for image segmentation, Comput. Med. Imaging Graph. 30 (1) (2006) 9–15, http://dx.doi.org/10.1016/j.compmedimag.2005. 10.001. T. Chan, L. Vese, Active contours without edges, IEEE Trans. Image Process. 10 (2) (2001) 266–277, http://dx.doi.org/10.1109/83.902291. S. Balla-Arabe, X. Gao, B. Wang, A fast and robust level set method for image segmentation using fuzzy clustering and lattice Boltzmann method, IEEE Trans. Cybern. 43 (3) (2013) 910–920, http://dx.doi.org/10.1109/TSMCB.2012. 2218233. K. Sun, Z. Chen, S. Jiang, Local morphology fitting active contour for automatic vascular segmentation, IEEE Trans. Biomed. Eng. 59 (2) (2012) 464–473, http://dx.doi.org/10.1109/TBME.2011.2174362. A. Mouelhi, M. Sayadi, F. Fnaiech, Hybrid segmentation of breast cancer cell images using a new fuzzy active contour model and an enhanced watershed method, in: Int. Conf. Control, Decis. Inf. Technol., CoDIT, 2013, pp. 382–387, http://dx.doi.org/10.1109/CoDIT.2013.6689575. H. Zhou, K.Z. Mao, Adaptive successive erosion-based cell image segmentation for p53 immunohistochemistry in bladder inverted papilloma, in: Proc. IEEE Annu. Conf. Eng. Med. Biol., 2006, pp. 6484–6487, http://dx.doi. org/10.1109/IEMBS.2005.1615984. F. Meyer, Topographic distance and watershed lines, Signal Process. 38 (1) (1994) 113–125, http://dx.doi.org/10.1016/0165-1684(94)90060-4. X. Bai, Z. Chen, Y. Zhang, Z. Liu, Spatial information based fcm for infrared ship target segmentation, in: Proc. IEEE Int. Conf. Image Process., 2014, pp. 5127–5131, http://dx.doi.org/10.1109/ICIP.2014.7026038. X. Bai, Z. Chen, Y. Zhang, Z. Liu, Y. Lu, Infrared ship target segmentation based on spatial information improved FCM, IEEE Trans. Cybern. PP (99) (2015) 1–13, http://dx.doi.org/10.1109/TCYB.2015.2501848. P. Liu, Mamdani fuzzy system: universal approximator to a class of random processes, IEEE Trans. Fuzzy Syst. 10 (6) (2002) 756–766, http://dx.doi.org/10. 1109/TFUZZ.2002.805890. W. Zhao, Q. Niu, K. Li, G.W. Irwin, A hybrid learning method for constructing compact rule-based fuzzy models, IEEE Trans. Cybern. 43 (6) (2013) 1807–1821, http://dx.doi.org/10.1109/TSMCB.2012.2231068. A. Sala, T.M. Guerra, R. Babuˇska, Perspectives of fuzzy systems and control, Fuzzy Sets Syst. 156 (3) (2005) 432–444, http://dx.doi.org/10.1016/j.fss.2005. 05.041. P. Moallem, B.S. Mousavi, S.A. Monadjemi, A novel fuzzy rule base system for pose independent faces detection, Appl. Soft Comput. 11 (2) (2011) 1801–1810, http://dx.doi.org/10.1016/j.asoc.2010.05.024. H. Taghavifar, A. Mardani, Fuzzy logic system based prediction effort: a case study on the effects of tire parameters on contact area and contact pressure, Appl. Soft Comput. 14 (2014) 390–396, http://dx.doi.org/10.1016/j.asoc.2013. 10.005. D. Wang, X. Zeng, J.A. Keane, An evolving-construction scheme for fuzzy systems, IEEE Trans. Fuzzy Syst. 18 (4) (2010) 755–770, http://dx.doi.org/10. 1109/TFUZZ.2010.2047949. X. Zeng, M.G. Singh, Approximation theory of fuzzy systems – SISO case, IEEE Trans. Fuzzy Syst. 2 (2) (1994) 162–176, http://dx.doi.org/10.1109/91.277964. X. Zeng, M.G. Singh, Approximation theory of fuzzy systems – MIMO case, IEEE Trans. Fuzzy Syst. 3 (2) (1995) 219–235, http://dx.doi.org/10.1109/91. 388175. M.R. Emami, B. Turksen, A.A. Goldenberg, Development of a systematic methodology of fuzzy modeling, IEEE Trans. Fuzzy Syst. 6 (3) (1998) 346–361, http://dx.doi.org/10.1109/91.705501. W. Pedrycz, Logic-based fuzzy neurocomputing with unineurons, IEEE Trans. Fuzzy Syst. 14 (6) (2006) 860–873, http://dx.doi.org/10.1109/TFUZZ.2006. 879977. S. Zhou, J.Q. Gan, Low-level interpretability and high-level interpretability: a unified view of data-driven interpretable fuzzy system modelling, Fuzzy Sets Syst. 159 (23) (2008) 3091–3131, http://dx.doi.org/10.1016/j.fss.2008.05. 016. T. Wang, X. Bai, Y. Zhang, Multiple features based low-contrast infrared ship image segmentation using fuzzy inference system, in: Proc. Int. Conf. Digital Image Computing: Techniques and Applications, 2014, pp. 1–6, http://dx.doi. org/10.1109/DICTA.2014.7008117. P.L. Rosin, Unimodal thresholding, Pattern Recognit. 34 (11) (2001) 2083–2096, http://dx.doi.org/10.1016/S0031-3203(00)00136-9.

[38] L. Itti, C. Koch, E. Niebur, A model of saliency based visual attention for rapid scene analysis, IEEE Trans. Pattern Anal. Mach. Intell. 20 (11) (1998) 1254–1259, http://dx.doi.org/10.1109/34.730558. [39] J. Harel, C. Koch, P. Perona, Graph-based visual saliency, Adv, Neural Inf. Process. Syst. (2007) 545–552. [40] S. Goferman, L. Zelnik-Manor, A. Tal, Context-aware saliency detection, IEEE Trans. Pattern Anal. Mach. Intell. 34 (10) (2012) 1915–1926, http://dx.doi.org/ 10.1109/TPAMI.2011.272. [41] S. Lu, C. Tan, J. Lim, Robust and efficient saliency modeling from image co-occurrence histograms, IEEE Trans. Pattern Anal. Mach. Intell. 36 (1) (2014) 195–201, http://dx.doi.org/10.1109/TPAMI.2013.158. [42] R. Adams, L. Bischof, Seeded region growing, IEEE Trans. Pattern Anal. Mach. Intell. 16 (6) (1994) 641–647, http://dx.doi.org/10.1109/34.295913. [43] X. Bai, F. Zhou, B. Xue, Image enhancement using multi scale image features extracted by top-hat transform, Opt. Laser Technol. 44 (2) (2012) 328–336, http://dx.doi.org/10.1016/j.optlastec.2011.07.009. [44] X. Bai, F. Zhou, B. Xue, Edge preserved image fusion based on multiscale toggle contrast operator, Image Vis. Comput. 29 (12) (2011) 829–839, http:// dx.doi.org/10.1016/j.imavis.2011.09.003. [45] X. Bai, F. Zhou, Infrared small target enhancement and detection based on modified top-hat transformations, Comput. Electr. Eng. 36 (6) (2010) 1193–1201, http://dx.doi.org/10.1016/j.compeleceng.2010.05.008. [46] T. Kubota, A.K. Jerebko, M. Dewan, M. Salganicoff, A. Krishnan, Segmentation of pulmonary nodules of various densities with morphological approaches and convexity models, Med. Image Anal. 15 (1) (2011) 133–154, http://dx.doi. org/10.1016/j.media.2010.08.005. [47] S.D. Cataldo, E. Ficarra, A. Acquaviva, E. Macii, Achieving the way for automated segmentation of nuclei in cancer tissue images through morphology-based approach: a quantitative evaluation, Comput. Med. Imaging Graph. 34 (6) (2010) 453–461, http://dx.doi.org/10.1016/j. compmedimag.2009.12.008. [48] C. Park, J.Z. Huang, J.X. Ji, Y. Ding, Segmentation, inference and classification of partially overlapping nanoparticles, IEEE Trans. Pattern Anal. Mach. Intell. 35 (3) (2013) 669–681, http://dx.doi.org/10.1109/TPAMI.2012.163. [49] Y. Tarabalka, J. Chanussot, J.A. Benediktsson, Segmentation and classification of hyperspectral images using watershed transformation, Pattern Recognit. 43 (7) (2010) 2367–2379, http://dx.doi.org/10.1016/j.patcog.2010.01.016. [50] J. Chen, C. Su, W.E.L. Grimson, J. Liu, D. Shiue, Object segmentation of database images by dual multiscale morphological reconstructions and retrieval applications, IEEE Trans. Image Process. 21 (2) (2012) 828–843, http://dx.doi. org/10.1109/TIP.2011.2166558. [51] C.R.M.R. Qi Jr., V. Raghavan, A linear time algorithm for computing exact Euclidean distance transforms of binary images in arbitrary dimensions, IEEE Trans. Pattern Anal. Mach. Intell. 25 (2) (2003) 265–270, http://dx.doi.org/10. 1109/TPAMI.2003.1177156. [52] F.-C. Shih, O.R. Mitchell, A mathematical morphology approach to Euclidean distance transformation, IEEE Trans. Image Process. 1 (2) (1992) 197–204, http://dx.doi.org/10.1109/83.136596. [53] C. Li, R. Huang, Z. Ding, J.C. Gatenby, D.N. Metaxas, J.C. Gore, A level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI, IEEE Trans. Image Process. 20 (7) (2011) 2007–2016, http://dx.doi.org/10.1109/TIP.2011.2146190. [54] E.H. Mamdani, Application of fuzzy algorithms for control of simple dynamic plant, Proc. Inst. Elect. Eng. 121 (12) (1974) 1585–1588, http://dx.doi.org/10. 1049/piee.1974.0328. [55] E.H. Mamdani, S. Assilian, An experiment in linguistic synthesis with a fuzzy logic controller, Int. J. Man Mach. Stud. 7 (1) (1975) 1–13, http://dx.doi.org/10. 1016/S0020-7373(75)80002-2. [56] E.H. Mamdani, Application of fuzzy logic to approximate reasoning using linguistic systems, IEEE Trans. Comput. C-26 (12) (1977) 1182–1191, http:// dx.doi.org/10.1109/TC.1977.1674779. [57] T.J. Ross, Fuzzy Logic with Engineering Applications, third ed., Wiley, 2010, http://dx.doi.org/10.1002/9781119994374. [58] L.A. Zadeh, Fuzzy sets, Inf. Control 8 (3) (1965) 338–353, http://dx.doi.org/10. 1016/S0019-9958(65)90241-X. [59] L.A. Zadeh, Outline of a new approach to the analysis of complex systems and decision processes, IEEE Trans. Syst. Man Cybern. SMC-3 (1) (1973) 24–28, http://dx.doi.org/10.1109/TSMC.1973.5408575. [60] W. Yasnoff, J. Mui, J. Bacus, Error measures for scene segmentation, Pattern Recognit. 9 (4) (1977) 217–231, http://dx.doi.org/10.1016/00313203(77)90006-1. [61] Z. Fan, C. Wang, X. Ma, Double-threshold image segmentation method based on gray gradient, in: Proc. SPIE Int. Conf. Opt. Instrum. Technol., vol. 7506, 2009, p. 75062O, http://dx.doi.org/10.1117/12.839424. [62] L. Vincent, P. Soille, Watersheds in digital spaces: an efficient algorithm based on immersion simulations, IEEE Trans. Pattern Anal. Mach. Intell. (6) (1991) 583–598, http://dx.doi.org/10.1109/34.87344.