- Email: [email protected]
Design of digital camouflage by recursive overlapping of pattern templates
Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
Contents lists available at ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
Design of digital camouflage by recursive overlapping of pattern templates Feng Xue n, Shan Xu, Yue-Tong Luo, Wei Jia School of Computer Science and Information Hefei University of Technology, Hefei, China
art ic l e i nf o
a b s t r a c t
Article history: Received 1 December 2013 Received in revised form 8 October 2014 Accepted 16 December 2014
Using high-tech means of detection, it is easy to discover the drawbacks of traditional camouflage; that is, a lack of sharp boundaries and poor camouflage effects. Digital camouflage textures overcome these drawbacks and produce good optical camouflage effects. In this paper, we present a novel design of digital camouflage based on algorithms for spot pattern template distribution. First, we extract primary colors from the background using a clustering method. Then, a spot template distribution algorithm is proposed to generate camouflage textures gradually. The output camouflage patterns generated by our proposed method blend naturally into the background and achieve good results in terms of optical camouflage. & 2015 Elsevier B.V. All rights reserved.
Keywords: Camouflage k-means Pattern template Saliency
1. Introduction Camouflage is the most common and effective means of dealing with military reconnaissance [1], since the use thereof can conceal military equipment in the natural background. Traditional camouflage relies mainly on the designer's experience and consists of irregular spots and stripes. However, because of the clear boundaries between different colors, traditional camouflage patterns show great visual discrimination and the optical camouflage thereof is weak, with the result that the “hidden target” can be easily found by the enemy's reconnaissance. Conversely, digital camouflage, which is based on visual psychological principles [2], uses computer techniques to extract the background information and forms camouflage patterns using various combinations of spots. It overcomes the disadvantages of traditional camouflage and can more easily blend into the natural background. Digital camouflage with good optical camouflage effects can greatly improve military strength in warfare, and has recently become a hot research topic [3,4]. In this paper, we propose a new digital camouflage pattern design method. First, a clustering method is employed to extract primary colors from the background image. These primary colors (typically 4 or 5 colors chosen according to general military standards) consist of color elements in camouflage textures.
n
Corresponding author. E-mail addresses: [email protected] (F. Xue), [email protected] (S. Xu), [email protected] (Y.-T. Luo), [email protected] (W. Jia).
Practically, while spraying the camouflage patterns onto trucks, tanks, or other equipment, various sheet templates with different layouts of square holes are used to improve spraying efficiency. We call these sheets pattern templates. To make the design process of camouflage patterns consistent with the process of spraying, we propose a new method that generates camouflage textures by iteratively overlapping pattern templates. Generating camouflage textures using pattern templates promotes production efficiency owing to the reuse of representative sheet templates. We summarize camouflage spot distribution rules as constraints according to camouflage industry standards. Then, the output textures with high camouflage effects with respect to the background are obtained by iteratively overlapping these pattern templates under the spot distribution constraints.
2. Overview of proposed algorithm Generally, high quality digital camouflage that blends well into the background must satisfy the following two constraints: (1) Color constraints: Primary colors of the digital camouflage texture should combine well with the background and be hard to identify by human eyes or optical instruments. In other words, (a) there should be some primary colors with differing brightness so that spots in primary colors can easily break up the original form of the camouflaged target, and (b) the differences between the primary color values and those of the background should not be too large. Normally, the
http://dx.doi.org/10.1016/j.neucom.2014.12.108 0925-2312/& 2015 Elsevier B.V. All rights reserved.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
2
brightness of the primary colors should be slightly lower than that of the background. (2) Spot shape constraints: The shapes of the spots should be similar to the patterns in the background. Specifically, spot distributions should hide the original form of the camouflaged target, and spots should be unevenly distributed so that the camouflaged target is hard to detect. In particular, large rectangular blocks or long straight lines should be avoided. Furthermore, the distribution of both large and small spots should be multilayered. In this paper, we consider the above two constraints and propose a novel digital camouflage design algorithm. Because we use two criteria as design constraints, the camouflage textures output by our method embody the essence of good optical camouflage effects. The flowchart of our algorithm is shown in Fig. 1. As shown in Fig. 1, first, we extract primary colors from the background using a clustering algorithm, such as k-means, and then, we use these colors as the basis of the primary color. Meanwhile, primary color constraints (1a) and (1b) are considered in the primary color clustering phase. Then, some spot pattern templates with different colors are used as a “jigsaw”, and these jigsaw templates are recursively overlapped according to the spot shape constraints. Finally, natural camouflage textures are generated that satisfy both the color and spot shape constraints.
since it more closely mimics the human vision system. Before generating the digital camouflage, we convert the background color space from an RGB space to an HSV one. 3.2. Extraction of primary colors using k-means clustering Clustering is the most common method for extracting primary colors from a background image [9]. Owing to its simplicity and rapid execution, we use the k-means algorithm to cluster primary colors in the background image. In the k-means clustering algorithm, if the dataset is well clustered, the clustering center does not change and the algorithm converges. The primary color extraction process using k-means is illustrated in Fig. 2. 3.2.1. Initialization First, we need to initialize cluster centroid C and primary color N. Normally, C is set in the range [8,10], and N is set to 4 or 5.
Start
Initilize cluster centroid number C and primary color number N
K-means clustering algorithm
3. Extraction of primary colors 3.1. Color space conversion
Extract C centroid colors
Red–green–blue (RGB), hue–saturation–value (HSV), and Lab color spaces are often used in image processing [5,6]. In this paper, we use the HSV model for camouflage color clustering. The RGB model uses three basic color components, R, G, and B, to represent the physical features of colors, but it fails to reproduce the essence of the human vision system. The HSV model [7,8] however, is more similar to the human vision mechanism, with H, S, and V denoting the hue, saturation, and value, respectively. Because the process of extracting primary colors from the background is the primary task in camouflage texture design, we chose the HSV color model,
Do centroid colors match industry standard colors
N
Y Use the matched industry standard colors as the primary colors
Use the first N colors as the primary main colors
Fig. 2. Extraction process of primary colors.
Background image
K-means clustering
Building pattern template
N color classes, primary colors candidates
Camouflage pattern template database
Primary colors constraints
Spots shape constraints
Industry standard color database Primary colors of camouflage
Iterative spots template distribution using greedy algorithm
Digital camouflage Textures Fig. 1. Outline of proposed digital camouflage design algorithm.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
3.2.2. Representative background color clustering We use the k-means clustering algorithm to cluster the colors in the background image. Pixels in the background image are classified into categories according to their distance from the clustering centroids. Then, the most representative color in each pixel category is selected as the representative color of each category. We define the brightness contrast between color i and color j as: vdði; jÞ ¼
j vðiÞ vðjÞj maxðvðiÞ; vðjÞÞ
As explained in Section 2, there are two color constraints for camouflage design.
3
4. Camouflage design based on pattern template distribution Most of the current camouflage design methods use image segmentation technology to extract background texture features, which are used to design the camouflage spots [10,11]. During the actual process of painting the camouflage patterns onto clothes, trucks, and other equipment, pattern templates are used as painting cover to avoid deformation or chaos in the final camouflage pattern, and to achieve good camouflage effects. In terms of the actual painting techniques, pattern templates not only improve painting efficiency when spraying a number of spots onto the surface of the equipment, but also make the digital spots more inconspicuous while satisfying the pattern constraints. 4.1. Pattern template database
(1) There should be at least two primary colors with sufficient brightness contrast. vdði; jÞ 4 d1 (2) The gap between the brightness of the primary colors and that of the background should not be too large. The best case is where the brightness of the primary colors is slightly lower than that of the background. vdði; bgcolor Þ o d2
Once color clustering has been completed while satisfying the two color constraints formulated above, we have C color centroids. To obtain N representative colors from the C centroids, we sort the color centroids according to their pixel proportions for the entire background in descending order, and select the top N colors as the representative colors of the background. The proportion of pixels of a color class c in the background image is computed as: pc ¼
#fij i A cg ; hnw
where pc is the proportion of pixels of class c in the background, i is a sample of class c, and h, w denote the height and width of the background, respectively.
3.2.3. Extracting primary colors by matching standard colors Because different backgrounds introduce different representative colors, these will lead to superfluous painting of variable colors in the actual painting phase. Depending on the geographical environment of the background, digital camouflage is usually categorized into four types, namely, woodland, desert, ocean, and city camouflage. Based on vast data collections and actual production experience, the standard primary colors and their proportions in each type of digital camouflage have been defined. In this study, after the representative colors of the background have been determined, we select the type of camouflage pattern whose standard colors most closely match the representative colors of the background, and use these standard colors as the primary colors for designing the target camouflage. We use the Euclidean distance to calculate the distance between the representative background colors and the standard colors of a certain camouflage type. The distance of one color pair d or, s4 is computed as: qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi d o r; s 4 ¼ ðhr hs Þ2 þ ðsr ss Þ2 þ ðvr vs Þ2
Starting from some well-shaped existing camouflage designs, we summarize some typical sub-shapes of the camouflage patterns, and then use a greedy algorithm recursively to spread these pattern templates within the spot shape constraints, until the whole surface has been fully covered. Depending on the spot size (which is usually directly proportional to the number of spots), pattern templates are classified into two kinds, namely, A (large) and B (small). Different kinds of templates have different uses in the spreading phase. Template A is used to form big spots and hide the original form of the target, while template B is used to decorate big spots to improve the hierarchy of the camouflage pattern. Fig. 3 shows some typical pattern templates for the different sizes. Templates A1–A12 are big spot templates, while B1–B8 are small spot templates. 4.2. Pattern distribution using a greedy algorithm A greedy algorithm is a fast simple method for solving optimization problems [12,13], where the solver searches processes step-by-step and uses local optimization as the solver taking account of the boundary conditions in the current step and ignoring any potential global possibilities. In one way, the distribution of camouflage spots can be regarded as an optimization problem within the boundary conditions of the shape constraints. Since the shape constraints are in some cases weak constraints, we use a greedy algorithm to complete the spot distribution. This speeds up the optimization process. 4.2.1. Constraints of template distribution Camouflage patterns with good optical camouflage effects not only require that the colors blend well into the background, but also demand that the shapes of the spots sprayed onto the surface of the camouflage target have good optical camouflage effects. To achieve these goals, the following constraints are imposed based on industry standards and experience: (1) Camouflage patterns should segment the target shape well so that the overall shape is broken into fragmented medium or small spots. On each surface, there should be at least two big spots with sufficient brightness contrast. (2) Spots in the camouflage texture should appear irregular, without large blocks or long stripes. (3) There should be some smaller decorated spots around a main spot, while various decorated spots in different colors should be placed within a large main spot so that the pattern appears more hierarchical and irregular. This causes the texture pattern to be more harmonious and create a better camouflage effect.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
4
A1
A2
A3
A4
A5
A6
A7
A8
A9
A1 0
A11
A1 2
B1
B2
B3
B4
B5
B6
B7
B8
Fig. 3. Typical pattern templates.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
To improve production efficiency and reduce production cost, camouflage coatings can be saved by avoiding overlapping of the template as much as possible when designing the camouflage pattern. 4.2.2. Optimization measure for spot template distribution Although the distribution of spots is irregular, constraints on the distribution are analogous to various forces guiding the distribution of the spots. We allocate each point a distribution cost c, so that we can quantitatively measure the possibility that a point may be a template center. The larger the cost is, the more likely the point is to be a center. Generally, the smaller the cost is, the more likely the point is to be a center where the corresponding template is distributed in the next iteration. During the process of digital camouflage generation, each distribution of a new template dynamically changes the cost of pixels in the coverage area of the template for the next template selection. We use the template distribution cost p (the cost of a pixel to be the center to distribute the corresponding pattern template) as the optimization measure in the following iteration process of the greedy algorithm. We need to set each pixel a distribution cost (usually initialized to 0), which denotes the cost of being covered by a pattern template. It is obvious that large overlapping areas will cause spot collisions, which will break the overall texture pattern. Therefore, each distribution of a template increases the pixel cost of all pixels covered by the template. In our current implementation, we simply update cost c by adding one when the pixel is covered once by a template. The template distribution cost Pt(x) represents the total cost of distributing template t in the rectangular area centered on pixel x (that is, the center of the template is at pixel x). The template distribution cost Pt is the result of convolution using the template matrix as the convolution kernel. Using template A1 as an example, Fig. 4 shows the matrix of template A1 (cells with a value of 0 depict the blank area in the template while those cells with a value of one depict the coverage area). The value of PA1(x) is equal to the convolution of matrix A1 and matrix Cx (matrix Cx is formed from the pixel costs C of the pixels in the rectangular area centered on pixel x). The template distribution cost Pt(x) is used as the optimization measure in the greedy algorithm for distribution of the spot templates. 4.2.3. Detailed camouflage design process The detailed algorithm for the camouflage design process illustrated in Fig. 5 is explained below.
5
0
0
0
0
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
1
1
0
0
1
1
1
1
0
0
0
1
1
1
1
1
1
1
1
1
0
0
0
1
1
1
1
1
1
1
1
1
0
0
0
1
1
1
1
0
1
1
1
0
0
0
1
1
1
1
0
0
0
1
0
0
1
1
1
1
1
1
1
0
0
0
0
1
1
1
0
0
1
1
0
0
0
0
1
1
0
0
1
1
1
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Fig. 4. Cost matrix of template A1.
Initilization of template distribution cost matrix M
Spray the canvas using bottom color BC
Sketch up the outline of camouflage
Distribute big spot templates using greedy agorithm
N
Max connected doamin area < T
Y Small pattern template decoration
Digital camouflage output Fig. 5. Camouflage design using pattern template and greedy algorithm.
(1) Initialization and drawing of background. Create matrix MC, which is used to store the costs, with size equal to that of the camouflage image. Initialize the output camouflage texture with the color with the greatest proportion of pixels in the clustered primary colors. (2) Outline of the output camouflage. Set up a coordinate on the camouflage canvas with the origin as the center of the output camouflage image. Then, draw a cosine curve on the image as an outline of the distribution of spots on the canvas. On the curve, randomly sample N1 points internally as the centers of the big spot pattern templates. At each center point, select a big spot pattern template in one of the primary colors with which to spray spots onto the image (each square hole is one pixel of the output image). After spraying, update the value of C for those pixels covered by the templates in matrix MC. (3) Main spot distribution based on greedy algorithm. The greedy algorithm is used for distributing the main spot
templates. In each iteration, the optimal distribution is calculated based on the template distribution cost Pi(x). The distribution sought is the one with the smallest total cost. Thus, in each step, pixel x with the smallest Pi(x) is selected as the center of a template to minimize the cost. The optimum of each step causes the overall distribution to be close to the optimal distribution. The detailed algorithm processes are as follows: (a) Select a color from the primary colors for the current pattern template. (b) Randomly select the main spot template t, and retrieve template matrix Mt. (c) Greedy choice: find the largest connected domain D in the digital camouflage pattern output in the previous step. Then, the convolution operation using matrix Mt is performed on each pixel in D, and pixel x with the smallest convolution sum Pt(x) is selected.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
6
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
(d) If the color c of the current template is the same as the color of domain D, select another primary color for the current template. (e) Spray the colors onto the camouflage canvas through the holes of current template t using the selected color. (f) Repeat steps (a)– (b) until the largest connected domain area is smaller than a given threshold.
(4) Distribution of decorated spots. After the main spots have been distributed, we need to scatter some smaller decorated spots around the main spots to make the camouflage pattern appear more natural. First, find the largest connected domain, and calculate its length–width ratio r. If r is within range o1/d, d 4, use the centroid as the decorated template center; otherwise, use a pixel around the domain as the decorated template center. Decorated spots are often sprayed in a different colors to that of the connected domain. In our algorithm, iteration of the distribution of decorated spots is limited to a fixed number N in the range [3,5].
5. Results and discussion
Fig. 6. Experimental results of proposed camouflage design method.
Some experimental results are shown in Fig. 6. In this experiment, two camouflage textures were generated from 26 pattern templates, sprayed in four primary colors. The output camouflage textures in Fig. 6 have an overall optical camouflage effect, where big spots decorated by smaller spots look very natural. More detailed experimental results are presented in Fig. 7, where (a) shows the original background images, with red
Fig. 7. More detailed experimental results. (a) Original images. (b) Primary colors of background extracted by k-means clustering. (c) Output camouflage textures. (d) Camouflaged images using textures in (c). (For interpretation of the references to color in this figure, the reader is referred to the web version of this article.)
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
7
Fig. 8. Evaluation of the effect of our camouflage image using saliency maps. (For interpretation of the references to color in this figure, the reader is referred to the web version of this article.)
rectangles indicating the targets to be camouflaged, (b) shows the primary colors extracted from the background, and (c) shows the camouflage texture patches designed by the proposed algorithm. As can be seen, the output camouflage textures satisfy the irregular spot shape requirement since (1) there are no large blocks of spots or long stripes across the whole image, and (2) some big spots with large brightness contrast are scattered around. In Fig. 7(d), the targets in (a) are camouflaged using the texture patches in (c). As can be seen, the camouflage textures designed by our algorithm match the background well. In human visual perception theory [14–17], human attention usually focuses on an interesting area based on visual information search, selection, and transfer. This is a rational and effective approach that evaluates camouflage performance by identifying a camouflaged target against the background based on the human visual attention mechanism. In this paper, we use a saliency map of the camouflaged target as the quantitative evaluation of the performance of our camouflage textures. The higher the salience value is, the more conspicuous is the foreground target, and the weaker is the camouflage effect. We present five groups of
experimental results in Fig. 8, where comparisons of two saliency maps of natural uncamouflaged images and camouflaged-target images are illustrated to evaluate camouflage effectiveness quantitatively. In Fig. 8, the first column shows natural images with various foreground targets highlighted with red rectangles, the second column shows the camouflage textures from our proposed method, and the third column shows the results of manually camouflaging the targets using the texture patches in the second column. To evaluate the camouflage performance of our output camouflage textures quantitatively, two saliency maps of the original images (in the first column) and the camouflaged images (in the third column) are presented in the fourth and fifth columns, respectively. Compared with the original image, the intensity and area of the “white regions” in the camouflaged images are both remarkably reduced, which indicates that the camouflaged targets are well hidden against the backgrounds according to the human vision system. In fact, the salience value is a good quantitative measure of camouflage performance.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
8
Table 1 Time taken to find targets in scenes. Scene Detection time for natural images (s)
Detection time for camouflaged images (s)
(1) (2) (3) (4)
0.84 1.22 0.28 0.19
0.28 0.31 0.1 0.1
Besides the adoption of saliency maps, we designed another quantitative evaluation of camouflage performance. Human–computer combined methods are widely used in various fields [18]. To human observers, the time to detect a camouflaged object against a mixed background can reflect the “hidden quality” of the camouflage design. The more conspicuous the target is, the shorter is the time an observer needs to find it. In our experiment, all the observed images were resized to a uniform size of 200 pixels 200 pixels. For each scene, five groups of observers (each consisting of 10 observers) were tasked with detecting the camouflaged targets hidden against the background. We recorded each observer's time for detecting the targets and used the average time as the detection time. Statistical data of the detection times are given in Table 1. As shown in the table, the detection times for the camouflaged targets are on average three to eight times longer than those of the natural images, given that the observers were informed that camouflaged targets were hidden in the observed images. The detection time comparison can to some extent quantitatively measure the “hidden quality” of our output camouflage textures.
6. Conclusions Vision research is developing rapidly and a number of new research directions [19–21] have appeared; however, the design of camouflage textures remains a very important research area. In this paper, a novel method was proposed to design camouflage textures. Target detection probability was used as a basic measure for evaluating camouflage effect [22], which is associated with the visibility of the target. However, visibility of the target is related to the brightness contrast between the target and the background [23]. By controlling the brightness contrast in the primary color clustering process, the probability of finding the target covered with camouflage textures generated by our method decreased markedly. Good digital camouflage patterns should satisfy two constraints: color and shape constraints. Regarding the color constraints, we extracted primary colors directly from the background, which allowed the output camouflage textures to blend naturally into the background. To satisfy the shape constraints, we used different sized pattern templates, which were iteratively overlapped by a greedy algorithm using template distribution cost as a guiding measure. This guarantees pattern irregularity with no large patches or long stripes observed in the output camouflage.
Acknowledgments This research is supported by the National Natural Science Foundation of China (Grant nos. 61202283 and 61472115), Chunhua Plan(2013HGCH0019) of Hefei University of Technology. References
[2] L.G. Shapiro, G.C. Stockman, Computer Vision, Prentice-Hall, Englewood Cliffs, NJ, 2001. [3] J. Yu, C. Wang, Z. Hu, Design on digital disruptive pattern for fixed targets, J. Comput. Digit. Eng. 39 (4) (2011) 134–154. [4] Y-P. Yao, Camouflage of Army Equipment, Military Science Press, Beijing, 2006. [5] B. Chanda, Digital Image Processing and Analysis, Prentice-Hall, Englewood Cliffs, NJ, 2007. [6] D. Li, Y. Wang, DUB: Research on segmentation methods of weed and soil background under HSI color model [C], in: Proceedings of the 2nd International Workshop on Knowledge Discovery and Data Mining, IEEE, 2009, pp. 628–631. [7] T. Acharya, Image Processing: Principles and Applications, Tsinghua University Press, Beijing, 2007. [8] R.C. Gonzalez, R.E. Woods, Digital Image Processing, 2nd Edition, Addison Wesley Publishing Company, Massachusetts, 2002. [9] J. Yu, M. Chu, S-H. Tian, Z-Y. Hu, Comparison and analysis for image clustering methods of camouflage design, J. Appl. Sci. 27 (5) (2009) 475–479. [10] N.C. Yeo, K.H. Lee, Y.V. Venkatesh, S-H. Ong, Color image segmentation using the self-organizing map and adaptive resonance theory, Image Vis. Comput. 23 (12) (2005) 1060–1079. [11] T. Konishi, S. Omatu, Y. Suga, Extraction of rice-planted area using a selforganizing feature map, Artif. Life Robot. 11 (2) (2007) 215–218. [12] A.J. Smola, B. Schölkopf, Sparse greedy matrix approximation for machine learning, in: Proceedings of the 17th International Conference on Machine Learning, 2000, pp. 911–918. [13] J.R. Yee, S.C. Liang, A second-order greedy algorithm for interconnecting ethernets, in: Proceedings of the IEEE International Conference on. IEEE, 3, 1991, pp. 1305–1310. [14] L. Itti, C. Koch, E. Niebur, A model of saliency-based visual attention for rapid scene analysis, IEEE Trans. PAMI 20 (11) (1998) 1254–1259. [15] L. Itti, C. Koch, Computational modeling of visual attention, Nat. Rev. Neurosci. 2 (3) (2001) 194–230. [16] P. Federico, K. Philipp, P. Yael, H. Alexander, Saliency Filters: Contrast Based Filtering for Salient Region Detection, in: Proceedings of the CVPR, 2012, pp. 733–740. [17] T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, Learning to detect a salient object, IEEE Trans. PAMI 33 (2) (2011) 353–365. [18] M. Wang, B. Ni, X-S. Hua, T-S. Chua, Assistive tagging: a survey of multimedia tagging with human–computer joint exploration, ACM Comput. Surv. 4 (4) (2012) 25. [19] Z-J. Zha, M. Wang, Y-T. Zheng, Y. Yang, R. Hong, T-S. Chua, Interactive video indexing with statistical active learning, IEEE Trans. Multimed. 14 (1) (2012) 17–27. [20] Z-J. Zha, H. Zhang, M. Wang, H-B. Luan, T-S. Chua, Detecting group activities with multi-camera context, IEEE Trans. Circuits Syst. Video Technol. 23 (5) (2013) 856–869. [21] M. Wang, H. Li, D. Tao, K. Lu, X. Wu, Multimodal graph-based reranking for web image search, IEEE Trans. Image Process. 21 (11) (2012) 4649–4661. [22] B. Peng, J. Yi, Z. Nai-Ren, Fuzzy C-means clustering based digital camouflage pattern design and its evaluation, in: Proceedings of the IEEE 10th International Conference on Signal Processing, 2010, pp. 1017–1020. [23] B-S. Cui, S-Q. Xue, Y-J. Ji, H-Y. Yu, Y. Zhang, Camouflage effectiveness evaluation based on image feature, Infrared Laser Eng. 39 (6) (2010) 1178–1183.
Feng Xue was born in Apr. 1978. He received his Ph.D. degree (June 2006) from the Deptartment of Computer Science of Hefei University of Technology. He is currently working as a faculty of Deptartment of Computer Science in Hefei University of Technology. His current research interests are in digital image processing and computer vision.
Shan Xu was born in Jan. 1990, She is currently studying for the master's degree at Deptartment of Computer Science of Hefei University of Technology. Her research interests are in digital image processing and artificial intelligence. Jing Gu was born in Sept. 1988, He is currently studying for the master's degree at Deptartment of Computer Science of Hefei University of Technology. His research interests are in digital image processing and artificial intelligence.
[1] Y. Xu, Camouflage color selection based on dominant color extraction, OptoElectron. Eng. 34 (1) (2007) 100–103.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎ Yue-Tong Luo was born in March 1978. He received his Ph.D. degree (June 2005) from the Deptartment of Computer Science of Hefei University of Technology. He is currently working as a faulty of the Deptartment of Computer Science of Hefei University of Technology. His current research interests are in digital image processing and scientific visualization.
9 Wei Jia received the M.Sc. degree in computer science from Hefei University of Technology, China, in 2004, and the Ph.D. degree in pattern recognition and intelligence system from University of Science and Technology of China, in 2008. He is currently an associate professor in Hefei Institutes of Physical Science, Chinese Academy of Science. His research interests include biometrics, pattern recognition, and image processing.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
Contents lists available at ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
Design of digital camouflage by recursive overlapping of pattern templates Feng Xue n, Shan Xu, Yue-Tong Luo, Wei Jia School of Computer Science and Information Hefei University of Technology, Hefei, China
art ic l e i nf o
a b s t r a c t
Article history: Received 1 December 2013 Received in revised form 8 October 2014 Accepted 16 December 2014
Using high-tech means of detection, it is easy to discover the drawbacks of traditional camouflage; that is, a lack of sharp boundaries and poor camouflage effects. Digital camouflage textures overcome these drawbacks and produce good optical camouflage effects. In this paper, we present a novel design of digital camouflage based on algorithms for spot pattern template distribution. First, we extract primary colors from the background using a clustering method. Then, a spot template distribution algorithm is proposed to generate camouflage textures gradually. The output camouflage patterns generated by our proposed method blend naturally into the background and achieve good results in terms of optical camouflage. & 2015 Elsevier B.V. All rights reserved.
Keywords: Camouflage k-means Pattern template Saliency
1. Introduction Camouflage is the most common and effective means of dealing with military reconnaissance [1], since the use thereof can conceal military equipment in the natural background. Traditional camouflage relies mainly on the designer's experience and consists of irregular spots and stripes. However, because of the clear boundaries between different colors, traditional camouflage patterns show great visual discrimination and the optical camouflage thereof is weak, with the result that the “hidden target” can be easily found by the enemy's reconnaissance. Conversely, digital camouflage, which is based on visual psychological principles [2], uses computer techniques to extract the background information and forms camouflage patterns using various combinations of spots. It overcomes the disadvantages of traditional camouflage and can more easily blend into the natural background. Digital camouflage with good optical camouflage effects can greatly improve military strength in warfare, and has recently become a hot research topic [3,4]. In this paper, we propose a new digital camouflage pattern design method. First, a clustering method is employed to extract primary colors from the background image. These primary colors (typically 4 or 5 colors chosen according to general military standards) consist of color elements in camouflage textures.
n
Corresponding author. E-mail addresses: [email protected] (F. Xue), [email protected] (S. Xu), [email protected] (Y.-T. Luo), [email protected] (W. Jia).
Practically, while spraying the camouflage patterns onto trucks, tanks, or other equipment, various sheet templates with different layouts of square holes are used to improve spraying efficiency. We call these sheets pattern templates. To make the design process of camouflage patterns consistent with the process of spraying, we propose a new method that generates camouflage textures by iteratively overlapping pattern templates. Generating camouflage textures using pattern templates promotes production efficiency owing to the reuse of representative sheet templates. We summarize camouflage spot distribution rules as constraints according to camouflage industry standards. Then, the output textures with high camouflage effects with respect to the background are obtained by iteratively overlapping these pattern templates under the spot distribution constraints.
2. Overview of proposed algorithm Generally, high quality digital camouflage that blends well into the background must satisfy the following two constraints: (1) Color constraints: Primary colors of the digital camouflage texture should combine well with the background and be hard to identify by human eyes or optical instruments. In other words, (a) there should be some primary colors with differing brightness so that spots in primary colors can easily break up the original form of the camouflaged target, and (b) the differences between the primary color values and those of the background should not be too large. Normally, the
http://dx.doi.org/10.1016/j.neucom.2014.12.108 0925-2312/& 2015 Elsevier B.V. All rights reserved.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
2
brightness of the primary colors should be slightly lower than that of the background. (2) Spot shape constraints: The shapes of the spots should be similar to the patterns in the background. Specifically, spot distributions should hide the original form of the camouflaged target, and spots should be unevenly distributed so that the camouflaged target is hard to detect. In particular, large rectangular blocks or long straight lines should be avoided. Furthermore, the distribution of both large and small spots should be multilayered. In this paper, we consider the above two constraints and propose a novel digital camouflage design algorithm. Because we use two criteria as design constraints, the camouflage textures output by our method embody the essence of good optical camouflage effects. The flowchart of our algorithm is shown in Fig. 1. As shown in Fig. 1, first, we extract primary colors from the background using a clustering algorithm, such as k-means, and then, we use these colors as the basis of the primary color. Meanwhile, primary color constraints (1a) and (1b) are considered in the primary color clustering phase. Then, some spot pattern templates with different colors are used as a “jigsaw”, and these jigsaw templates are recursively overlapped according to the spot shape constraints. Finally, natural camouflage textures are generated that satisfy both the color and spot shape constraints.
since it more closely mimics the human vision system. Before generating the digital camouflage, we convert the background color space from an RGB space to an HSV one. 3.2. Extraction of primary colors using k-means clustering Clustering is the most common method for extracting primary colors from a background image [9]. Owing to its simplicity and rapid execution, we use the k-means algorithm to cluster primary colors in the background image. In the k-means clustering algorithm, if the dataset is well clustered, the clustering center does not change and the algorithm converges. The primary color extraction process using k-means is illustrated in Fig. 2. 3.2.1. Initialization First, we need to initialize cluster centroid C and primary color N. Normally, C is set in the range [8,10], and N is set to 4 or 5.
Start
Initilize cluster centroid number C and primary color number N
K-means clustering algorithm
3. Extraction of primary colors 3.1. Color space conversion
Extract C centroid colors
Red–green–blue (RGB), hue–saturation–value (HSV), and Lab color spaces are often used in image processing [5,6]. In this paper, we use the HSV model for camouflage color clustering. The RGB model uses three basic color components, R, G, and B, to represent the physical features of colors, but it fails to reproduce the essence of the human vision system. The HSV model [7,8] however, is more similar to the human vision mechanism, with H, S, and V denoting the hue, saturation, and value, respectively. Because the process of extracting primary colors from the background is the primary task in camouflage texture design, we chose the HSV color model,
Do centroid colors match industry standard colors
N
Y Use the matched industry standard colors as the primary colors
Use the first N colors as the primary main colors
Fig. 2. Extraction process of primary colors.
Background image
K-means clustering
Building pattern template
N color classes, primary colors candidates
Camouflage pattern template database
Primary colors constraints
Spots shape constraints
Industry standard color database Primary colors of camouflage
Iterative spots template distribution using greedy algorithm
Digital camouflage Textures Fig. 1. Outline of proposed digital camouflage design algorithm.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
3.2.2. Representative background color clustering We use the k-means clustering algorithm to cluster the colors in the background image. Pixels in the background image are classified into categories according to their distance from the clustering centroids. Then, the most representative color in each pixel category is selected as the representative color of each category. We define the brightness contrast between color i and color j as: vdði; jÞ ¼
j vðiÞ vðjÞj maxðvðiÞ; vðjÞÞ
As explained in Section 2, there are two color constraints for camouflage design.
3
4. Camouflage design based on pattern template distribution Most of the current camouflage design methods use image segmentation technology to extract background texture features, which are used to design the camouflage spots [10,11]. During the actual process of painting the camouflage patterns onto clothes, trucks, and other equipment, pattern templates are used as painting cover to avoid deformation or chaos in the final camouflage pattern, and to achieve good camouflage effects. In terms of the actual painting techniques, pattern templates not only improve painting efficiency when spraying a number of spots onto the surface of the equipment, but also make the digital spots more inconspicuous while satisfying the pattern constraints. 4.1. Pattern template database
(1) There should be at least two primary colors with sufficient brightness contrast. vdði; jÞ 4 d1 (2) The gap between the brightness of the primary colors and that of the background should not be too large. The best case is where the brightness of the primary colors is slightly lower than that of the background. vdði; bgcolor Þ o d2
Once color clustering has been completed while satisfying the two color constraints formulated above, we have C color centroids. To obtain N representative colors from the C centroids, we sort the color centroids according to their pixel proportions for the entire background in descending order, and select the top N colors as the representative colors of the background. The proportion of pixels of a color class c in the background image is computed as: pc ¼
#fij i A cg ; hnw
where pc is the proportion of pixels of class c in the background, i is a sample of class c, and h, w denote the height and width of the background, respectively.
3.2.3. Extracting primary colors by matching standard colors Because different backgrounds introduce different representative colors, these will lead to superfluous painting of variable colors in the actual painting phase. Depending on the geographical environment of the background, digital camouflage is usually categorized into four types, namely, woodland, desert, ocean, and city camouflage. Based on vast data collections and actual production experience, the standard primary colors and their proportions in each type of digital camouflage have been defined. In this study, after the representative colors of the background have been determined, we select the type of camouflage pattern whose standard colors most closely match the representative colors of the background, and use these standard colors as the primary colors for designing the target camouflage. We use the Euclidean distance to calculate the distance between the representative background colors and the standard colors of a certain camouflage type. The distance of one color pair d or, s4 is computed as: qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi d o r; s 4 ¼ ðhr hs Þ2 þ ðsr ss Þ2 þ ðvr vs Þ2
Starting from some well-shaped existing camouflage designs, we summarize some typical sub-shapes of the camouflage patterns, and then use a greedy algorithm recursively to spread these pattern templates within the spot shape constraints, until the whole surface has been fully covered. Depending on the spot size (which is usually directly proportional to the number of spots), pattern templates are classified into two kinds, namely, A (large) and B (small). Different kinds of templates have different uses in the spreading phase. Template A is used to form big spots and hide the original form of the target, while template B is used to decorate big spots to improve the hierarchy of the camouflage pattern. Fig. 3 shows some typical pattern templates for the different sizes. Templates A1–A12 are big spot templates, while B1–B8 are small spot templates. 4.2. Pattern distribution using a greedy algorithm A greedy algorithm is a fast simple method for solving optimization problems [12,13], where the solver searches processes step-by-step and uses local optimization as the solver taking account of the boundary conditions in the current step and ignoring any potential global possibilities. In one way, the distribution of camouflage spots can be regarded as an optimization problem within the boundary conditions of the shape constraints. Since the shape constraints are in some cases weak constraints, we use a greedy algorithm to complete the spot distribution. This speeds up the optimization process. 4.2.1. Constraints of template distribution Camouflage patterns with good optical camouflage effects not only require that the colors blend well into the background, but also demand that the shapes of the spots sprayed onto the surface of the camouflage target have good optical camouflage effects. To achieve these goals, the following constraints are imposed based on industry standards and experience: (1) Camouflage patterns should segment the target shape well so that the overall shape is broken into fragmented medium or small spots. On each surface, there should be at least two big spots with sufficient brightness contrast. (2) Spots in the camouflage texture should appear irregular, without large blocks or long stripes. (3) There should be some smaller decorated spots around a main spot, while various decorated spots in different colors should be placed within a large main spot so that the pattern appears more hierarchical and irregular. This causes the texture pattern to be more harmonious and create a better camouflage effect.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
4
A1
A2
A3
A4
A5
A6
A7
A8
A9
A1 0
A11
A1 2
B1
B2
B3
B4
B5
B6
B7
B8
Fig. 3. Typical pattern templates.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
To improve production efficiency and reduce production cost, camouflage coatings can be saved by avoiding overlapping of the template as much as possible when designing the camouflage pattern. 4.2.2. Optimization measure for spot template distribution Although the distribution of spots is irregular, constraints on the distribution are analogous to various forces guiding the distribution of the spots. We allocate each point a distribution cost c, so that we can quantitatively measure the possibility that a point may be a template center. The larger the cost is, the more likely the point is to be a center. Generally, the smaller the cost is, the more likely the point is to be a center where the corresponding template is distributed in the next iteration. During the process of digital camouflage generation, each distribution of a new template dynamically changes the cost of pixels in the coverage area of the template for the next template selection. We use the template distribution cost p (the cost of a pixel to be the center to distribute the corresponding pattern template) as the optimization measure in the following iteration process of the greedy algorithm. We need to set each pixel a distribution cost (usually initialized to 0), which denotes the cost of being covered by a pattern template. It is obvious that large overlapping areas will cause spot collisions, which will break the overall texture pattern. Therefore, each distribution of a template increases the pixel cost of all pixels covered by the template. In our current implementation, we simply update cost c by adding one when the pixel is covered once by a template. The template distribution cost Pt(x) represents the total cost of distributing template t in the rectangular area centered on pixel x (that is, the center of the template is at pixel x). The template distribution cost Pt is the result of convolution using the template matrix as the convolution kernel. Using template A1 as an example, Fig. 4 shows the matrix of template A1 (cells with a value of 0 depict the blank area in the template while those cells with a value of one depict the coverage area). The value of PA1(x) is equal to the convolution of matrix A1 and matrix Cx (matrix Cx is formed from the pixel costs C of the pixels in the rectangular area centered on pixel x). The template distribution cost Pt(x) is used as the optimization measure in the greedy algorithm for distribution of the spot templates. 4.2.3. Detailed camouflage design process The detailed algorithm for the camouflage design process illustrated in Fig. 5 is explained below.
5
0
0
0
0
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
1
1
0
0
1
1
1
1
0
0
0
1
1
1
1
1
1
1
1
1
0
0
0
1
1
1
1
1
1
1
1
1
0
0
0
1
1
1
1
0
1
1
1
0
0
0
1
1
1
1
0
0
0
1
0
0
1
1
1
1
1
1
1
0
0
0
0
1
1
1
0
0
1
1
0
0
0
0
1
1
0
0
1
1
1
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Fig. 4. Cost matrix of template A1.
Initilization of template distribution cost matrix M
Spray the canvas using bottom color BC
Sketch up the outline of camouflage
Distribute big spot templates using greedy agorithm
N
Max connected doamin area < T
Y Small pattern template decoration
Digital camouflage output Fig. 5. Camouflage design using pattern template and greedy algorithm.
(1) Initialization and drawing of background. Create matrix MC, which is used to store the costs, with size equal to that of the camouflage image. Initialize the output camouflage texture with the color with the greatest proportion of pixels in the clustered primary colors. (2) Outline of the output camouflage. Set up a coordinate on the camouflage canvas with the origin as the center of the output camouflage image. Then, draw a cosine curve on the image as an outline of the distribution of spots on the canvas. On the curve, randomly sample N1 points internally as the centers of the big spot pattern templates. At each center point, select a big spot pattern template in one of the primary colors with which to spray spots onto the image (each square hole is one pixel of the output image). After spraying, update the value of C for those pixels covered by the templates in matrix MC. (3) Main spot distribution based on greedy algorithm. The greedy algorithm is used for distributing the main spot
templates. In each iteration, the optimal distribution is calculated based on the template distribution cost Pi(x). The distribution sought is the one with the smallest total cost. Thus, in each step, pixel x with the smallest Pi(x) is selected as the center of a template to minimize the cost. The optimum of each step causes the overall distribution to be close to the optimal distribution. The detailed algorithm processes are as follows: (a) Select a color from the primary colors for the current pattern template. (b) Randomly select the main spot template t, and retrieve template matrix Mt. (c) Greedy choice: find the largest connected domain D in the digital camouflage pattern output in the previous step. Then, the convolution operation using matrix Mt is performed on each pixel in D, and pixel x with the smallest convolution sum Pt(x) is selected.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
6
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
(d) If the color c of the current template is the same as the color of domain D, select another primary color for the current template. (e) Spray the colors onto the camouflage canvas through the holes of current template t using the selected color. (f) Repeat steps (a)– (b) until the largest connected domain area is smaller than a given threshold.
(4) Distribution of decorated spots. After the main spots have been distributed, we need to scatter some smaller decorated spots around the main spots to make the camouflage pattern appear more natural. First, find the largest connected domain, and calculate its length–width ratio r. If r is within range o1/d, d 4, use the centroid as the decorated template center; otherwise, use a pixel around the domain as the decorated template center. Decorated spots are often sprayed in a different colors to that of the connected domain. In our algorithm, iteration of the distribution of decorated spots is limited to a fixed number N in the range [3,5].
5. Results and discussion
Fig. 6. Experimental results of proposed camouflage design method.
Some experimental results are shown in Fig. 6. In this experiment, two camouflage textures were generated from 26 pattern templates, sprayed in four primary colors. The output camouflage textures in Fig. 6 have an overall optical camouflage effect, where big spots decorated by smaller spots look very natural. More detailed experimental results are presented in Fig. 7, where (a) shows the original background images, with red
Fig. 7. More detailed experimental results. (a) Original images. (b) Primary colors of background extracted by k-means clustering. (c) Output camouflage textures. (d) Camouflaged images using textures in (c). (For interpretation of the references to color in this figure, the reader is referred to the web version of this article.)
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
7
Fig. 8. Evaluation of the effect of our camouflage image using saliency maps. (For interpretation of the references to color in this figure, the reader is referred to the web version of this article.)
rectangles indicating the targets to be camouflaged, (b) shows the primary colors extracted from the background, and (c) shows the camouflage texture patches designed by the proposed algorithm. As can be seen, the output camouflage textures satisfy the irregular spot shape requirement since (1) there are no large blocks of spots or long stripes across the whole image, and (2) some big spots with large brightness contrast are scattered around. In Fig. 7(d), the targets in (a) are camouflaged using the texture patches in (c). As can be seen, the camouflage textures designed by our algorithm match the background well. In human visual perception theory [14–17], human attention usually focuses on an interesting area based on visual information search, selection, and transfer. This is a rational and effective approach that evaluates camouflage performance by identifying a camouflaged target against the background based on the human visual attention mechanism. In this paper, we use a saliency map of the camouflaged target as the quantitative evaluation of the performance of our camouflage textures. The higher the salience value is, the more conspicuous is the foreground target, and the weaker is the camouflage effect. We present five groups of
experimental results in Fig. 8, where comparisons of two saliency maps of natural uncamouflaged images and camouflaged-target images are illustrated to evaluate camouflage effectiveness quantitatively. In Fig. 8, the first column shows natural images with various foreground targets highlighted with red rectangles, the second column shows the camouflage textures from our proposed method, and the third column shows the results of manually camouflaging the targets using the texture patches in the second column. To evaluate the camouflage performance of our output camouflage textures quantitatively, two saliency maps of the original images (in the first column) and the camouflaged images (in the third column) are presented in the fourth and fifth columns, respectively. Compared with the original image, the intensity and area of the “white regions” in the camouflaged images are both remarkably reduced, which indicates that the camouflaged targets are well hidden against the backgrounds according to the human vision system. In fact, the salience value is a good quantitative measure of camouflage performance.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎
8
Table 1 Time taken to find targets in scenes. Scene Detection time for natural images (s)
Detection time for camouflaged images (s)
(1) (2) (3) (4)
0.84 1.22 0.28 0.19
0.28 0.31 0.1 0.1
Besides the adoption of saliency maps, we designed another quantitative evaluation of camouflage performance. Human–computer combined methods are widely used in various fields [18]. To human observers, the time to detect a camouflaged object against a mixed background can reflect the “hidden quality” of the camouflage design. The more conspicuous the target is, the shorter is the time an observer needs to find it. In our experiment, all the observed images were resized to a uniform size of 200 pixels 200 pixels. For each scene, five groups of observers (each consisting of 10 observers) were tasked with detecting the camouflaged targets hidden against the background. We recorded each observer's time for detecting the targets and used the average time as the detection time. Statistical data of the detection times are given in Table 1. As shown in the table, the detection times for the camouflaged targets are on average three to eight times longer than those of the natural images, given that the observers were informed that camouflaged targets were hidden in the observed images. The detection time comparison can to some extent quantitatively measure the “hidden quality” of our output camouflage textures.
6. Conclusions Vision research is developing rapidly and a number of new research directions [19–21] have appeared; however, the design of camouflage textures remains a very important research area. In this paper, a novel method was proposed to design camouflage textures. Target detection probability was used as a basic measure for evaluating camouflage effect [22], which is associated with the visibility of the target. However, visibility of the target is related to the brightness contrast between the target and the background [23]. By controlling the brightness contrast in the primary color clustering process, the probability of finding the target covered with camouflage textures generated by our method decreased markedly. Good digital camouflage patterns should satisfy two constraints: color and shape constraints. Regarding the color constraints, we extracted primary colors directly from the background, which allowed the output camouflage textures to blend naturally into the background. To satisfy the shape constraints, we used different sized pattern templates, which were iteratively overlapped by a greedy algorithm using template distribution cost as a guiding measure. This guarantees pattern irregularity with no large patches or long stripes observed in the output camouflage.
Acknowledgments This research is supported by the National Natural Science Foundation of China (Grant nos. 61202283 and 61472115), Chunhua Plan(2013HGCH0019) of Hefei University of Technology. References
[2] L.G. Shapiro, G.C. Stockman, Computer Vision, Prentice-Hall, Englewood Cliffs, NJ, 2001. [3] J. Yu, C. Wang, Z. Hu, Design on digital disruptive pattern for fixed targets, J. Comput. Digit. Eng. 39 (4) (2011) 134–154. [4] Y-P. Yao, Camouflage of Army Equipment, Military Science Press, Beijing, 2006. [5] B. Chanda, Digital Image Processing and Analysis, Prentice-Hall, Englewood Cliffs, NJ, 2007. [6] D. Li, Y. Wang, DUB: Research on segmentation methods of weed and soil background under HSI color model [C], in: Proceedings of the 2nd International Workshop on Knowledge Discovery and Data Mining, IEEE, 2009, pp. 628–631. [7] T. Acharya, Image Processing: Principles and Applications, Tsinghua University Press, Beijing, 2007. [8] R.C. Gonzalez, R.E. Woods, Digital Image Processing, 2nd Edition, Addison Wesley Publishing Company, Massachusetts, 2002. [9] J. Yu, M. Chu, S-H. Tian, Z-Y. Hu, Comparison and analysis for image clustering methods of camouflage design, J. Appl. Sci. 27 (5) (2009) 475–479. [10] N.C. Yeo, K.H. Lee, Y.V. Venkatesh, S-H. Ong, Color image segmentation using the self-organizing map and adaptive resonance theory, Image Vis. Comput. 23 (12) (2005) 1060–1079. [11] T. Konishi, S. Omatu, Y. Suga, Extraction of rice-planted area using a selforganizing feature map, Artif. Life Robot. 11 (2) (2007) 215–218. [12] A.J. Smola, B. Schölkopf, Sparse greedy matrix approximation for machine learning, in: Proceedings of the 17th International Conference on Machine Learning, 2000, pp. 911–918. [13] J.R. Yee, S.C. Liang, A second-order greedy algorithm for interconnecting ethernets, in: Proceedings of the IEEE International Conference on. IEEE, 3, 1991, pp. 1305–1310. [14] L. Itti, C. Koch, E. Niebur, A model of saliency-based visual attention for rapid scene analysis, IEEE Trans. PAMI 20 (11) (1998) 1254–1259. [15] L. Itti, C. Koch, Computational modeling of visual attention, Nat. Rev. Neurosci. 2 (3) (2001) 194–230. [16] P. Federico, K. Philipp, P. Yael, H. Alexander, Saliency Filters: Contrast Based Filtering for Salient Region Detection, in: Proceedings of the CVPR, 2012, pp. 733–740. [17] T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, Learning to detect a salient object, IEEE Trans. PAMI 33 (2) (2011) 353–365. [18] M. Wang, B. Ni, X-S. Hua, T-S. Chua, Assistive tagging: a survey of multimedia tagging with human–computer joint exploration, ACM Comput. Surv. 4 (4) (2012) 25. [19] Z-J. Zha, M. Wang, Y-T. Zheng, Y. Yang, R. Hong, T-S. Chua, Interactive video indexing with statistical active learning, IEEE Trans. Multimed. 14 (1) (2012) 17–27. [20] Z-J. Zha, H. Zhang, M. Wang, H-B. Luan, T-S. Chua, Detecting group activities with multi-camera context, IEEE Trans. Circuits Syst. Video Technol. 23 (5) (2013) 856–869. [21] M. Wang, H. Li, D. Tao, K. Lu, X. Wu, Multimodal graph-based reranking for web image search, IEEE Trans. Image Process. 21 (11) (2012) 4649–4661. [22] B. Peng, J. Yi, Z. Nai-Ren, Fuzzy C-means clustering based digital camouflage pattern design and its evaluation, in: Proceedings of the IEEE 10th International Conference on Signal Processing, 2010, pp. 1017–1020. [23] B-S. Cui, S-Q. Xue, Y-J. Ji, H-Y. Yu, Y. Zhang, Camouflage effectiveness evaluation based on image feature, Infrared Laser Eng. 39 (6) (2010) 1178–1183.
Feng Xue was born in Apr. 1978. He received his Ph.D. degree (June 2006) from the Deptartment of Computer Science of Hefei University of Technology. He is currently working as a faculty of Deptartment of Computer Science in Hefei University of Technology. His current research interests are in digital image processing and computer vision.
Shan Xu was born in Jan. 1990, She is currently studying for the master's degree at Deptartment of Computer Science of Hefei University of Technology. Her research interests are in digital image processing and artificial intelligence. Jing Gu was born in Sept. 1988, He is currently studying for the master's degree at Deptartment of Computer Science of Hefei University of Technology. His research interests are in digital image processing and artificial intelligence.
[1] Y. Xu, Camouflage color selection based on dominant color extraction, OptoElectron. Eng. 34 (1) (2007) 100–103.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i
F. Xue et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎ Yue-Tong Luo was born in March 1978. He received his Ph.D. degree (June 2005) from the Deptartment of Computer Science of Hefei University of Technology. He is currently working as a faulty of the Deptartment of Computer Science of Hefei University of Technology. His current research interests are in digital image processing and scientific visualization.
9 Wei Jia received the M.Sc. degree in computer science from Hefei University of Technology, China, in 2004, and the Ph.D. degree in pattern recognition and intelligence system from University of Science and Technology of China, in 2008. He is currently an associate professor in Hefei Institutes of Physical Science, Chinese Academy of Science. His research interests include biometrics, pattern recognition, and image processing.
Please cite this article as: F. Xue, et al., Design of digital camouflage by recursive overlapping of pattern templates, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2014.12.108i