- Email: [email protected]
Detect image splicing with artificial blurred boundary
Mathematical and Computer Modelling 57 (2013) 2647–2659
Contents lists available at SciVerse ScienceDirect
Mathematical and Computer Modelling journal homepage: www.elsevier.com/locate/mcm
Detect image splicing with artificial blurred boundary Guangjie Liu a,∗ , Junwen Wang b , Shiguo Lian c , Yuewei Dai a a
School of Automation, Nanjing University of Science & Technology, Nanjing, 210094, PR China
b
CETC 38th Institude, Hefei, 230088, PR China
c
Corporate Research, Huawei Technologies, Beijing, 100034, PR China
article
info
Article history: Received 25 January 2011 Received in revised form 27 May 2011 Accepted 11 June 2011 Keywords: Image splicing Blurring Non-subsampled contourlet Phase congruency Image passive authentication
abstract In this paper, a novel method is proposed to detect image splicing with artificial blurred boundary based on image edge analysis and blur detection. Different from existing algorithms, the image edges are divided into three types based on the coefficients of the non-subsampled contourlet transform. And, the six-dimensional feature of each edge point is extracted, which is composed of two non-subsampled contourlet coefficients and four statistics based on the phase congruency. Then, three support vector machines for each edge type are trained and used to detect the blurred edge points. And, the local feature is defined to distinguish artificial blurred edge points from defocus ones. The proposed method can be used to detect either the image blur or the image splicing with artificial blurred boundary, and it is shown by experimental results. © 2011 Elsevier Ltd. All rights reserved.
1. Introduction In the past when film was used, it needed professional skills and some additional conditions such as dark-room equipment, special developer, photographic paper and so on to tamper with a photograph. Nowadays, with the wide applications of digital cameras and powerful digital image processing software, it has become easier and easier to create a tampered photo. For example, in 2007, some photos of South China Tigers misled many people in China to believe the existence of wild South China Tigers, and made the government launch laws and regulations to protect them. Finally, these photos were proved to be ‘‘paper’’ tigers [1]. In 2008, a newspaper Iran published a tampered missiles emission picture in which one of four missiles was found to be copied from another [1]. Image forgery may cause trouble on various occasions. For instance, in a photography contest, some cameramen make forged photos to gain honors, which is against the basic principle of fairness. In news reports, forged pictures will distort the truth and mislead public opinion, and even influence politics. Someone may move one person’s face onto another photo, and put the forged image on Internet, which can destroy an individual’s reputation. A faked image can be used in an academic paper to support some important conclusion. Furthermore, an important object may be wiped off from an evidence image, which will cause a miscarriage of justice. Thus it is important to judge whether an image is trustworthy. It belongs to the field of image authentication to answer whether an image is true or false. In recent years, image authentication, as a new and important direction of multimedia security, has attracted lots of attention. Usually, there are two kinds of methods, the active authentication and passive one. The active methods can further be divided into two categories. The first category is based on digital watermarking that embeds a watermark into an image at the acquiring end, and extracts it at the authentication end to check whether the image is tampered. The second category is based on a digital signature. The signature is generated at the acquiring end, and regenerated at the authentication end to make a comparison and produce the authentication result. The mechanisms of these two kinds of methods can be illustrated through Fig. 1.
∗
Corresponding author. E-mail addresses: [email protected] (G. Liu), [email protected] (S. Lian).
0895-7177/$ – see front matter © 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.mcm.2011.06.026
2648
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
(a) Watermarking-based scheme.
(b) Signature-based scheme. Fig. 1. Image authentication based on digital watermarking or signature.
Fig. 2. Image splicing forgery.
Passive authentication, also called digital forensics on some occasions, is used to authenticate without any additional information. The main issues in passive authentication include source identification [2] and forgery detection [3]. The latter commonly utilizes the distinct properties of natural images [4] or tampering traces [5] to perform authentication. Image splicing is one of the most common manipulations to create forged images. As Fig. 2 shows, the selected part in the source image A is copied, resized and pasted onto the target image B, and the composite image C is obtained. Although experienced experts can identify the non-sophisticated splicing, it is still a challenging issue to perform automatic detection by machines. And more, the sophisticated splicing forgery usually contains some post-processing operations to make the composite image more real. These operations include blurring, desalting, shading, JPEG compression, etc. It will make the forgery detection more difficult. In the paper, a special post-processing operation, blurring spliced boundary, is considered, and the corresponding splicing detection method is presented. The remainder of paper is organized as follows. In Section 2, related work about passive image authentication, existing detection schemes of image splicing and the non-subsampled contourlet transform are introduced. In Section 3, image edges are analyzed and the feature extraction method is given. Section 4 introduces the detection algorithm in detail. Experimental results are given in Section 5 and concluding remarks are presented in Section 6. 2. Related work In recent years, researchers have developed lots of passive techniques for detecting forgeries with various forms. Farid et al. developed several statistical methods for detecting forgeries based on region duplication [6], color filter interpolation [6], re-sampling [6] and lamp direction [7]. Fridrich et al. presented methods for detecting copy–move forgery [8] and camera identification [9] based on the pattern noise of the sensor in the digital cameras. Readers can refer to the summary of digital forensics written by Mahdian and Saic [10] and Wang et al. [11]. In this paper, we pay attention to image splicing detection. 2.1. Methods for detecting image splicing Existing methods for detecting image splicing can be roughly divided into three categories: universal methods, regionbased methods and boundary-based methods.
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
2649
The universal methods take the image splicing detection as the steganalysis problem. Most of them consist of three steps: extracting features, training the classifier, and making the detection. Shi et al. analyzed image splicing in detail [12] and proposed methods based on Hilbert–Huang transform [13] and statistics of 2-D phase congruency [14]. Wang et al. [15] proposed an image splicing detection method based on a gray level co-occurrence matrix. Zhang et al. [16] proposed a detection scheme based on image quality features and moments features extracted from the DCT domain. The region-based methods generally utilize the model of natural images and identify the forgery by the inconsistence of these image characteristics. For images captured by digital cameras, the used models include light [17], optic lens characteristics [18], sensor pattern noise [19], CFA interpolation [20], and recompression artifacts [21]. The key issue of boundary-based methods is to differentiate the abnormal edges from the normal ones. Zhang and Wang [22] proposed a method for detecting image splicing based on a Sobel edge detector, derivative operation and Hough transform. Qu [5] proposed a method based on visual cues to detect the abnormal sharp boundaries. Hsiao and Pei [23] proposed a tampering detection method based on blur estimation in the DCT domain. Zhou et al. [24] proposed a method to detect blurred edges based on mathematical morphology. Some researchers used the wavelet analysis to analyze image edges. Both Lipschitz regularity [25] and the phase congruency approach [14] are used to define the local smoothing level according to the wavelet coefficients. To eliminate visual artifacts on splicing boundary, forgers usually adopt retouching operations to remove unwanted traces caused by splicing. Blurring is just one kind of retouching operation which averages neighboring pixels’ values to achieve a smooth effect. It should be noticed that all the methods mentioned above can only deal with the simple splicing without post-processing or splicing just with global blurring or JPEG compression. Those blur detection methods are also designed to find the global blurring but not the boundary blurring after splicing. When the forger performs artificial boundary blurring after splicing, the existing methods are all invalid. So, in this paper, a new method is proposed to detect image splicing with artificial blurred edges. 2.2. Preliminary on non-subsampled contourlet transform The contourlet transform is a directional multi-scale transform that is constructed by combining a Laplacian pyramid (LP) and a directional filter bank (DFB). Due to down-samplers and up-samplers presented in both LP and DFB, the contourlet transform is not shift-invariant. The non-subsampled contourlet transform (NSCT) [26] is obtained by coupling a non-subsampled pyramid structure with the non-subsampled DFB. The NSCT construction can thus be divided into two parts: (1) a non-subsampled pyramid structure which ensures the multi-scale property, and (2) a non-subsampled DFB structure which gives the directionality. Next, we describe each part in detail. 2.2.1. Pyramid filter bank for NSCT The non-subsampled pyramid filter bank is different from the counterpart of the contourlet transform. The building block of the non-subsampled pyramid is a two-channel non-subsampled filter bank. A non-subsampled filter bank has no down-sampling or up-sampling. Hence it is shift-invariant. The perfect reconstruction condition is given as Eq. (1). H0 (Z )G0 (Z ) + H1 (Z )G1 (Z ) = 1.
(1)
Here, H0 (Z ) is the lowpass decomposition filter, H1 (Z ) the highpass decomposition filter, G0 (Z ) the lowpass reconstruction filter, and G1 (Z ) the highpass reconstruction filter. In order to obtain the multi-scale decomposition, non-subsampled pyramids are constructed by iterated non-subsampled filter banks. For the next level, all filters are up-sampled by 2 in both dimensions. Therefore, they also satisfy the perfect reconstruction identity. Note that the filtering with the up-sampled filter has the same complexity as filtering with using the à trous algorithm. The equivalent filters of a l-th level cascading non-subsampled pyramid are given by the equation below.
eq
Hd ( z ) =
d−2 j 2d−1 H ( z ) H0 (z 2 ) 1 ≤ d < 2l 1 j =0
d−1 2j l H0 (z ) d = 2 .
(2)
j =0
j
j
Here, z j denotes [z1 , z2 ]. 2.2.2. Directional filter banks for NSCT The non-subsampled directional filter bank is a shift-invariant version of the critically sampled DFB in the contourlet transform. The building block of a non-subsampled DFB is also a two-channel non-subsampled filter bank. Non-subsampled directional filter banks are iterated in order to obtain finer directional decomposition. For the next level, all filters up-sampled by a quincunx matrix given by
2650
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
Fig. 3. Non-subsampled contourlet transform.
Q =
1 1
1
−1
.
(3)
Then four-direction frequency divisions are obtained. The high level decompositions follow a similar strategy. 2.2.3. Non-subsampled contourlet transform The non-subsampled contourlet transform is constructed by combining the non-subsampled pyramid filter bank(NSPFB) and the non-subsampled directional filter bank (NSDFB) as shown in Fig. 3. NSPFB provides multi-scale decomposition and NSDFB provides directional decomposition. The scheme can be iterated repeatedly on the lowpass subband outputs of NSPFB. Firstly, NSPFB splits the input image into a lowpass subband and a highpass subband. Then, a NSDFB decomposes the highpass subband into several directional subbands. The scheme is iterated repeatedly on the lowpass subband. Fig. 4 shows the demonstration example by NSCT: (a) shows an original image, and it includes low frequency and high frequency information; (b)–(e) show the results of NSCT in different scales and different frequency divisions. We use decomposition with three levels for NSCT. NSCT is a novel shift-invariant, multi-scale and multi-directional rapid transformation. When an image is decomposed by NSCT, we can get more details of the high-frequency components with the size of all the directions the same as that of the original image. It can ensure the synchronization between the spatial domain and frequency domain. 3. Image edges analysis and features extraction 3.1. Edge types According to the variation mode of pixel values beside edges, the edges can be generally divided into three types: stepstructure, roof-structure and convex circular-structure. A graphical illustration of these three types is given in Fig. 5. The edge sharpness decreases in turn from left to right. Inspired by [27], we can use the coefficients in NSCT domain to judge the edge types for the image edge points. Perform NSCT on the original image with size of m × n. According to our experiments, when the decomposition level is set to 3, and each level has 8 directions, the decomposed component can be used to distinguish the blurred edges and normal edges existing in natural images. So in this paper, we use 3-level decomposition and make the decomposition along 8 directions at each level. Suppose that φtk (i, j) denotes the kth sub-band directional component of the tth level at the point (i, j) with i, j ∈ {1, 2, . . . , m}, {1, 2, . . . , n}. Φt (i, j) is defined as the maximum NSCT coefficient of 8 directional components of the tth level at the position (i, j). Suppose that the image edge point set is E, which contains all the pixels found by an edge detection algorithm, such as the Canny or Sobel edge detector, the image edge point set E can be further divided into three subsets S , M and W according to the process below. Step 1: Set S = φ, M = φ, W = φ ; Step 2: Scan all the points in E. For the edge point e(i, j) ∈ E, if Φ3 (i, j) = maxt ={1,2,3} Φt (i, j), S = S ∪ {e(i, j)}, else if Φ2 (i, j) = maxt ={1,2,3} Φt (i, j), M = M ∪ {e(i, j)}, else W = W ∪ {e(i, j)}. Step 3: After all the pixels in E are scanned, the three subsets S , M and W are obtained, and |S | + |M | + |W | = |E |. The three subsets S , M and W correspond to the step, roof and convex type respectively. In general, most natural images contain all three types. When an image is blurred, whether caused by the defocusing or in an artificial manner, the number of step edge points will decrease. What is more, both step and roof edges tend to lose their sharpness. In other words, the
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
b
a
2651
c
d
e
Fig. 4. NSCT demonstration images (a) Original image; (b) Low frequency image; (c) NSCT coefficients- level 1 with 2 directions; (d) NSCT coefficientslevel 2 with 4 directions; (e) NSCT coefficients- level 3 with 8 directions.
(a) Step.
(b) Roof.
(c) Convex. Fig. 5. Edge types.
cardinality of the set S will become smaller, while the case is contrary for that of the set W . Taking the gray-value image Lena with size 256 × 256 as example, adopting Canny edge detector, we can draw the cardinalities of the three sets for the original image, the blurred image I (Gaussian blurring n1 = n2 = 5, σ 2 = 1) and the blurred image II (Gaussian blurring n1 = n2 = 5, σ 2 = 2) shown as Fig. 6. The example above indicates the effect of blurring on the image edges. 3.2. Feature extraction When images are blurred with image processing software, such as, Photoshop, users may be asked to choose the blurring mode. Since some blurring modes support user-defined filters, people can manipulate images with diverse blurring radiuses and intensity. According to the algorithm realized in Photoshop, the blurring is motion averaging filtering of chosen image regions to smooth the image. Different modes involve different motion averaging filtering functions. The width of filter window determines the blur radius, and filter parameters reflect the blurring intensity. In other words, blurring is the operation of replacing the original pixel point values with the neighborhood average values.
2652
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
Fig. 6. The cardinalities of three edge sets before and after blurring.
(a) Set W .
(b) Set M.
(c) Set S. Fig. 7. NCST coefficients before and after Gaussian blurring.
3.2.1. Blur features In order to discover the difference between normal edges and blurred edges, we select one hundred images, which include several content types such as landscapes, figures, construction and so on. The sets S , M and W of each image are obtained after the edge detection. 10 elements in each set of each image are randomly selected, the component Φ3 (i, j) of which are shown with black spots in Fig. 7 (a) With the same operation to the 100 blurred images (Gaussian blurring n1 = n2 = 5, σ 2 = 1.5), the component Φ3 (i, j) are shown by green spots in Fig. 7(b) and (c). Regardless of edge types, it can be seen from Fig. 7 that the components Φ3 (i, j) of normal images are larger than those of the blurred images. Therefore we take Φ3 (i, j) as the one-dimensional feature of the edge point. In order to enhance the high frequency components that contain edge information, we propose to predict each pixel value in the original image by using its neighboring pixels’ values, and obtain the prediction-error image by subtracting the predicted image from the original image. It is expected that the process will remove most low frequency information, and make the detection more accurate. The prediction-error image is the difference between the image and its predicted
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
2653
Fig. 8. Prediction context.
version. The prediction algorithm is given by Eq. (4).
xˆ =
max(a, b) if c ≤ min(a, b) min(a, b) if c ≥ max(a, b) a + b − c otherwise.
(4)
Here a, b, c are the context pixels of the pixel x, xˆ is the prediction value of x as illustrated in Fig. 8. When the image is blurred, the prediction error becomes smaller than that of the normal. So we also extract features in the same manner from the prediction error image (1x = |x − xˆ |), Therefore, the component Φ3′ (i, j) which is the NSCT coefficient of the prediction-error image at the position (i, j) that is the position of the edge point in the original image, is also extracted as one of the features. 3.2.2. Phase congruency feature Phase congruency (PC) was first defined by Morrone and Burr [28] in terms of Fourier series expansion of a signal at some location x as
PC2 (x) =
W (x) ⌊An (x)1Ψn (x) − T ⌋
n
An (x) + ε
,
(5)
n
where An (x) is the amplitude of the nth Fourier component, An (x)1Ψn (x) is the local energy function at position x, W (x) is the weight function of the filter at position x, and T is the estimated noise energy. The phase deviation function 1Ψn (x) is defined as follows
1Ψn (x) = cos(Ψn (x) − Ψ¯ n (x)) − sin(Ψn (x) − Ψ¯ n (x)) .
(6)
If PC equals to the maximal value of 1, all frequency components are in phase, otherwise, PC takes on values between 0 and 1. PC of image edges will become smaller after blur operation. When the phase congruency of the prediction error image is obtained, at the corresponding positions of the image edge points, the first four moments, mean (µ), variance (ϖ ), skewness (σ ) and kurtosis (κ) of a 3 × 3 region are obtained as the features. Totally, for an edge point at position (i, j), the six-dimensional feature vector F = [Φ3 , Φ3′ , µ, ϖ , σ , κ] is obtained. We can perform the blur detection by F to distinguish the blurred edges from normal edges. Then the artificial blurred edges are differentiated from the defocused blurred edges by the local definition. The detailed detection algorithm is described in the next section. 4. Detection algorithm 4.1. Classifier training In this paper, Support Vector Machines (SVM) are adopted to distinguish blurred edges from normal ones. The SVM classifier is very popular and has drawn lots of attentions in pattern recognition and machine learning field. It does not depend explicitly on the dimensionality of the problem thus the solution is always globally optimal. At the training stage, given a set of training sample–label pairs (⃗ vi , yi ), i = 1, 2, . . . , L where v⃗i is the feature vector of the ith training sample and yi ∈ {1, −1} is the corresponding class label, SVM solves the following optimization problem.
min
w, ⃗ b,αi
1 2
w ⃗Tw + C
f
αi
(7)
i =1
s.t. yi (w ⃗ T φ(⃗vi ) + b) ≥ 1 − αi ,
αi ≥ 0.
Here, C is the penalty parameter of the error term, and the function φ maps the training feature vector v ⃗i into a higher dimensional space. The kernel function K (⃗ vi , v⃗j ) = φ(⃗vi )T φ(⃗vj ) is used to avoid the explicit computation of the function φ . In our experiment, we use the radial basis function (RBF) as the kernel function, which is given in Eq. (8). K (⃗ vi , v⃗j ) = exp(−γ ∥⃗vi − v⃗j ∥2 ),
γ > 0.
(8)
2654
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
Fig. 9. Detection process.
Here, γ is the kernel parameter. The penalty parameter C and kernel parameter γ must be carefully chosen so that the SVM can give accurate classification results. The ‘‘grid search’’ method with cross-validation is employed to find the optimal C and γ . During the training process, for each edge type, the corresponding classifiers SVM1, SVM2, SVM3 are trained for the edge set S , M and W respectively. 100 images are selected including gray-scale images and color images. Training data 1: Perform NSCT on each image and its prediction-error image. For each image, edge detection is adopted. Therefore, the edge sets S , M and W of each image can be obtained. 10 elements of each set in each image are selected randomly. The six-dimensional feature vector F = [Φ3 , Φ3′ , µ, ϖ , σ , κ] is extracted for those edge points. So we can obtain 1000 six-dimensional feature samples for each edge type. Training data 2: Perform NSCT on each blurred image (50 images Gaussian blur n1 = n2 = 5, σ 2 = 1, 50 images Gaussian blur n1 = n2 = 5, σ 2 = 2) and its prediction-error image. For each image, the edge sets S , M and W of each image can be obtained by the edge detection algorithm. 10 elements of each set in each image are selected randomly. The same features are extracted from the blurred images. The 1000 six-dimensional feature samples for each edge type of blurred images are obtained. Using the above datum, the corresponding SVM1, SVM2, and SVM3 can be trained. 4.2. Local definition It is important to tell the differences between the defocus blur and the artificial one to judge whether there exists the forgery. As we know, the artificial blurring is the result of filtering the edge region with the blur radius decided manually, so pixels outside the blur radius are not processed usually. But the camera defocus blurring does not have a fixed blur radius. It is the most important difference which can be exploited. In this paper we use the local definition G defined by Eq. (9) to detect the artificial blurring operation. G(i, j) =
1
θ
[θ/ 2]
[θ/ 2]
2
((∂ f (i, j)/∂ x)2 + (∂ f (i, j)/∂ y)2 )/2.
(9)
i=−[θ/2] j=−[θ/2]
Here, ∂ f (i, j)/∂ x and ∂ f (i, j)/∂ y denote the differential along x and y direction respectively at the edge point position (i, j). θ , a non-negative integer, is the width of the window, whose value indicates the size of the partial region with the center edge point position (i, j). During the experiment, we find that the values of G(i, j) for the artificial blurred edge points are larger than those of the defocused blurred edge point. So the local definition can be used to judge whether the edge is artificially blurred. While, the question is how to select the threshold, θ and Gth . The larger θ is, the higher the computational complexity is. And when the value of θ is smaller, the result of the local definition cannot tell the difference between the artificial blurring and the defocus blurring. According to the common practices of image splicing forgery, forgers often blur the splicing edges with the radius equaling to 2 or 3 pixels, because a large radius would bring about obvious blurring traces. So in this paper, the width of local definition window θ is chosen as 8, which means that the radius equals to 4 and is a little larger than the common used size. In order to determine the threshold of local definition, we perform an additional experiments on lots of artificial and defocus blurred images, and find when the threshold of Gth is chosen equal to 9.7, the two kinds of blurring can be distinguished well. 4.3. Detection process According to the above analysis and discussion, the whole detection process can be described by Fig. 9. The detailed steps are introduced below. Step 1: Perform the edge detection on a given image I. The edge set E is obtained. Step 2: Perform NSCT on the image and its prediction-error image with the decomposition level equal to 3 and the number of the decomposition directions equal to 8. According to Section 3.1, the edge sets S , M and W can be obtained.
1
1
0.9
0.9
0.8
0.8
0.7
0.7
True positive probability
True positive probability
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
0.6 0.5 0.4 0.3
0.5 0.4 0.3 0.2
0.1
0.1 0
0.2
0.4 0.6 False positive probability
0.8
0
1
(a) JPEG compression (80) and blurring modes A.
1
1
0.9
0.9
0.8
0.8
0.7
0.7
0.6 0.5 0.4 0.3
1
0.8
1
0.3
0.1 0.4 0.6 False positive probability
(c) JPEG compression (80) and blurring modes B.
0.8
0.4
0.1 0.2
0.4 0.6 False positive probability
0.5
0.2
0
0.2
0.6
0.2
0
0
(b) JPEG compression (60) and blurring modes A.
True positive probability
True positive probability
0.6
0.2
0
2655
0.8
1
0
0
0.2
0.4 0.6 False positive probability
(d) JPEG Compression (60) and blurring modes B. Fig. 10. Comparison results of ROC curves.
Step 3: Initialize an all-zero matrix Ξ with the size equal to that of the given image. Scan all the edge points. For each edge point, extract the six-dimensional feature and use SVM1, SVM2 and SVM3 respectively to find the blurred edge point. If an edge point at the position (i, j) is judged as a blurred one, recode it by setting Ξ (i, j) = 1. Step 4: Initialize an all-zero matrix Θ with the size equal to that of the given image. Scan all the blurred edge points with Ξ (i, j) equal to 1. For a blurred edge point at the position (i, j), calculate the local definition Gij , if Gij > Gth , the edge point is more likely to be an artificial blurred edge point, recode the result by setting Θ (i, j) = 1. Step 5: Initialize an all-zero matrix Λ with the size equal to that of the given image. Scan all the edge points with Θ (i, j) equal to 1. For the possibly artificial blurred edge point at the position (i, j), calculate Π (i, j) by the equation given below.
Π (i, j) =
1 1
Θ (i + p, j + q).
(10)
p=−1 q=−1
If Π (i, j) ≥ 3, the edge point at the position (i, j) is determined as the artificial blurred edge point ultimately, and record the result by setting Λ(i, j) = 1. Finally, we can plot the matrix Λ as a binary image and locate the tampered region.
2656
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
(a) Un-tampered image.
(b) Tampered image.
(c) Detection result. Fig. 11. The detection result of the gray-level image.
5. Experiment results and analysis The proposed algorithm mentioned above is fit for gray-level images. There are two ways to extend it to the color case. One is to convert a color image to a gray-scale one by the formula I = 0.299R + 0.587G + 0.114B, the other is to process each color channel (RGB) to yield three detection results independently and get the final result by performing an ‘‘AND’’ operation and voting mechanism. In our experiments, the first method is adopted. Two experiments are carried out to evaluate the performance of our method. 5.1. Blur detection experiment Although the proposed method is mainly designed to detect image splicing with blurred boundary, it can also be used to detect the image blur itself. The ratio µ defined by |Ξ |/|E | can be used to judge whether an image is blurred by being compared with a threshold. The wavelet-based blur detection method [27] is selected to make the comparison. In [27], the image is decomposed by a 3-level Haar wavelet, the edge points are found by comparing the local maxima in three scales with a threshold, and the Dirac- and Astep-structure edge point is determined by the order relationship of the maxima in three scales. The image is judged as a blurred one if the ratio between the number of all Dirac- and Astep- structure edge points and the number of edge points is less than a given value which is called MinZero in [27]. In the experiment, the threshold to judge an edge point is chosen as 35 as suggested in [27]. Considering that the spliced image is commonly saved in a low-quality JPEG format to further reduce the tampering traces in most forgery processes of image splicing. The experiment is designed to investigate the robustness of our blur detection method against JPEG compression. There are total of 1000 images captured by a Canon A710 camera that are used in the experiment. These images are all clear and un-blurred. Firstly, the clear un-blurred images are blurred by two Guassian blurring modes (mode A : 5 × 5, σ 2 = 1 and mode B : 4 × 4, σ 2 = 1) respectively. Secondly, those blurred images are recompressed with the quality factor equal
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
(a) Un-tampered image.
2657
(b) Tampered image.
(c) Detection result. Fig. 12. The detection result of the color image.
to 80 and 60 respectively. Those clear images are also recompressed with the two quality factors directly. The blurred and recompressed images and only recompressed images are input into the blur detection algorithms of our method and the method in [27]. The MinZero and µ = |Ξ |/|E | are chosen from their minimum value and maximum value to compute the true and false positive probabilities under different threshold for plotting the ROC curves. The results are shown in Fig. 10. From the experimental results, it can be observed that our method is robust against JPEG compression and outperforms the wavelet-based method at all testing conditions. According to our experiment, the detection results of the wavelet-based method are not very reliable under JPEG compression. The main reason is that there are only the Dirac- and Astep- structure edge points that are used to judge the blurring. Meanwhile, it can be found that the JPEG compression also cause a certain effect on the detection accuracy because of the noises introduced by the compression. 5.2. Splicing detection experiment The second experiment is performed to test the validity of our splicing detection method. Two examples are given by Figs. 11 and 12. Fig. 11(b) is a fake image that is generated by copying the truck in the bottom left corner of Fig. 11(a), pasting it onto itself and blurring the spliced edges. Fig. 11(c) is the detection result according to our method. In Fig. 12(b), the head of the person in Fig. 12(a) is replaced. Moreover the edge blurring operation is also carried out. Fig. 12(c) is the detection result. To test the performance of our method, we also perform another experiment. Firstly, 41 color images captured by a DSLR Camera Pentax K100D (the soft focus function of the camera will cause defocus blur) and 48 color images captured by a Canon A710 are chosen as testing images. Secondly, image splicing forgery with artificial edge blurring is performed on every image using Photoshop CS3. The tampering process is composed of copying a region of one image, pasting it onto the target image and adopting the blur tool provided by Photoshop to eliminate visual distortion around the spliced boundary. The blurring radius is chosen as 2 or 3 pixels. After these operations, a total of 89 tampered images are obtained. Furthermore,
2658
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659 Table 1 Splicing detection results. Image source
TP (%)
FP (%)
Pentax K100D Canon A710
95.12 91.67
8.17 6.54
there are another 147 and 153 un-tampered images captured by the two cameras which are selected in the experiment to test the false detection level. The experimental results are listed in Table 1, where TP represents the true positive probability of the splicing detection, FP represents the false positive probability. According to the results, we can see that our method can detect the image splicing with artificial blurred edges well. 6. Conclusions In this paper, the detection problem of image splicing with artificial blurred boundary is studied, and a new detection method is proposed. In our method, NSCT is used to distinguish the three edge types. For an edge point, two NSCT coefficients, four statistical values from the phase congruency are composed as the feature. Three SVMs are used to detect the blurred edge points for various edge types. And a local definition is defined and used to filter the blurred edge points caused by the defocus. Experimental results show that the proposed method can detect the splicing with artificial blurred edges well. It should be mentioned that the proposed method can just be taken as an auxiliary manner to judge whether an image is tampered because there is only the cue of the artificial blurring operation is used, and the accuracy of our method is not high enough to avoid people’s participation. In future work, we will pay more attention to look for more efficient features to improve the detection performance against image splicing with blurred boundaries. And since there are various complex forgery ways of tampering an image, how to find and use those tampering traces is also an open issue to be studied in the future. Acknowledgments This study was supported by the Post Doctor Foundation of China (Grant no. 20070421017), NSF of Jiangsu province (Grant no. BK2008403, BK2010484), NUST Research Funding (Grant no. 2010ZYTS048), and NSF of China (Grant no. 60974129). References [1] S.-G. Lian, Y. Zhang, Multimedia forensics for detecting forgeries, in: Handbook on Communications and Information Security, 2010, pp. 801–820. [2] T.-T. Ng, M. Tsui, Camera response function signature for digital forensics-part I: theory and data selection, in: Proceedings of IEEE Workshop on Information Forensics and Security, 2009, pp. 156–160. [3] G.-J. Liu, J.-W. Wang, S.-G. Lian, Z.-Q. Wang, A passive image authentication scheme for detecting region-duplication forgery with rotation, Journal of Network and Computer Applications (2010) doi:10.1016/j.jnca.2010.09.001. [4] P. Zhang, X.-W. Kong, Detecting image tampering using feature fusion, in: Proceedings of International Conference on Availability, Reliability and Security, 2009, pp. 335–340. [5] Z.-H. Qu, G.-P. Qiu, J.-W. Huang, Detect digital image splicing with visual cues, in: Proceedings of International Workshop on Information Hiding, 2009, pp. 247–261. [6] A.-C. Popescu, H. Farid, Exposing digital forgeries by detecting duplicated image regions, Dartmouth College, USA, 2004. [7] M.-K. Johnson, H. Farid, Exposing digital forgeries by detecting inconsistencies in lighting, Presented at the Proceedings of ACM Multimedia Security Workshop, 2005. [8] J. Fridrich, D. Soukal, J. Lukas, Detection of copy–move forgery in digital images, Presented at the Proceedings of Digital Forensic Research Workshop, 2003. [9] J. Lukas, J. Fridrich, M. Goljan, Digital camera identification from sensor pattern noise, IEEE Transactions on Information Forensics and Security 1 (2006) 205–214. [10] B. Mahdian, S. Saic, A bibliography on blind methods for identifying image forgery, Signal Processing: Image Communication 25 (2010) 389–399. [11] W. Wang, J. Dong, T.-N. Tan, A survey of passive image tampering detection, in: Proceeding of International Workshop on Digital Watermarking, in: LNCS, vol. 5370, 2009, pp. 308–322. [12] Y.-Q. Shi, C.-H. Chen, G.-R. Xuan, W. Su, Steanalysis versus splicing detection, in: Proceeding of International Workshop on Digital Watermarking, in: LNCS, vol. 5041, 2007, pp. 4–13. [13] D.-D. Fu, Y.-Q. Shi, W. Su, Detection of image splicing based on Hilbert–Huang transform and moments of characteristic functions with wavelet decomposition, in: Proceeding of International Workshop on Digital Watermarking, in: LNCS, vol. 4283, 2006, pp. 177–187. [14] W. Chen, Y.-Q. Shi, W. Su, Image splicing detection using 2-D phase congruency and statistical moments of characteristic function, in: Proceedings of SPIE Security and Watermarking of Multimedia Contents IX, vol. 6505, 2007, pp. 6505R.1–6505R.8. [15] W. Wang, J. Dong, T.-N. Tan, Effective image splicing detection based on image chroma, in: Proceedings of International Conference on Image Processing, 2009, pp. 1257–1260. [16] Z. Zhang, J.-Q. Kang, Y. Ren, An effective algorithm of image splicing detection, in: Proceeding of International Conference on Computer Science and Software Engineering, 2008, pp. 1035–1039. [17] M.-K. Johnson, H. Farid, Exposing digital forgeries in complex lighting environments, IEEE Transactions on Information Forensics and Security 2 (2007) 450–461. [18] M.-K. Johnson, H. Farid, Exposing digital forgeries through chromatic aberration, in: Proceeding of ACM Multimedia and Security, 2006, pp. 48–55. [19] M. Chen, J. Fridrich, J. Lukas, M. Goljan, Imaging sensor noise as digital X-ray for revealing forgeries, in: Proceeding of International Workshop on Information Hiding, in: LNCS, vol. 4567, 2008, pp. 342–358.
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
2659
[20] A.-C. Popescu, H. Farid, Exposing digital forgeries in color filter array interpolated images, Signal Processing: Image Communication 53 (2005) 3948–3959. [21] J.-W. Wang, G.-J. Liu, Y.-W. Dai, Z.-Q. Wang, Detecting JPEG image forgery based on double compression, Journal of Systems Engineering and Electronics 20 (2009) 1096–1103. [22] J. Wang, H.-B. Zhang, Exposing digital forgeries by detecting traces of image splicing, in: Proceeding of International Conference on Signal Processing, 2006, pp. 16–20. [23] D.-Y. Hsiao, S.-C. Pei, Detecting digital tampering by blur estimation, in: Proceeding of the First International Workshop on Systematic Approaches to Digital Forensic Engineering, 2005, pp. 264–267. [24] L.-N. Zhou, D.-M. Wang, Y.-B. Guo, J.-F. Zhang, Blur detection of digital forgery using mathematical morphology, in: Proceeding of KES AMSTA, 2007, pp. 990–998. [25] Y. Sutcu, B. Coskun, H.T. Sencar, N. Memon, Tamper detection based on regularity of wavelet transform coefficients, in: Proceeding of International Conference on Image Processing, 2007, pp. 397–400. [26] A.-L. Cunha, J. Zhou, M.-N. Do, Non-subsampled contourlet transform: filter design and applications in denoising, in: Proceeding of International Conference on Image Processing, 2005, pp. 749–752. [27] H.-H. Tong, M.-J.H.-J. Zhang, C.-S. Zhang, Blur detection for digital images using wavelet transform, in: Proceedings of IEEE Conference on Multimedia and Expo., 2004, pp. 17–20. [28] M.-C. Morrone, D.-C. Burr, Feature detection in human vision: a phase-dependent energy model, Proceedings of the Royal Society of London, Series B, Biological Sciences 235 (1988) 221–245.
Contents lists available at SciVerse ScienceDirect
Mathematical and Computer Modelling journal homepage: www.elsevier.com/locate/mcm
Detect image splicing with artificial blurred boundary Guangjie Liu a,∗ , Junwen Wang b , Shiguo Lian c , Yuewei Dai a a
School of Automation, Nanjing University of Science & Technology, Nanjing, 210094, PR China
b
CETC 38th Institude, Hefei, 230088, PR China
c
Corporate Research, Huawei Technologies, Beijing, 100034, PR China
article
info
Article history: Received 25 January 2011 Received in revised form 27 May 2011 Accepted 11 June 2011 Keywords: Image splicing Blurring Non-subsampled contourlet Phase congruency Image passive authentication
abstract In this paper, a novel method is proposed to detect image splicing with artificial blurred boundary based on image edge analysis and blur detection. Different from existing algorithms, the image edges are divided into three types based on the coefficients of the non-subsampled contourlet transform. And, the six-dimensional feature of each edge point is extracted, which is composed of two non-subsampled contourlet coefficients and four statistics based on the phase congruency. Then, three support vector machines for each edge type are trained and used to detect the blurred edge points. And, the local feature is defined to distinguish artificial blurred edge points from defocus ones. The proposed method can be used to detect either the image blur or the image splicing with artificial blurred boundary, and it is shown by experimental results. © 2011 Elsevier Ltd. All rights reserved.
1. Introduction In the past when film was used, it needed professional skills and some additional conditions such as dark-room equipment, special developer, photographic paper and so on to tamper with a photograph. Nowadays, with the wide applications of digital cameras and powerful digital image processing software, it has become easier and easier to create a tampered photo. For example, in 2007, some photos of South China Tigers misled many people in China to believe the existence of wild South China Tigers, and made the government launch laws and regulations to protect them. Finally, these photos were proved to be ‘‘paper’’ tigers [1]. In 2008, a newspaper Iran published a tampered missiles emission picture in which one of four missiles was found to be copied from another [1]. Image forgery may cause trouble on various occasions. For instance, in a photography contest, some cameramen make forged photos to gain honors, which is against the basic principle of fairness. In news reports, forged pictures will distort the truth and mislead public opinion, and even influence politics. Someone may move one person’s face onto another photo, and put the forged image on Internet, which can destroy an individual’s reputation. A faked image can be used in an academic paper to support some important conclusion. Furthermore, an important object may be wiped off from an evidence image, which will cause a miscarriage of justice. Thus it is important to judge whether an image is trustworthy. It belongs to the field of image authentication to answer whether an image is true or false. In recent years, image authentication, as a new and important direction of multimedia security, has attracted lots of attention. Usually, there are two kinds of methods, the active authentication and passive one. The active methods can further be divided into two categories. The first category is based on digital watermarking that embeds a watermark into an image at the acquiring end, and extracts it at the authentication end to check whether the image is tampered. The second category is based on a digital signature. The signature is generated at the acquiring end, and regenerated at the authentication end to make a comparison and produce the authentication result. The mechanisms of these two kinds of methods can be illustrated through Fig. 1.
∗
Corresponding author. E-mail addresses: [email protected] (G. Liu), [email protected] (S. Lian).
0895-7177/$ – see front matter © 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.mcm.2011.06.026
2648
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
(a) Watermarking-based scheme.
(b) Signature-based scheme. Fig. 1. Image authentication based on digital watermarking or signature.
Fig. 2. Image splicing forgery.
Passive authentication, also called digital forensics on some occasions, is used to authenticate without any additional information. The main issues in passive authentication include source identification [2] and forgery detection [3]. The latter commonly utilizes the distinct properties of natural images [4] or tampering traces [5] to perform authentication. Image splicing is one of the most common manipulations to create forged images. As Fig. 2 shows, the selected part in the source image A is copied, resized and pasted onto the target image B, and the composite image C is obtained. Although experienced experts can identify the non-sophisticated splicing, it is still a challenging issue to perform automatic detection by machines. And more, the sophisticated splicing forgery usually contains some post-processing operations to make the composite image more real. These operations include blurring, desalting, shading, JPEG compression, etc. It will make the forgery detection more difficult. In the paper, a special post-processing operation, blurring spliced boundary, is considered, and the corresponding splicing detection method is presented. The remainder of paper is organized as follows. In Section 2, related work about passive image authentication, existing detection schemes of image splicing and the non-subsampled contourlet transform are introduced. In Section 3, image edges are analyzed and the feature extraction method is given. Section 4 introduces the detection algorithm in detail. Experimental results are given in Section 5 and concluding remarks are presented in Section 6. 2. Related work In recent years, researchers have developed lots of passive techniques for detecting forgeries with various forms. Farid et al. developed several statistical methods for detecting forgeries based on region duplication [6], color filter interpolation [6], re-sampling [6] and lamp direction [7]. Fridrich et al. presented methods for detecting copy–move forgery [8] and camera identification [9] based on the pattern noise of the sensor in the digital cameras. Readers can refer to the summary of digital forensics written by Mahdian and Saic [10] and Wang et al. [11]. In this paper, we pay attention to image splicing detection. 2.1. Methods for detecting image splicing Existing methods for detecting image splicing can be roughly divided into three categories: universal methods, regionbased methods and boundary-based methods.
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
2649
The universal methods take the image splicing detection as the steganalysis problem. Most of them consist of three steps: extracting features, training the classifier, and making the detection. Shi et al. analyzed image splicing in detail [12] and proposed methods based on Hilbert–Huang transform [13] and statistics of 2-D phase congruency [14]. Wang et al. [15] proposed an image splicing detection method based on a gray level co-occurrence matrix. Zhang et al. [16] proposed a detection scheme based on image quality features and moments features extracted from the DCT domain. The region-based methods generally utilize the model of natural images and identify the forgery by the inconsistence of these image characteristics. For images captured by digital cameras, the used models include light [17], optic lens characteristics [18], sensor pattern noise [19], CFA interpolation [20], and recompression artifacts [21]. The key issue of boundary-based methods is to differentiate the abnormal edges from the normal ones. Zhang and Wang [22] proposed a method for detecting image splicing based on a Sobel edge detector, derivative operation and Hough transform. Qu [5] proposed a method based on visual cues to detect the abnormal sharp boundaries. Hsiao and Pei [23] proposed a tampering detection method based on blur estimation in the DCT domain. Zhou et al. [24] proposed a method to detect blurred edges based on mathematical morphology. Some researchers used the wavelet analysis to analyze image edges. Both Lipschitz regularity [25] and the phase congruency approach [14] are used to define the local smoothing level according to the wavelet coefficients. To eliminate visual artifacts on splicing boundary, forgers usually adopt retouching operations to remove unwanted traces caused by splicing. Blurring is just one kind of retouching operation which averages neighboring pixels’ values to achieve a smooth effect. It should be noticed that all the methods mentioned above can only deal with the simple splicing without post-processing or splicing just with global blurring or JPEG compression. Those blur detection methods are also designed to find the global blurring but not the boundary blurring after splicing. When the forger performs artificial boundary blurring after splicing, the existing methods are all invalid. So, in this paper, a new method is proposed to detect image splicing with artificial blurred edges. 2.2. Preliminary on non-subsampled contourlet transform The contourlet transform is a directional multi-scale transform that is constructed by combining a Laplacian pyramid (LP) and a directional filter bank (DFB). Due to down-samplers and up-samplers presented in both LP and DFB, the contourlet transform is not shift-invariant. The non-subsampled contourlet transform (NSCT) [26] is obtained by coupling a non-subsampled pyramid structure with the non-subsampled DFB. The NSCT construction can thus be divided into two parts: (1) a non-subsampled pyramid structure which ensures the multi-scale property, and (2) a non-subsampled DFB structure which gives the directionality. Next, we describe each part in detail. 2.2.1. Pyramid filter bank for NSCT The non-subsampled pyramid filter bank is different from the counterpart of the contourlet transform. The building block of the non-subsampled pyramid is a two-channel non-subsampled filter bank. A non-subsampled filter bank has no down-sampling or up-sampling. Hence it is shift-invariant. The perfect reconstruction condition is given as Eq. (1). H0 (Z )G0 (Z ) + H1 (Z )G1 (Z ) = 1.
(1)
Here, H0 (Z ) is the lowpass decomposition filter, H1 (Z ) the highpass decomposition filter, G0 (Z ) the lowpass reconstruction filter, and G1 (Z ) the highpass reconstruction filter. In order to obtain the multi-scale decomposition, non-subsampled pyramids are constructed by iterated non-subsampled filter banks. For the next level, all filters are up-sampled by 2 in both dimensions. Therefore, they also satisfy the perfect reconstruction identity. Note that the filtering with the up-sampled filter has the same complexity as filtering with using the à trous algorithm. The equivalent filters of a l-th level cascading non-subsampled pyramid are given by the equation below.
eq
Hd ( z ) =
d−2 j 2d−1 H ( z ) H0 (z 2 ) 1 ≤ d < 2l 1 j =0
d−1 2j l H0 (z ) d = 2 .
(2)
j =0
j
j
Here, z j denotes [z1 , z2 ]. 2.2.2. Directional filter banks for NSCT The non-subsampled directional filter bank is a shift-invariant version of the critically sampled DFB in the contourlet transform. The building block of a non-subsampled DFB is also a two-channel non-subsampled filter bank. Non-subsampled directional filter banks are iterated in order to obtain finer directional decomposition. For the next level, all filters up-sampled by a quincunx matrix given by
2650
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
Fig. 3. Non-subsampled contourlet transform.
Q =
1 1
1
−1
.
(3)
Then four-direction frequency divisions are obtained. The high level decompositions follow a similar strategy. 2.2.3. Non-subsampled contourlet transform The non-subsampled contourlet transform is constructed by combining the non-subsampled pyramid filter bank(NSPFB) and the non-subsampled directional filter bank (NSDFB) as shown in Fig. 3. NSPFB provides multi-scale decomposition and NSDFB provides directional decomposition. The scheme can be iterated repeatedly on the lowpass subband outputs of NSPFB. Firstly, NSPFB splits the input image into a lowpass subband and a highpass subband. Then, a NSDFB decomposes the highpass subband into several directional subbands. The scheme is iterated repeatedly on the lowpass subband. Fig. 4 shows the demonstration example by NSCT: (a) shows an original image, and it includes low frequency and high frequency information; (b)–(e) show the results of NSCT in different scales and different frequency divisions. We use decomposition with three levels for NSCT. NSCT is a novel shift-invariant, multi-scale and multi-directional rapid transformation. When an image is decomposed by NSCT, we can get more details of the high-frequency components with the size of all the directions the same as that of the original image. It can ensure the synchronization between the spatial domain and frequency domain. 3. Image edges analysis and features extraction 3.1. Edge types According to the variation mode of pixel values beside edges, the edges can be generally divided into three types: stepstructure, roof-structure and convex circular-structure. A graphical illustration of these three types is given in Fig. 5. The edge sharpness decreases in turn from left to right. Inspired by [27], we can use the coefficients in NSCT domain to judge the edge types for the image edge points. Perform NSCT on the original image with size of m × n. According to our experiments, when the decomposition level is set to 3, and each level has 8 directions, the decomposed component can be used to distinguish the blurred edges and normal edges existing in natural images. So in this paper, we use 3-level decomposition and make the decomposition along 8 directions at each level. Suppose that φtk (i, j) denotes the kth sub-band directional component of the tth level at the point (i, j) with i, j ∈ {1, 2, . . . , m}, {1, 2, . . . , n}. Φt (i, j) is defined as the maximum NSCT coefficient of 8 directional components of the tth level at the position (i, j). Suppose that the image edge point set is E, which contains all the pixels found by an edge detection algorithm, such as the Canny or Sobel edge detector, the image edge point set E can be further divided into three subsets S , M and W according to the process below. Step 1: Set S = φ, M = φ, W = φ ; Step 2: Scan all the points in E. For the edge point e(i, j) ∈ E, if Φ3 (i, j) = maxt ={1,2,3} Φt (i, j), S = S ∪ {e(i, j)}, else if Φ2 (i, j) = maxt ={1,2,3} Φt (i, j), M = M ∪ {e(i, j)}, else W = W ∪ {e(i, j)}. Step 3: After all the pixels in E are scanned, the three subsets S , M and W are obtained, and |S | + |M | + |W | = |E |. The three subsets S , M and W correspond to the step, roof and convex type respectively. In general, most natural images contain all three types. When an image is blurred, whether caused by the defocusing or in an artificial manner, the number of step edge points will decrease. What is more, both step and roof edges tend to lose their sharpness. In other words, the
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
b
a
2651
c
d
e
Fig. 4. NSCT demonstration images (a) Original image; (b) Low frequency image; (c) NSCT coefficients- level 1 with 2 directions; (d) NSCT coefficientslevel 2 with 4 directions; (e) NSCT coefficients- level 3 with 8 directions.
(a) Step.
(b) Roof.
(c) Convex. Fig. 5. Edge types.
cardinality of the set S will become smaller, while the case is contrary for that of the set W . Taking the gray-value image Lena with size 256 × 256 as example, adopting Canny edge detector, we can draw the cardinalities of the three sets for the original image, the blurred image I (Gaussian blurring n1 = n2 = 5, σ 2 = 1) and the blurred image II (Gaussian blurring n1 = n2 = 5, σ 2 = 2) shown as Fig. 6. The example above indicates the effect of blurring on the image edges. 3.2. Feature extraction When images are blurred with image processing software, such as, Photoshop, users may be asked to choose the blurring mode. Since some blurring modes support user-defined filters, people can manipulate images with diverse blurring radiuses and intensity. According to the algorithm realized in Photoshop, the blurring is motion averaging filtering of chosen image regions to smooth the image. Different modes involve different motion averaging filtering functions. The width of filter window determines the blur radius, and filter parameters reflect the blurring intensity. In other words, blurring is the operation of replacing the original pixel point values with the neighborhood average values.
2652
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
Fig. 6. The cardinalities of three edge sets before and after blurring.
(a) Set W .
(b) Set M.
(c) Set S. Fig. 7. NCST coefficients before and after Gaussian blurring.
3.2.1. Blur features In order to discover the difference between normal edges and blurred edges, we select one hundred images, which include several content types such as landscapes, figures, construction and so on. The sets S , M and W of each image are obtained after the edge detection. 10 elements in each set of each image are randomly selected, the component Φ3 (i, j) of which are shown with black spots in Fig. 7 (a) With the same operation to the 100 blurred images (Gaussian blurring n1 = n2 = 5, σ 2 = 1.5), the component Φ3 (i, j) are shown by green spots in Fig. 7(b) and (c). Regardless of edge types, it can be seen from Fig. 7 that the components Φ3 (i, j) of normal images are larger than those of the blurred images. Therefore we take Φ3 (i, j) as the one-dimensional feature of the edge point. In order to enhance the high frequency components that contain edge information, we propose to predict each pixel value in the original image by using its neighboring pixels’ values, and obtain the prediction-error image by subtracting the predicted image from the original image. It is expected that the process will remove most low frequency information, and make the detection more accurate. The prediction-error image is the difference between the image and its predicted
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
2653
Fig. 8. Prediction context.
version. The prediction algorithm is given by Eq. (4).
xˆ =
max(a, b) if c ≤ min(a, b) min(a, b) if c ≥ max(a, b) a + b − c otherwise.
(4)
Here a, b, c are the context pixels of the pixel x, xˆ is the prediction value of x as illustrated in Fig. 8. When the image is blurred, the prediction error becomes smaller than that of the normal. So we also extract features in the same manner from the prediction error image (1x = |x − xˆ |), Therefore, the component Φ3′ (i, j) which is the NSCT coefficient of the prediction-error image at the position (i, j) that is the position of the edge point in the original image, is also extracted as one of the features. 3.2.2. Phase congruency feature Phase congruency (PC) was first defined by Morrone and Burr [28] in terms of Fourier series expansion of a signal at some location x as
PC2 (x) =
W (x) ⌊An (x)1Ψn (x) − T ⌋
n
An (x) + ε
,
(5)
n
where An (x) is the amplitude of the nth Fourier component, An (x)1Ψn (x) is the local energy function at position x, W (x) is the weight function of the filter at position x, and T is the estimated noise energy. The phase deviation function 1Ψn (x) is defined as follows
1Ψn (x) = cos(Ψn (x) − Ψ¯ n (x)) − sin(Ψn (x) − Ψ¯ n (x)) .
(6)
If PC equals to the maximal value of 1, all frequency components are in phase, otherwise, PC takes on values between 0 and 1. PC of image edges will become smaller after blur operation. When the phase congruency of the prediction error image is obtained, at the corresponding positions of the image edge points, the first four moments, mean (µ), variance (ϖ ), skewness (σ ) and kurtosis (κ) of a 3 × 3 region are obtained as the features. Totally, for an edge point at position (i, j), the six-dimensional feature vector F = [Φ3 , Φ3′ , µ, ϖ , σ , κ] is obtained. We can perform the blur detection by F to distinguish the blurred edges from normal edges. Then the artificial blurred edges are differentiated from the defocused blurred edges by the local definition. The detailed detection algorithm is described in the next section. 4. Detection algorithm 4.1. Classifier training In this paper, Support Vector Machines (SVM) are adopted to distinguish blurred edges from normal ones. The SVM classifier is very popular and has drawn lots of attentions in pattern recognition and machine learning field. It does not depend explicitly on the dimensionality of the problem thus the solution is always globally optimal. At the training stage, given a set of training sample–label pairs (⃗ vi , yi ), i = 1, 2, . . . , L where v⃗i is the feature vector of the ith training sample and yi ∈ {1, −1} is the corresponding class label, SVM solves the following optimization problem.
min
w, ⃗ b,αi
1 2
w ⃗Tw + C
f
αi
(7)
i =1
s.t. yi (w ⃗ T φ(⃗vi ) + b) ≥ 1 − αi ,
αi ≥ 0.
Here, C is the penalty parameter of the error term, and the function φ maps the training feature vector v ⃗i into a higher dimensional space. The kernel function K (⃗ vi , v⃗j ) = φ(⃗vi )T φ(⃗vj ) is used to avoid the explicit computation of the function φ . In our experiment, we use the radial basis function (RBF) as the kernel function, which is given in Eq. (8). K (⃗ vi , v⃗j ) = exp(−γ ∥⃗vi − v⃗j ∥2 ),
γ > 0.
(8)
2654
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
Fig. 9. Detection process.
Here, γ is the kernel parameter. The penalty parameter C and kernel parameter γ must be carefully chosen so that the SVM can give accurate classification results. The ‘‘grid search’’ method with cross-validation is employed to find the optimal C and γ . During the training process, for each edge type, the corresponding classifiers SVM1, SVM2, SVM3 are trained for the edge set S , M and W respectively. 100 images are selected including gray-scale images and color images. Training data 1: Perform NSCT on each image and its prediction-error image. For each image, edge detection is adopted. Therefore, the edge sets S , M and W of each image can be obtained. 10 elements of each set in each image are selected randomly. The six-dimensional feature vector F = [Φ3 , Φ3′ , µ, ϖ , σ , κ] is extracted for those edge points. So we can obtain 1000 six-dimensional feature samples for each edge type. Training data 2: Perform NSCT on each blurred image (50 images Gaussian blur n1 = n2 = 5, σ 2 = 1, 50 images Gaussian blur n1 = n2 = 5, σ 2 = 2) and its prediction-error image. For each image, the edge sets S , M and W of each image can be obtained by the edge detection algorithm. 10 elements of each set in each image are selected randomly. The same features are extracted from the blurred images. The 1000 six-dimensional feature samples for each edge type of blurred images are obtained. Using the above datum, the corresponding SVM1, SVM2, and SVM3 can be trained. 4.2. Local definition It is important to tell the differences between the defocus blur and the artificial one to judge whether there exists the forgery. As we know, the artificial blurring is the result of filtering the edge region with the blur radius decided manually, so pixels outside the blur radius are not processed usually. But the camera defocus blurring does not have a fixed blur radius. It is the most important difference which can be exploited. In this paper we use the local definition G defined by Eq. (9) to detect the artificial blurring operation. G(i, j) =
1
θ
[θ/ 2]
[θ/ 2]
2
((∂ f (i, j)/∂ x)2 + (∂ f (i, j)/∂ y)2 )/2.
(9)
i=−[θ/2] j=−[θ/2]
Here, ∂ f (i, j)/∂ x and ∂ f (i, j)/∂ y denote the differential along x and y direction respectively at the edge point position (i, j). θ , a non-negative integer, is the width of the window, whose value indicates the size of the partial region with the center edge point position (i, j). During the experiment, we find that the values of G(i, j) for the artificial blurred edge points are larger than those of the defocused blurred edge point. So the local definition can be used to judge whether the edge is artificially blurred. While, the question is how to select the threshold, θ and Gth . The larger θ is, the higher the computational complexity is. And when the value of θ is smaller, the result of the local definition cannot tell the difference between the artificial blurring and the defocus blurring. According to the common practices of image splicing forgery, forgers often blur the splicing edges with the radius equaling to 2 or 3 pixels, because a large radius would bring about obvious blurring traces. So in this paper, the width of local definition window θ is chosen as 8, which means that the radius equals to 4 and is a little larger than the common used size. In order to determine the threshold of local definition, we perform an additional experiments on lots of artificial and defocus blurred images, and find when the threshold of Gth is chosen equal to 9.7, the two kinds of blurring can be distinguished well. 4.3. Detection process According to the above analysis and discussion, the whole detection process can be described by Fig. 9. The detailed steps are introduced below. Step 1: Perform the edge detection on a given image I. The edge set E is obtained. Step 2: Perform NSCT on the image and its prediction-error image with the decomposition level equal to 3 and the number of the decomposition directions equal to 8. According to Section 3.1, the edge sets S , M and W can be obtained.
1
1
0.9
0.9
0.8
0.8
0.7
0.7
True positive probability
True positive probability
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
0.6 0.5 0.4 0.3
0.5 0.4 0.3 0.2
0.1
0.1 0
0.2
0.4 0.6 False positive probability
0.8
0
1
(a) JPEG compression (80) and blurring modes A.
1
1
0.9
0.9
0.8
0.8
0.7
0.7
0.6 0.5 0.4 0.3
1
0.8
1
0.3
0.1 0.4 0.6 False positive probability
(c) JPEG compression (80) and blurring modes B.
0.8
0.4
0.1 0.2
0.4 0.6 False positive probability
0.5
0.2
0
0.2
0.6
0.2
0
0
(b) JPEG compression (60) and blurring modes A.
True positive probability
True positive probability
0.6
0.2
0
2655
0.8
1
0
0
0.2
0.4 0.6 False positive probability
(d) JPEG Compression (60) and blurring modes B. Fig. 10. Comparison results of ROC curves.
Step 3: Initialize an all-zero matrix Ξ with the size equal to that of the given image. Scan all the edge points. For each edge point, extract the six-dimensional feature and use SVM1, SVM2 and SVM3 respectively to find the blurred edge point. If an edge point at the position (i, j) is judged as a blurred one, recode it by setting Ξ (i, j) = 1. Step 4: Initialize an all-zero matrix Θ with the size equal to that of the given image. Scan all the blurred edge points with Ξ (i, j) equal to 1. For a blurred edge point at the position (i, j), calculate the local definition Gij , if Gij > Gth , the edge point is more likely to be an artificial blurred edge point, recode the result by setting Θ (i, j) = 1. Step 5: Initialize an all-zero matrix Λ with the size equal to that of the given image. Scan all the edge points with Θ (i, j) equal to 1. For the possibly artificial blurred edge point at the position (i, j), calculate Π (i, j) by the equation given below.
Π (i, j) =
1 1
Θ (i + p, j + q).
(10)
p=−1 q=−1
If Π (i, j) ≥ 3, the edge point at the position (i, j) is determined as the artificial blurred edge point ultimately, and record the result by setting Λ(i, j) = 1. Finally, we can plot the matrix Λ as a binary image and locate the tampered region.
2656
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
(a) Un-tampered image.
(b) Tampered image.
(c) Detection result. Fig. 11. The detection result of the gray-level image.
5. Experiment results and analysis The proposed algorithm mentioned above is fit for gray-level images. There are two ways to extend it to the color case. One is to convert a color image to a gray-scale one by the formula I = 0.299R + 0.587G + 0.114B, the other is to process each color channel (RGB) to yield three detection results independently and get the final result by performing an ‘‘AND’’ operation and voting mechanism. In our experiments, the first method is adopted. Two experiments are carried out to evaluate the performance of our method. 5.1. Blur detection experiment Although the proposed method is mainly designed to detect image splicing with blurred boundary, it can also be used to detect the image blur itself. The ratio µ defined by |Ξ |/|E | can be used to judge whether an image is blurred by being compared with a threshold. The wavelet-based blur detection method [27] is selected to make the comparison. In [27], the image is decomposed by a 3-level Haar wavelet, the edge points are found by comparing the local maxima in three scales with a threshold, and the Dirac- and Astep-structure edge point is determined by the order relationship of the maxima in three scales. The image is judged as a blurred one if the ratio between the number of all Dirac- and Astep- structure edge points and the number of edge points is less than a given value which is called MinZero in [27]. In the experiment, the threshold to judge an edge point is chosen as 35 as suggested in [27]. Considering that the spliced image is commonly saved in a low-quality JPEG format to further reduce the tampering traces in most forgery processes of image splicing. The experiment is designed to investigate the robustness of our blur detection method against JPEG compression. There are total of 1000 images captured by a Canon A710 camera that are used in the experiment. These images are all clear and un-blurred. Firstly, the clear un-blurred images are blurred by two Guassian blurring modes (mode A : 5 × 5, σ 2 = 1 and mode B : 4 × 4, σ 2 = 1) respectively. Secondly, those blurred images are recompressed with the quality factor equal
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
(a) Un-tampered image.
2657
(b) Tampered image.
(c) Detection result. Fig. 12. The detection result of the color image.
to 80 and 60 respectively. Those clear images are also recompressed with the two quality factors directly. The blurred and recompressed images and only recompressed images are input into the blur detection algorithms of our method and the method in [27]. The MinZero and µ = |Ξ |/|E | are chosen from their minimum value and maximum value to compute the true and false positive probabilities under different threshold for plotting the ROC curves. The results are shown in Fig. 10. From the experimental results, it can be observed that our method is robust against JPEG compression and outperforms the wavelet-based method at all testing conditions. According to our experiment, the detection results of the wavelet-based method are not very reliable under JPEG compression. The main reason is that there are only the Dirac- and Astep- structure edge points that are used to judge the blurring. Meanwhile, it can be found that the JPEG compression also cause a certain effect on the detection accuracy because of the noises introduced by the compression. 5.2. Splicing detection experiment The second experiment is performed to test the validity of our splicing detection method. Two examples are given by Figs. 11 and 12. Fig. 11(b) is a fake image that is generated by copying the truck in the bottom left corner of Fig. 11(a), pasting it onto itself and blurring the spliced edges. Fig. 11(c) is the detection result according to our method. In Fig. 12(b), the head of the person in Fig. 12(a) is replaced. Moreover the edge blurring operation is also carried out. Fig. 12(c) is the detection result. To test the performance of our method, we also perform another experiment. Firstly, 41 color images captured by a DSLR Camera Pentax K100D (the soft focus function of the camera will cause defocus blur) and 48 color images captured by a Canon A710 are chosen as testing images. Secondly, image splicing forgery with artificial edge blurring is performed on every image using Photoshop CS3. The tampering process is composed of copying a region of one image, pasting it onto the target image and adopting the blur tool provided by Photoshop to eliminate visual distortion around the spliced boundary. The blurring radius is chosen as 2 or 3 pixels. After these operations, a total of 89 tampered images are obtained. Furthermore,
2658
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659 Table 1 Splicing detection results. Image source
TP (%)
FP (%)
Pentax K100D Canon A710
95.12 91.67
8.17 6.54
there are another 147 and 153 un-tampered images captured by the two cameras which are selected in the experiment to test the false detection level. The experimental results are listed in Table 1, where TP represents the true positive probability of the splicing detection, FP represents the false positive probability. According to the results, we can see that our method can detect the image splicing with artificial blurred edges well. 6. Conclusions In this paper, the detection problem of image splicing with artificial blurred boundary is studied, and a new detection method is proposed. In our method, NSCT is used to distinguish the three edge types. For an edge point, two NSCT coefficients, four statistical values from the phase congruency are composed as the feature. Three SVMs are used to detect the blurred edge points for various edge types. And a local definition is defined and used to filter the blurred edge points caused by the defocus. Experimental results show that the proposed method can detect the splicing with artificial blurred edges well. It should be mentioned that the proposed method can just be taken as an auxiliary manner to judge whether an image is tampered because there is only the cue of the artificial blurring operation is used, and the accuracy of our method is not high enough to avoid people’s participation. In future work, we will pay more attention to look for more efficient features to improve the detection performance against image splicing with blurred boundaries. And since there are various complex forgery ways of tampering an image, how to find and use those tampering traces is also an open issue to be studied in the future. Acknowledgments This study was supported by the Post Doctor Foundation of China (Grant no. 20070421017), NSF of Jiangsu province (Grant no. BK2008403, BK2010484), NUST Research Funding (Grant no. 2010ZYTS048), and NSF of China (Grant no. 60974129). References [1] S.-G. Lian, Y. Zhang, Multimedia forensics for detecting forgeries, in: Handbook on Communications and Information Security, 2010, pp. 801–820. [2] T.-T. Ng, M. Tsui, Camera response function signature for digital forensics-part I: theory and data selection, in: Proceedings of IEEE Workshop on Information Forensics and Security, 2009, pp. 156–160. [3] G.-J. Liu, J.-W. Wang, S.-G. Lian, Z.-Q. Wang, A passive image authentication scheme for detecting region-duplication forgery with rotation, Journal of Network and Computer Applications (2010) doi:10.1016/j.jnca.2010.09.001. [4] P. Zhang, X.-W. Kong, Detecting image tampering using feature fusion, in: Proceedings of International Conference on Availability, Reliability and Security, 2009, pp. 335–340. [5] Z.-H. Qu, G.-P. Qiu, J.-W. Huang, Detect digital image splicing with visual cues, in: Proceedings of International Workshop on Information Hiding, 2009, pp. 247–261. [6] A.-C. Popescu, H. Farid, Exposing digital forgeries by detecting duplicated image regions, Dartmouth College, USA, 2004. [7] M.-K. Johnson, H. Farid, Exposing digital forgeries by detecting inconsistencies in lighting, Presented at the Proceedings of ACM Multimedia Security Workshop, 2005. [8] J. Fridrich, D. Soukal, J. Lukas, Detection of copy–move forgery in digital images, Presented at the Proceedings of Digital Forensic Research Workshop, 2003. [9] J. Lukas, J. Fridrich, M. Goljan, Digital camera identification from sensor pattern noise, IEEE Transactions on Information Forensics and Security 1 (2006) 205–214. [10] B. Mahdian, S. Saic, A bibliography on blind methods for identifying image forgery, Signal Processing: Image Communication 25 (2010) 389–399. [11] W. Wang, J. Dong, T.-N. Tan, A survey of passive image tampering detection, in: Proceeding of International Workshop on Digital Watermarking, in: LNCS, vol. 5370, 2009, pp. 308–322. [12] Y.-Q. Shi, C.-H. Chen, G.-R. Xuan, W. Su, Steanalysis versus splicing detection, in: Proceeding of International Workshop on Digital Watermarking, in: LNCS, vol. 5041, 2007, pp. 4–13. [13] D.-D. Fu, Y.-Q. Shi, W. Su, Detection of image splicing based on Hilbert–Huang transform and moments of characteristic functions with wavelet decomposition, in: Proceeding of International Workshop on Digital Watermarking, in: LNCS, vol. 4283, 2006, pp. 177–187. [14] W. Chen, Y.-Q. Shi, W. Su, Image splicing detection using 2-D phase congruency and statistical moments of characteristic function, in: Proceedings of SPIE Security and Watermarking of Multimedia Contents IX, vol. 6505, 2007, pp. 6505R.1–6505R.8. [15] W. Wang, J. Dong, T.-N. Tan, Effective image splicing detection based on image chroma, in: Proceedings of International Conference on Image Processing, 2009, pp. 1257–1260. [16] Z. Zhang, J.-Q. Kang, Y. Ren, An effective algorithm of image splicing detection, in: Proceeding of International Conference on Computer Science and Software Engineering, 2008, pp. 1035–1039. [17] M.-K. Johnson, H. Farid, Exposing digital forgeries in complex lighting environments, IEEE Transactions on Information Forensics and Security 2 (2007) 450–461. [18] M.-K. Johnson, H. Farid, Exposing digital forgeries through chromatic aberration, in: Proceeding of ACM Multimedia and Security, 2006, pp. 48–55. [19] M. Chen, J. Fridrich, J. Lukas, M. Goljan, Imaging sensor noise as digital X-ray for revealing forgeries, in: Proceeding of International Workshop on Information Hiding, in: LNCS, vol. 4567, 2008, pp. 342–358.
G. Liu et al. / Mathematical and Computer Modelling 57 (2013) 2647–2659
2659
[20] A.-C. Popescu, H. Farid, Exposing digital forgeries in color filter array interpolated images, Signal Processing: Image Communication 53 (2005) 3948–3959. [21] J.-W. Wang, G.-J. Liu, Y.-W. Dai, Z.-Q. Wang, Detecting JPEG image forgery based on double compression, Journal of Systems Engineering and Electronics 20 (2009) 1096–1103. [22] J. Wang, H.-B. Zhang, Exposing digital forgeries by detecting traces of image splicing, in: Proceeding of International Conference on Signal Processing, 2006, pp. 16–20. [23] D.-Y. Hsiao, S.-C. Pei, Detecting digital tampering by blur estimation, in: Proceeding of the First International Workshop on Systematic Approaches to Digital Forensic Engineering, 2005, pp. 264–267. [24] L.-N. Zhou, D.-M. Wang, Y.-B. Guo, J.-F. Zhang, Blur detection of digital forgery using mathematical morphology, in: Proceeding of KES AMSTA, 2007, pp. 990–998. [25] Y. Sutcu, B. Coskun, H.T. Sencar, N. Memon, Tamper detection based on regularity of wavelet transform coefficients, in: Proceeding of International Conference on Image Processing, 2007, pp. 397–400. [26] A.-L. Cunha, J. Zhou, M.-N. Do, Non-subsampled contourlet transform: filter design and applications in denoising, in: Proceeding of International Conference on Image Processing, 2005, pp. 749–752. [27] H.-H. Tong, M.-J.H.-J. Zhang, C.-S. Zhang, Blur detection for digital images using wavelet transform, in: Proceedings of IEEE Conference on Multimedia and Expo., 2004, pp. 17–20. [28] M.-C. Morrone, D.-C. Burr, Feature detection in human vision: a phase-dependent energy model, Proceedings of the Royal Society of London, Series B, Biological Sciences 235 (1988) 221–245.