- Email: [email protected]
Object based dual watermarking for video authentication
G Model IJLEO-52914; No. of Pages 8
ARTICLE IN PRESS Optik xxx (2013) xxx–xxx
Contents lists available at SciVerse ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
Object based dual watermarking for video authentication Yanjiao Shi a,b , Miao Qi a , Yugen Yi a , Ming Zhang a,∗ , Jun Kong a,b,∗ a School of Computer Science and Information Technology, Northeast Normal University, Key Laboratory of Intelligent Information Processing of Jilin Universities, Changchun 130117, China b School of Mathematics and Statistics, Northeast Normal University, Changchun 130117, China
a r t i c l e
i n f o
Article history: Received 25 June 2012 Accepted 16 November 2012 Keywords: Video authentication Self-embedding Video tampering Synthesized frame
a b s t r a c t Video cameras have been widely installed in public facilities for the surveillance applications. So, video authentication has becoming increasingly attractive. This paper presents a dual watermarking for video authentication based on moving objects. For each frame, the frame index, as a watermark is first embedded into the moving objects of the corresponding frame using a reversible watermarking method, aiming to detect the temporal tampering. Then the principle content and the details of the moving objects combined with the authentication code, as the other watermark, are embedded into the frame for spatial tampering location and recovery. Specially, a synthesized frame method is proposed for lossless recovery of moving objects and effective extraction of frame index. Statistical analysis and experiment results show that the proposed method can locate spatial, temporal and spatio-temporal tampering accurately. The spatial tampered regions can be recovered approximately and the moving objects can be restored completely when the tampered area is limited. © 2013 Elsevier GmbH. All rights reserved.
1. Introduction With the development of multimedia and Internet technologies, video cameras have been widely installed in public facilities for the surveillance applications. The integrity and validity of video play an important role in applications such as intelligence information gathering, criminal evidence, security surveillance, and insurance claims. However, the trustworthiness of video content could no longer be granted since users can easily manipulate, modify or forge digital content without causing noticeable traces by using easy-to-use editing software. The edited videos do not have any value for legal proof. Therefore, video authentication is becoming a hot research topic nowadays. Fragile watermarking technology provides useful solution and has been widely researched for multimedia authentication [1,2]. In the past two decades, many fragile watermarking methods with good tamper detection and location ability have been proposed for image authentication [3–7]. Due to the development of video and surveillance technology, video authentication watermarking has been widely researched [8–13]. Literature [8] could detect cut-and-splice or cut-insert-splice operation by embedding a watermark with a strong timing content. In literature [9], a secure and robust authentication scheme for scalable video streaming by
∗ Corresponding author. E-mail addresses: [email protected] (M. Zhang), [email protected] (J. Kong).
employing ECC was proposed which was insensitive to those incidental distortions while sensitive to intentional distortions such as frame alterations and insertion. In [10], the watermark which represented the serial numbers of video segments was embedded into nonzero quantization indices of frames. It could locate the edited segments in the tampered video and the embedded watermarks could survive the allowed transcoding processes. A novel video content authentication algorithm for MPEG-2 was described in literature [11]. The watermark bits were generated according to the image features of I-frame and embedded into the low-frequency DCT coefficients. However, these video authentication methods can only detect the tamper of either temporal domain or spatial domain. For enhancing the performance and detecting the tamper both in spatial domain and temporal domain, the spatial tampering was located by comparing the extracted with the original feature-based watermarks in [12]. The frame index of each video frame embedded in the residual coefficients was used to reveal the temporal tempering. It could discriminate the malicious tampering from the mild signal processing. In [13], the temporal information of each frame was modulated into the parameters of a chaotic system and the output was embedded into DCT coefficients as watermark. By inspecting on the difference between the original and the extracted watermark, the performed tampering was detected. However, these two methods have not considered restoring the tampered content. It is a waste of resources to retransmit any images or videos under the condition that tampering is detected. So it is desired and necessary to restore the tampered content in practical applications. A watermarking method named
0030-4026/$ – see front matter © 2013 Elsevier GmbH. All rights reserved. http://dx.doi.org/10.1016/j.ijleo.2012.11.078
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model IJLEO-52914; No. of Pages 8
ARTICLE IN PRESS Y. Shi et al. / Optik xxx (2013) xxx–xxx
2
“self-embedding” was first proposed for image to reconstruct the tampered regions [14]. In self-embedding, the compressed version of an image or video was embedded into itself. At the authentication end, the extracted compressed version used to reconstruct the tempered portions. Recently, some self-embedding methods have been proposed for image and video [15–20]. In [15,16], data representing the principal content of a region were hidden in a different region. If both regions were tampered, restoration would fail. This phenomenon was called tampering coincidence problem [19]. To reduce the probability of tampering coincidence, multiple description coding was adopted in [17,18]. Nevertheless, it increases the probability of self-recovery at the cost of recover quality. Later, [19,20] proposed a reference sharing mechanism to solve this problem, in which the watermark was a reference derived from the original principal content of different regions and shared by these regions for content restoration. After identifying tampered regions, both the reference data and the original content in the reserved area were used to recover the tampered area. By this inspired, a novel video authentication method is proposed in this paper. This paper proposes an object based dual-watermarking method which not only can detect spatial, temporal and spatiotemporal tampering, but also can restore the spatial tampered area when the tampered area is limited. The index of each frame is embedded into moving objects using a reversible watermarking method for revealing temporal tampering. In spatial authentication phase, the principle content of a frame and the details of the moving objects are collected and embedded into the corresponding frame combined with the hash code. Spatial tampering can be located by comparing the extracted hash code with the computed one. If the watermarked video is authentic, the moving objects can be restored to the state before the embedding of frame index. In the case that the video has been spatially tampered and the tampered area is not too extensive, the tampered area can be restored approximately and the moving objects can also be restored to the state before the embedding of the frame index. After the spatial authentication and the restoration of the moving objects, the frame index can be extracted precisely and the objects can be restored completely. The rest of this paper is organized as follows. Section 2 describes the proposed method including watermark embedding, authentication and restoration. In Section 3, statistical analysis of recovery capability is depicted. Experimental results are given in Section 4. Finally, the conclusions are drawn in Section 5.
2. The proposed method 2.1. Dual watermark embedding For each frame, two watermarks are embedded for detecting tamper and recovery. The first watermark representing the frame index is embedded into the moving objects firstly. Then the reference of principle content and the details of moving objects, with the authentication code are embedded into the corresponding frame as the second watermark. For completely recovery, the moving objects should be consistent before and after embedding. However, the embedding of second watermark may induce distortion to the host frame and this distortion will result in different moving objects extraction results. So a novel synthesized frame method is proposed to solve this problem. To begin the process, the host video is decomposed frames. For a color video frame I, we first decompose its three color channels r, g and b to eight bit-plans bc7 , bc6 , bc5 , bc4 , bc3 , bc2 , bc1 , bc0 , c = r, g, b, respectively. We call bc7 , bc6 , bc5 , bc4 , bc3 the most significant bit (MSB) dataset and bc2 , bc1 , bc0 the least significant bit (LSB) dataset. The flowchart of embedding is given in Fig. 1.
Fig. 1. Flowchart of watermark embedding.
2.1.1. Frame synthesis and moving objects extraction For each video frame I, a synthesized frame Is is constructed. The property of synthesized frame is that the synthesized at the embedding end is the same as the one at the authentication end. Taking into account the changes of the pixels by embedding the dual watermarks, the bc2 , bc1 , bc0 are set as ‘100’, so as to approximate to the original frame. The synthesized Is is generated by downsampling and up-sampling the modified frame. Fig. 2 shows an example of generating a synthesized frame. The moving objects mask Obw is obtained as: dif = abs(gray(Is ) − gray(Ibkg )),
Obw (i, j) =
1
dif(i, j) ≥ T,
0
otherwise,
(1)
where Ibkg is the synthesized background image as the same way as synthesizing Is . T is a binarized threshold. The final moving objects O are obtained by Eq. (2) and it is 2 × 2 block wised.
O(i, j, c) =
⎧ ⎪ ⎨ ⎪ ⎩
I(i, j, c)
i/2 ×2
j/2 ×2
Obw (m, n) > 2,
m=i/2×2−1n=j/2×2−1
0
(2)
otherwise
Fig. 3 shows the results of moving objects extraction. It can be seen that this method can approximately extract the moving objects. 2.1.2. The first watermark embedding The first watermark is embedded into the O to get watermarked moving objects Ow1 and watermarked frame Iw1 . For convenient embedding, the frame index is modulated into binary sequence and embedded into the moving objects using the difference expansion method [21]. It is used to reveal temporal tampering. The reason of this embedding manner is that the moving objects can be restored losslessly. In the process of embedding, the pixels of moving objects are divided into two categories, as shown in Fig. 4. The pixels labeled by ‘’ and ‘’ are the unchangeable and changeable pixels, respectively. Scan the changeable pixels from
Fig. 2. An example of synthesizing a frame.
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model IJLEO-52914; No. of Pages 8
ARTICLE IN PRESS Y. Shi et al. / Optik xxx (2013) xxx–xxx
3
Fig. 3. Results of moving objects extraction.
top-left one by one for every color channels until the whole watermark is embedded. The rule of embedding is as follows: g˜ c = gu + [gc − gu ] × 2 + w,
(3)
where gc , gu and g˜ c are changeable, unchangeable and watermarked changeable pixels in one color channel, w is the watermark bit to be embedded. If there will be underflow or overflow, the current changeable pixels is set as 0 or 255 as following:
g˜ c =
0
gc
< gu and gu + [gc − gu ] × 2 ≤ 0,
255 gc ≥ gu and gu + [gc − gu ] × 2 + 1 ≥ 255.
Fig. 5. Flowchart of authentication and restoration.
(4)
The watermark bit is embedded into the next changeable pixels. 2.1.3. The second watermark generation and embedding The MSB of one frame and LSB of the moving objects are collected to form a binary string B, which is regarded as the important content to be protected. Motived by literature [17], the data embedded into the frame is reference data but not the dataset itself. Denote the width and height of each frame as w and h, the pixels number of moving objects as N. LMSB and LLSB represent the length of MSB and LSB respectively, so the length of B is LMSB + LLSB , where LLSB = N × 3 × 3. We permute and divide B into M subsets with equal length Ld bits. Denote the bits in kth subset as dk,1 , dk,2 , . . . , dk,Ld and its reference bits rk,1 , rk,2 , . . . , rk,Lr are generated as following:
⎡
rk,1
⎤
⎡d
k,1
⎢r ⎥ ⎢d ⎢ k,2 ⎥ ⎢ k,2 ⎢ ⎥=A×⎢ ⎢ . ⎥ ⎢ . ⎣ .. ⎦ ⎣ .. rk,Lr
⎤ ⎥ ⎥ ⎥ , k = 1, 2, . . . , M ⎥ ⎦
dk,Ld
Fig. 4. The mask of two categories pixels.
(5)
where Lr is the length of each group of reference bits, A is a pseudorandom binary matrix sized Lr × Ld . The arithmetic in Eq. (5) is modulo-2. Because of the permutation of B dataset, the Lr reference bits are generated from the Ld bits that scattered into the entire frame. The number of reference bits is Nref (Nref = 15/2 × w × h), so the length of reference bits of each group is Lr = Nref /M . Then we permute the Nref reference bits as a part of the second watermark. The permutation and generation of matrix A are keys depended. For detecting the spatial tempering, the hash function is adopted to generate the check codes as the other part of the second watermark. First, divide the frame into blocks sized 8 × 8. Accordingly, divide the Nref reference bits into (w × h)/64 groups. Therefore, a one-to-one relationship is established. For one block, its MSB and corresponding reference bits are fed to hash function to get 96 hash bits. Then the 480 reference bits and 96 hash bits are permuted and embedded into the LSB. So, a watermarked video frame Iw2 including a watermarked objects Ow2 is obtained. 2.2. Authentication and restoration At the authentication end, the watermarked video is decomposed into frames denoted by Iw2 . A flowchart illustrating the authentication and restoration procedure is given in Fig. 5. For each frame Iw2 , divide it into blocks sized 8 × 8 first, and then extract the LSB of every block. For one block, the extracted LSB dataset is de-permuted and decomposed into reference bits and hash bits. We feed the MSB of the current block and the extracted reference bits into hash function. If the generated hash bits are equal to the extracted hash bits, the block is judged as reserved; otherwise, the block is tampered. If all blocks in a frame are reserved, the moving objects can be restored as following: Step 1: Synthesized frame Is is constructed firstly. Because the down-sample of the MSB in a frame is not changed in watermark
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model
ARTICLE IN PRESS
IJLEO-52914; No. of Pages 8
Y. Shi et al. / Optik xxx (2013) xxx–xxx
4
embedding section, it can be guaranteed that the Is in this step is same with the one at the embedding end. In this step, the extracted moving objects are denoted as Ow2 . Step 2: Then the LSB data can be solved using MSB and extracted reference bits as following:
⎡
rk,1
⎤
⎢r ⎥ ⎢ k,2 ⎥ ⎢ ⎥ ⎢ . ⎥ − AM × DM = AL × DL , ⎣ .. ⎦
(6)
rk,Lr where DM and DL are MSB and LSB, AM and AL are matrices whose columns are corresponding to MSB in DM and LSB in DL . The left side and AL are known and the purpose is to find DL . If the linear equations have a unique solution, the LSB can be solved successfully. Denote the length of DL as nL , if and only if the rank of AL equals to nL , that is, the columns of AL are linearly independent, a unique solution to Eq. (2) exists. The probability of successfully restored is analyzed in Section 3.1. After getting the LSB data, the moving objects with first watermark Ow1 and corresponding frame Iw1 are obtained. Step 3: Lastly, the index of the frame can be extracted. Using the pattern defined in embedding phase, one bit can be extracted using the following rule w = mod [˜gc − gu , 2] .
(7)
And the original pixel in one color channel can be recovered as: gc = gu +
g˜ − g c u
rk,e(1)
video. For security consideration, it can be protected by an image authentication method. In the process of moving object extraction, two cases may occur and can be solved as: (1) If the number of the moving pixels is too small to carry the frame index-watermark bits, a dilating operation will be done to the moving object mask until all watermark bits can be embedded. (2) In the case that there is no moving object in the video frame, that is, the number of moving pixel is 0, the sufficient pixels on fixed position (for example, the left corner) are selected to carry the frame index.
(8)
2
3. Statistical analysis
Then the original moving objects O can be got. If one or more blocks are judged as tampered, the MSB of the tampered blocks and LSB of the moving objects can be restored as
⎡
Fig. 6. The recovering probability under different ˛.
⎤
⎡ ⎤ D(R,M) ⎢r ⎥ ⎢ k,e(2) ⎥ ⎢ ⎥ E ⎢ ⎥ ⎢ . ⎥ = A × ⎣ D(T,M) ⎦ , ⎣ .. ⎦
3.1. The probability of successfully restored in the un-tampered case For a random binary matrix sized i × j, probability of its columns being linearly dependent is denoted q(i, j),
(9) q(i, 1) =
DL
rk,e(v) AE
where rk,e(1) , rk,e(2) , . . . , rk,e(v) are extractable reference bits, is a matrix whose rows are those of A that corresponding to extractable reference bits, D(R,M) , D(T,M) and DL are reserved MSB, tampered MSB and LSB respectively. We can reformulate Eq. (6) as:
⎡
rk,e(1)
⎤
⎢r ⎥ D(T,M) ⎢ k,e(2) ⎥ E E E ⎢ ⎥ , ⎢ . ⎥ − A(R,M) × D(R,M) = [A(T,M) AL ] × DL ⎣ .. ⎦
(10)
,
q(i, j + 1) = q(i, j) + [1 − q(i, j)] × q(i, j) = 1,
2j 2i
,
j = 1, 2, . . . , i − 1,
(11)
if j > i.
Denote LSB rate LLSB / (LLSB + LMSB ) as ˛, and nL obeys a binomial distribution
rk,e(v) AE(R,M) , AE(T,M)
1 2i
PnL (j) =
Ld j
× ˛j × (1 − ˛)(Ld −j) ,
j = 0, 1, . . . , Ld .
(12)
The probability of all columns of AL being linearly independent is AEL
where and are matrix whose columns are those of A that corresponding to reserved MSB, tampered MSB and LSB, respectively. The left side of Eq. (10) and AE(T,M) , AEL are known and the purpose is to find D(T,M) and DL . The probability of successfully restored is analyzed in Section 3.2. A synthesized frame Is is constructed just like the step 1 of Section 2.2. The solved DL is used to restore the moving objects to Ow1 . Then the frame index can be extracted and the original moving objects O can be got similarly as the step 3 of Section 2.2. The background image used at authentication end should be the same with that at the embedding end. However, the same background is difficult to get in case the watermarked video is tempered. So the background image should be transmission together with the
L PLI =
Ld
PnL (j) × [1 − q(Lr , j)] .
(13)
j=0
All blocks in a frame can be restored with the probability L(LLSB +LMSB )/Ld P L = PLI .
(14)
The value of PL depends on Ld and ˛. The larger the value of Ld is, the higher the probability of successful restoration. However, a larger Ld will lead to higher computation complexity. To make a tradeoff, the Ld is set to 200 in this article. Fig. 6 shows the recovering probability under different ˛.
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model
ARTICLE IN PRESS
IJLEO-52914; No. of Pages 8
Y. Shi et al. / Optik xxx (2013) xxx–xxx
5
Fig. 7. The recovering probability under different ˛ and ˇ.
3.2. The probability of successfully restored in the tampered case Denote tampered rate ˇ as the ratio between the number of tampered blocks and all blocks, and then the probability of successfully recovering can be estimated. The number of extractable reference bits in one group v obeys a binomial distribution
Pv (i) =
Lr
i
× (1 − ˇ) × ˇ
i
(Lr −i)
Fig. 9. Watermarked frames (a) 50th frame of video I, (b) 100th frame of video I, (c) 50th frame of video II and (d) 100th frame of video II.
i = 0, 1, . . . , Lr .
,
(15)
The columns of matrix AE(T,M) nT also obeys a binomial distribution
PnT (j1 ) =
Ld
j1
j
× (ˇ) 1 × (1 − ˇ)
(Ld −j1 )
,
j1 = 0, 1, . . . , Ld .
T(LLSB +LMSB )/Ld P T = PLI .
j
PnT (k) × PnL (j − k),
The value of PT depends on LSB rate ˛, tampered rate ˇ and Ld . The value of PT relatives to ˛ and ˇ is shown in Fig. 7.
E
j = 0, 1, . . . Ld .
k=0
So the probability of all columns of pendent is as follows: T PLI
=
Lr Ld
AE(T,M) AEL
Pv (i) × PnTL (j) [1 − q(i, j)] .
(19)
(16)
Denote the columns of matrix AE(T,M) AL as nTL , and PnTL (j) =
All blocks in one frame can be restored with the probability
4. Experimental results (17)
being linearly inde-
The test videos used in experiment are two color surveillance videos both with 150 frames sized 384 × 288. Two frames of each test video are shown in Fig. 8. The frame index are modulated to binary sequence sized 9 bits.
(18)
i=0 j=0
Fig. 8. Original frames (a) 50th frame of video I, (b) 100th frame of video I, (c) 50th frame of video II and (d) 100th frame of video II.
Fig. 10. Imperceptibility evaluation results. (a) PSNR of video I and (b) PSNR of video II.
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model IJLEO-52914; No. of Pages 8
ARTICLE IN PRESS Y. Shi et al. / Optik xxx (2013) xxx–xxx
6
Fig. 11. (a) The ˛ values of each frame in video I. (b) The ˛ values of each frame in video II.
4.1. Imperceptibility evaluation Peak Signal to Noise Ratio (PSNR) is used to evaluate the perceptual quality. The watermarked frames corresponding to Fig. 8 are shown in Fig. 9. There is no visible artifact that can be observed in all of the test video sequences. Fig. 10 shows the PSNR values of all frames. It can be seen that the PSNR values of video I and video II are from 37.84 dB to 37.93 dB and 37.67 dB to 37.76 dB, respectively, which are acceptable for the imperceptibility requirement.
Fig. 13. (a) Tampered version of 100th frame in video I by replacing parts of background. (b) Location of tampered regions for (a). (c) Restored version of (a). (d) Difference magnified by a factor 30 of original 100th frame and (c). (e) Tampered version of 100th frame in video II by deleting moving object. (f) Location of tampered regions for (e). (g) Restored version of (e). (h) Difference magnified by a factor 30 of original 100th frame and (g).
4.2. The recovering ability evaluation in the un-tamper case In the un-tamper case, the moving objects can be recovered losslessly. Fig. 11 shows the ˛ values of every frame in each test video. We can see that the ␣ values of all frames are from 1.54% to 3.30% in video I and are from 0.05% to 1.06% in video II. According to Fig. 6, it can be got that the moving objects in both test videos can be successfully recovered with probability 1.0. The recovered frames and the difference between original frames are shown in Fig. 12.
Table 1 The related parameters of spatial tampering test.
Fig. 12. (a) Recovered 100th frame of video I. (b) Difference between (a) and corresponding original frame. (c) Recovered 100th frame of video II. (b) Difference between (c) and corresponding original frame.
Video I Video II
LSB rate (%)
Tampering rate (%)
PSNR after recovering (dB)
2.92 0.79
2.89 2.08
38.03 37.78
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model IJLEO-52914; No. of Pages 8
ARTICLE IN PRESS Y. Shi et al. / Optik xxx (2013) xxx–xxx
7
Fig. 15. (a) Original 40th frame. (b) Spatial tampered version of (a). (c) Location of tampered regions for (b). (d) Restored version of (a). (e) Difference magnified by a factor 30 of (a) and (d).
4.3. Spatial tampering In order to illustrate the ability of recovering both moving objects and the background, two types of spatial tampering are designed. One is to tamper the moving object pixels and the other one is to tamper the background. For video I, the regions of background are replaced by two other images (shown as Fig. 13(a)). And the moving object is deleted for video II (shown as Fig. 13(e)). The results of tamper location are shown in Fig. 13(b) and (f) where black regions are tampered. Fig. 13(c) and (g) are the recovered version for Fig. 13(a) and (e), respectively. Fig. 13(d) and (h) are the differences between original frame and restored one from which we can see that the moving objects have been restored completely. The related parameters in this test are shown in Table 1.
Fig. 14. Results of temporal tampering detection: (a) without tampering, (b) frame replacing, (c) frame dropping and (d) frame exchanging.
Fig. 16. Results of temporal tampering detection under spatial tampering case.
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model IJLEO-52914; No. of Pages 8
ARTICLE IN PRESS Y. Shi et al. / Optik xxx (2013) xxx–xxx
8
4.4. Temporal tampering
Acknowledgments
In this experiment, frame replacing, frame dropping and frame exchanging are performed to video I. Without any temporal tampering manipulation, the extracted frame number versus the observed frame number is shown in Fig. 14(a). In the frame replacing test, the frames 51–100 are replaced by frames 1–50 sequentially. The authentication result is shown in Fig. 14(b). It illustrates that the observed frame indexes are consistent with the extracted ones for frames 1–50 and 101–150, but not for frames 51–100. That is to say, there is temporal tampering to frames 51–100. In the frame dropping test, frames 51–100 are dropped. From Fig. 14(c), it can be seen that there are 100 frames in the current video, but the maximum extracted frame index is 150 and there is a jump between frames 50 and 101. That demonstrates frames 51–100 have been tampered in temporal domain. In the frame exchanging test, the first 30 frames and last 30 frames are exchanged. Fig. 14(d) shows the results of authentication. From Fig. 14 we can see that proposed method can detect the temporal tampering and tampering type properly. Furthermore, this method can re-range the tampered frames by the extracted frame indexes, that is, it can recover the frame-exchanging tampered video to the original one.
This work is supported by the Young Scientific Research Foundation of Jilin Province Science and Technology Development Project (No. 201201070, No. 201201063), the Jilin Provincial Natural Science Foundation (No. 201115003), the Fund of Jilin Provincial Science & Technology Department (No. 20111804, NO. 20110364), the Science Foundation for Post-doctor of Jilin Province (No. 2011274), and the Program for Changjiang Scholars and Innovative Research Team in University (PCSIRT).
4.5. Spatio-temporal tampering In this test, the test video I is tampered in spatial domain firstly. The tempering style is to delete all moving object pixels and add two images for all frames (shown as Fig. 15(b)). Then, in temporal domain, the frames 51–100 and 101–150 are replaced by frames 1–50 simultaneously. Spatial authentication and restoration are performed first and all frames can be restored in spatial domain. The results are shown in Fig. 15(c)–(e). In temporal authentication, the frames 51–100 can be judged as tempered and tampering type can be got also. So the frames 1–50 are the trusted frames. The result of temporal authentication is shown in Fig. 16. It illustrates that the proposed method can detect the temporal tampering even after the video has suffered spatial tampering. 5. Conclusions A novel video authentication method taking the moving objects into consideration has been proposed in this paper. The synthesized frame method has been proposed to ensure the same moving objects are obtained in embedding end and authentication end. By embedding dual watermark to the host video, the proposed method can locate spatial, temporal and spatio-temporal tampering accurately. When there is spatial tempering and the tampering area is not too extensive, the tampered regions can be recovered approximately and the moving objects can be restored completely. If there is no tempering, the moving objects can be restored losslessly. At the embedding of the first watermark, for the pixels that with pixel value 0 or 255, that is the underflow or overflow case, the distortion cannot be avoided. Fortunately, such case is rare. So it can be overlooked. Besides, the compression versions of the digital videos are abounded around, so a video authentication method that can resist video compression is more practical and will be further studied in the future.
References [1] F.A.P. Petitcolas, R.J. Anderson, M.G. Kuhn, Information hiding-a survey, Proc. IEEE 87 (1999) 1062–1078. [2] D. Kundur, D. Hatzinaoks, Digital watermarking for telltale tamper proofing and authentication, Proc. IEEE 87 (1999) 1167–1180. [3] J. Fridrich, Methods for tamper detection in digital images, in: Multimedia and Security Workshop at ACM Multimedia 99, Orlando, FL, USA, 1999. [4] M. Celik, G. Sharma, E. Saber, A.M. Tekalp, Hierarchical watermarking for secure image authentication with localization, IEEE Trans. Image Process. 11 (2002) 585–595. [5] W.C. Chen, M.S. Wang, A fuzzy c-means clustering-based fragile watermarking scheme for image authentication, Expert Syst. Appl. 36 (2009) 1300–1307. [6] S.H. Liu, H.X. Yao, W. Gao, Y.L. Liu, An image fragile watermark scheme based on chaotic image pattern and pixel-pairs, Appl. Math. Comput. 185 (2007) 869–882. [7] H.J. He, J.S. Zhang, F. Chen, Adjacent-block based statistical detection method for self-embedding watermarking techniques, Signal Process. 89 (2009) 1557–1566. [8] B.G. Mobasseri, M.J. Sieffert, Content authentication and tamper detection in digital video, in: Proceedings of the IEEE International Conference on Image Processing, Vancouver, BC, Canada, 2000. [9] Q.B. Sun, D.J. He, Z.S. Zhang, Q. Tian, A secure and robust approach to scalable video authentication, in: Proceedings of 2003 International Conference on Multimedia and Expo, vol. 2, 2003, pp. 209–212. [10] P.C. Su, C.S. Wu, I.F. Chen, C.Y. Wu, Y.C. Wu, A practical design of digital video watermarking in H.264/AVC for content authentication, Signal Process Image Commun. 26 (2011) 413–426. [11] X.L. Chen, H.M. Zhao, A novel video content authentication algorithm combined semi-fragile watermarking with compressive sensing, in: Second International Conference on Intelligent Systems Design and Engineering Application, Sanya, Hainan, 2012. [12] D.W. Xu, R.D. Wang, J.C. Wang, A novel watermarking scheme for H.264/AVC video authentication, Signal Process Image Commun. 26 (2011) 267–279. [13] S.Y. Chen, H. Leung, Chaotic watermarking for video authentication in surveillance applications, IEEE Trans. Circuits Syst. Video Technol. 18 (2008) 704–709. [14] J. Fridrich, M. Goljan, Images with self-correcting capabilities, in: Proceedings of the IEEE International Conference on Image Processing, Kobe, Japan, 1999. [15] C.Y. Lin, S.F. Chang, SARI: self-authentication and -recover image watermarking system, in: Proceedings of the 9th ACM International Conference on Multimedia, Ottawa, Canada, 2001. [16] X. Zhu, A. Ho, P. Marziliano, A new semi-fragile image watermarking with robust tampering restoration using irregular sampling, Signal Process Image Commun. 22 (2007) 515–528. [17] M.U. Celik, G. Sharma, A.M. Tekalp, E. Saber, Video authentication with self recovery, in: Proceedings of Security and Watermarking of Multimedia Contents IV, vol. 4314, 2002, pp. 531–541. [18] A.M. Hassan, A. Al-Hamadi, Y.M.Y. Hasan, M.A.A. Wahab, B. Michaelis, Secure Block-based video authentication with localization and self-recovery, Proceedings of World Academy of Science, Eng. Technol. (2009) 69–74. [19] X.P. Zhang, S.Z. Wang, Fragile watermarking with error-free restoration capability, IEEE Trans. Multimedia 10 (2008) 1490–1499. [20] X.P. Zhang, S.Z. Wang, Z.X. Qian, G.R. Feng, Reference sharing mechanism for watermark self-Embedding, IEEE Trans. Image Process. 20 (2011) 485–495. [21] J. Tian, Reversible data embedding using a difference expansion, IEEE Trans. Circuits Syst. Video Technol. 13 (2003) 890–896.
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
ARTICLE IN PRESS Optik xxx (2013) xxx–xxx
Contents lists available at SciVerse ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
Object based dual watermarking for video authentication Yanjiao Shi a,b , Miao Qi a , Yugen Yi a , Ming Zhang a,∗ , Jun Kong a,b,∗ a School of Computer Science and Information Technology, Northeast Normal University, Key Laboratory of Intelligent Information Processing of Jilin Universities, Changchun 130117, China b School of Mathematics and Statistics, Northeast Normal University, Changchun 130117, China
a r t i c l e
i n f o
Article history: Received 25 June 2012 Accepted 16 November 2012 Keywords: Video authentication Self-embedding Video tampering Synthesized frame
a b s t r a c t Video cameras have been widely installed in public facilities for the surveillance applications. So, video authentication has becoming increasingly attractive. This paper presents a dual watermarking for video authentication based on moving objects. For each frame, the frame index, as a watermark is first embedded into the moving objects of the corresponding frame using a reversible watermarking method, aiming to detect the temporal tampering. Then the principle content and the details of the moving objects combined with the authentication code, as the other watermark, are embedded into the frame for spatial tampering location and recovery. Specially, a synthesized frame method is proposed for lossless recovery of moving objects and effective extraction of frame index. Statistical analysis and experiment results show that the proposed method can locate spatial, temporal and spatio-temporal tampering accurately. The spatial tampered regions can be recovered approximately and the moving objects can be restored completely when the tampered area is limited. © 2013 Elsevier GmbH. All rights reserved.
1. Introduction With the development of multimedia and Internet technologies, video cameras have been widely installed in public facilities for the surveillance applications. The integrity and validity of video play an important role in applications such as intelligence information gathering, criminal evidence, security surveillance, and insurance claims. However, the trustworthiness of video content could no longer be granted since users can easily manipulate, modify or forge digital content without causing noticeable traces by using easy-to-use editing software. The edited videos do not have any value for legal proof. Therefore, video authentication is becoming a hot research topic nowadays. Fragile watermarking technology provides useful solution and has been widely researched for multimedia authentication [1,2]. In the past two decades, many fragile watermarking methods with good tamper detection and location ability have been proposed for image authentication [3–7]. Due to the development of video and surveillance technology, video authentication watermarking has been widely researched [8–13]. Literature [8] could detect cut-and-splice or cut-insert-splice operation by embedding a watermark with a strong timing content. In literature [9], a secure and robust authentication scheme for scalable video streaming by
∗ Corresponding author. E-mail addresses: [email protected] (M. Zhang), [email protected] (J. Kong).
employing ECC was proposed which was insensitive to those incidental distortions while sensitive to intentional distortions such as frame alterations and insertion. In [10], the watermark which represented the serial numbers of video segments was embedded into nonzero quantization indices of frames. It could locate the edited segments in the tampered video and the embedded watermarks could survive the allowed transcoding processes. A novel video content authentication algorithm for MPEG-2 was described in literature [11]. The watermark bits were generated according to the image features of I-frame and embedded into the low-frequency DCT coefficients. However, these video authentication methods can only detect the tamper of either temporal domain or spatial domain. For enhancing the performance and detecting the tamper both in spatial domain and temporal domain, the spatial tampering was located by comparing the extracted with the original feature-based watermarks in [12]. The frame index of each video frame embedded in the residual coefficients was used to reveal the temporal tempering. It could discriminate the malicious tampering from the mild signal processing. In [13], the temporal information of each frame was modulated into the parameters of a chaotic system and the output was embedded into DCT coefficients as watermark. By inspecting on the difference between the original and the extracted watermark, the performed tampering was detected. However, these two methods have not considered restoring the tampered content. It is a waste of resources to retransmit any images or videos under the condition that tampering is detected. So it is desired and necessary to restore the tampered content in practical applications. A watermarking method named
0030-4026/$ – see front matter © 2013 Elsevier GmbH. All rights reserved. http://dx.doi.org/10.1016/j.ijleo.2012.11.078
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model IJLEO-52914; No. of Pages 8
ARTICLE IN PRESS Y. Shi et al. / Optik xxx (2013) xxx–xxx
2
“self-embedding” was first proposed for image to reconstruct the tampered regions [14]. In self-embedding, the compressed version of an image or video was embedded into itself. At the authentication end, the extracted compressed version used to reconstruct the tempered portions. Recently, some self-embedding methods have been proposed for image and video [15–20]. In [15,16], data representing the principal content of a region were hidden in a different region. If both regions were tampered, restoration would fail. This phenomenon was called tampering coincidence problem [19]. To reduce the probability of tampering coincidence, multiple description coding was adopted in [17,18]. Nevertheless, it increases the probability of self-recovery at the cost of recover quality. Later, [19,20] proposed a reference sharing mechanism to solve this problem, in which the watermark was a reference derived from the original principal content of different regions and shared by these regions for content restoration. After identifying tampered regions, both the reference data and the original content in the reserved area were used to recover the tampered area. By this inspired, a novel video authentication method is proposed in this paper. This paper proposes an object based dual-watermarking method which not only can detect spatial, temporal and spatiotemporal tampering, but also can restore the spatial tampered area when the tampered area is limited. The index of each frame is embedded into moving objects using a reversible watermarking method for revealing temporal tampering. In spatial authentication phase, the principle content of a frame and the details of the moving objects are collected and embedded into the corresponding frame combined with the hash code. Spatial tampering can be located by comparing the extracted hash code with the computed one. If the watermarked video is authentic, the moving objects can be restored to the state before the embedding of frame index. In the case that the video has been spatially tampered and the tampered area is not too extensive, the tampered area can be restored approximately and the moving objects can also be restored to the state before the embedding of the frame index. After the spatial authentication and the restoration of the moving objects, the frame index can be extracted precisely and the objects can be restored completely. The rest of this paper is organized as follows. Section 2 describes the proposed method including watermark embedding, authentication and restoration. In Section 3, statistical analysis of recovery capability is depicted. Experimental results are given in Section 4. Finally, the conclusions are drawn in Section 5.
2. The proposed method 2.1. Dual watermark embedding For each frame, two watermarks are embedded for detecting tamper and recovery. The first watermark representing the frame index is embedded into the moving objects firstly. Then the reference of principle content and the details of moving objects, with the authentication code are embedded into the corresponding frame as the second watermark. For completely recovery, the moving objects should be consistent before and after embedding. However, the embedding of second watermark may induce distortion to the host frame and this distortion will result in different moving objects extraction results. So a novel synthesized frame method is proposed to solve this problem. To begin the process, the host video is decomposed frames. For a color video frame I, we first decompose its three color channels r, g and b to eight bit-plans bc7 , bc6 , bc5 , bc4 , bc3 , bc2 , bc1 , bc0 , c = r, g, b, respectively. We call bc7 , bc6 , bc5 , bc4 , bc3 the most significant bit (MSB) dataset and bc2 , bc1 , bc0 the least significant bit (LSB) dataset. The flowchart of embedding is given in Fig. 1.
Fig. 1. Flowchart of watermark embedding.
2.1.1. Frame synthesis and moving objects extraction For each video frame I, a synthesized frame Is is constructed. The property of synthesized frame is that the synthesized at the embedding end is the same as the one at the authentication end. Taking into account the changes of the pixels by embedding the dual watermarks, the bc2 , bc1 , bc0 are set as ‘100’, so as to approximate to the original frame. The synthesized Is is generated by downsampling and up-sampling the modified frame. Fig. 2 shows an example of generating a synthesized frame. The moving objects mask Obw is obtained as: dif = abs(gray(Is ) − gray(Ibkg )),
Obw (i, j) =
1
dif(i, j) ≥ T,
0
otherwise,
(1)
where Ibkg is the synthesized background image as the same way as synthesizing Is . T is a binarized threshold. The final moving objects O are obtained by Eq. (2) and it is 2 × 2 block wised.
O(i, j, c) =
⎧ ⎪ ⎨ ⎪ ⎩
I(i, j, c)
i/2 ×2
j/2 ×2
Obw (m, n) > 2,
m=i/2×2−1n=j/2×2−1
0
(2)
otherwise
Fig. 3 shows the results of moving objects extraction. It can be seen that this method can approximately extract the moving objects. 2.1.2. The first watermark embedding The first watermark is embedded into the O to get watermarked moving objects Ow1 and watermarked frame Iw1 . For convenient embedding, the frame index is modulated into binary sequence and embedded into the moving objects using the difference expansion method [21]. It is used to reveal temporal tampering. The reason of this embedding manner is that the moving objects can be restored losslessly. In the process of embedding, the pixels of moving objects are divided into two categories, as shown in Fig. 4. The pixels labeled by ‘’ and ‘’ are the unchangeable and changeable pixels, respectively. Scan the changeable pixels from
Fig. 2. An example of synthesizing a frame.
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model IJLEO-52914; No. of Pages 8
ARTICLE IN PRESS Y. Shi et al. / Optik xxx (2013) xxx–xxx
3
Fig. 3. Results of moving objects extraction.
top-left one by one for every color channels until the whole watermark is embedded. The rule of embedding is as follows: g˜ c = gu + [gc − gu ] × 2 + w,
(3)
where gc , gu and g˜ c are changeable, unchangeable and watermarked changeable pixels in one color channel, w is the watermark bit to be embedded. If there will be underflow or overflow, the current changeable pixels is set as 0 or 255 as following:
g˜ c =
0
gc
< gu and gu + [gc − gu ] × 2 ≤ 0,
255 gc ≥ gu and gu + [gc − gu ] × 2 + 1 ≥ 255.
Fig. 5. Flowchart of authentication and restoration.
(4)
The watermark bit is embedded into the next changeable pixels. 2.1.3. The second watermark generation and embedding The MSB of one frame and LSB of the moving objects are collected to form a binary string B, which is regarded as the important content to be protected. Motived by literature [17], the data embedded into the frame is reference data but not the dataset itself. Denote the width and height of each frame as w and h, the pixels number of moving objects as N. LMSB and LLSB represent the length of MSB and LSB respectively, so the length of B is LMSB + LLSB , where LLSB = N × 3 × 3. We permute and divide B into M subsets with equal length Ld bits. Denote the bits in kth subset as dk,1 , dk,2 , . . . , dk,Ld and its reference bits rk,1 , rk,2 , . . . , rk,Lr are generated as following:
⎡
rk,1
⎤
⎡d
k,1
⎢r ⎥ ⎢d ⎢ k,2 ⎥ ⎢ k,2 ⎢ ⎥=A×⎢ ⎢ . ⎥ ⎢ . ⎣ .. ⎦ ⎣ .. rk,Lr
⎤ ⎥ ⎥ ⎥ , k = 1, 2, . . . , M ⎥ ⎦
dk,Ld
Fig. 4. The mask of two categories pixels.
(5)
where Lr is the length of each group of reference bits, A is a pseudorandom binary matrix sized Lr × Ld . The arithmetic in Eq. (5) is modulo-2. Because of the permutation of B dataset, the Lr reference bits are generated from the Ld bits that scattered into the entire frame. The number of reference bits is Nref (Nref = 15/2 × w × h), so the length of reference bits of each group is Lr = Nref /M . Then we permute the Nref reference bits as a part of the second watermark. The permutation and generation of matrix A are keys depended. For detecting the spatial tempering, the hash function is adopted to generate the check codes as the other part of the second watermark. First, divide the frame into blocks sized 8 × 8. Accordingly, divide the Nref reference bits into (w × h)/64 groups. Therefore, a one-to-one relationship is established. For one block, its MSB and corresponding reference bits are fed to hash function to get 96 hash bits. Then the 480 reference bits and 96 hash bits are permuted and embedded into the LSB. So, a watermarked video frame Iw2 including a watermarked objects Ow2 is obtained. 2.2. Authentication and restoration At the authentication end, the watermarked video is decomposed into frames denoted by Iw2 . A flowchart illustrating the authentication and restoration procedure is given in Fig. 5. For each frame Iw2 , divide it into blocks sized 8 × 8 first, and then extract the LSB of every block. For one block, the extracted LSB dataset is de-permuted and decomposed into reference bits and hash bits. We feed the MSB of the current block and the extracted reference bits into hash function. If the generated hash bits are equal to the extracted hash bits, the block is judged as reserved; otherwise, the block is tampered. If all blocks in a frame are reserved, the moving objects can be restored as following: Step 1: Synthesized frame Is is constructed firstly. Because the down-sample of the MSB in a frame is not changed in watermark
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model
ARTICLE IN PRESS
IJLEO-52914; No. of Pages 8
Y. Shi et al. / Optik xxx (2013) xxx–xxx
4
embedding section, it can be guaranteed that the Is in this step is same with the one at the embedding end. In this step, the extracted moving objects are denoted as Ow2 . Step 2: Then the LSB data can be solved using MSB and extracted reference bits as following:
⎡
rk,1
⎤
⎢r ⎥ ⎢ k,2 ⎥ ⎢ ⎥ ⎢ . ⎥ − AM × DM = AL × DL , ⎣ .. ⎦
(6)
rk,Lr where DM and DL are MSB and LSB, AM and AL are matrices whose columns are corresponding to MSB in DM and LSB in DL . The left side and AL are known and the purpose is to find DL . If the linear equations have a unique solution, the LSB can be solved successfully. Denote the length of DL as nL , if and only if the rank of AL equals to nL , that is, the columns of AL are linearly independent, a unique solution to Eq. (2) exists. The probability of successfully restored is analyzed in Section 3.1. After getting the LSB data, the moving objects with first watermark Ow1 and corresponding frame Iw1 are obtained. Step 3: Lastly, the index of the frame can be extracted. Using the pattern defined in embedding phase, one bit can be extracted using the following rule w = mod [˜gc − gu , 2] .
(7)
And the original pixel in one color channel can be recovered as: gc = gu +
g˜ − g c u
rk,e(1)
video. For security consideration, it can be protected by an image authentication method. In the process of moving object extraction, two cases may occur and can be solved as: (1) If the number of the moving pixels is too small to carry the frame index-watermark bits, a dilating operation will be done to the moving object mask until all watermark bits can be embedded. (2) In the case that there is no moving object in the video frame, that is, the number of moving pixel is 0, the sufficient pixels on fixed position (for example, the left corner) are selected to carry the frame index.
(8)
2
3. Statistical analysis
Then the original moving objects O can be got. If one or more blocks are judged as tampered, the MSB of the tampered blocks and LSB of the moving objects can be restored as
⎡
Fig. 6. The recovering probability under different ˛.
⎤
⎡ ⎤ D(R,M) ⎢r ⎥ ⎢ k,e(2) ⎥ ⎢ ⎥ E ⎢ ⎥ ⎢ . ⎥ = A × ⎣ D(T,M) ⎦ , ⎣ .. ⎦
3.1. The probability of successfully restored in the un-tampered case For a random binary matrix sized i × j, probability of its columns being linearly dependent is denoted q(i, j),
(9) q(i, 1) =
DL
rk,e(v) AE
where rk,e(1) , rk,e(2) , . . . , rk,e(v) are extractable reference bits, is a matrix whose rows are those of A that corresponding to extractable reference bits, D(R,M) , D(T,M) and DL are reserved MSB, tampered MSB and LSB respectively. We can reformulate Eq. (6) as:
⎡
rk,e(1)
⎤
⎢r ⎥ D(T,M) ⎢ k,e(2) ⎥ E E E ⎢ ⎥ , ⎢ . ⎥ − A(R,M) × D(R,M) = [A(T,M) AL ] × DL ⎣ .. ⎦
(10)
,
q(i, j + 1) = q(i, j) + [1 − q(i, j)] × q(i, j) = 1,
2j 2i
,
j = 1, 2, . . . , i − 1,
(11)
if j > i.
Denote LSB rate LLSB / (LLSB + LMSB ) as ˛, and nL obeys a binomial distribution
rk,e(v) AE(R,M) , AE(T,M)
1 2i
PnL (j) =
Ld j
× ˛j × (1 − ˛)(Ld −j) ,
j = 0, 1, . . . , Ld .
(12)
The probability of all columns of AL being linearly independent is AEL
where and are matrix whose columns are those of A that corresponding to reserved MSB, tampered MSB and LSB, respectively. The left side of Eq. (10) and AE(T,M) , AEL are known and the purpose is to find D(T,M) and DL . The probability of successfully restored is analyzed in Section 3.2. A synthesized frame Is is constructed just like the step 1 of Section 2.2. The solved DL is used to restore the moving objects to Ow1 . Then the frame index can be extracted and the original moving objects O can be got similarly as the step 3 of Section 2.2. The background image used at authentication end should be the same with that at the embedding end. However, the same background is difficult to get in case the watermarked video is tempered. So the background image should be transmission together with the
L PLI =
Ld
PnL (j) × [1 − q(Lr , j)] .
(13)
j=0
All blocks in a frame can be restored with the probability L(LLSB +LMSB )/Ld P L = PLI .
(14)
The value of PL depends on Ld and ˛. The larger the value of Ld is, the higher the probability of successful restoration. However, a larger Ld will lead to higher computation complexity. To make a tradeoff, the Ld is set to 200 in this article. Fig. 6 shows the recovering probability under different ˛.
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model
ARTICLE IN PRESS
IJLEO-52914; No. of Pages 8
Y. Shi et al. / Optik xxx (2013) xxx–xxx
5
Fig. 7. The recovering probability under different ˛ and ˇ.
3.2. The probability of successfully restored in the tampered case Denote tampered rate ˇ as the ratio between the number of tampered blocks and all blocks, and then the probability of successfully recovering can be estimated. The number of extractable reference bits in one group v obeys a binomial distribution
Pv (i) =
Lr
i
× (1 − ˇ) × ˇ
i
(Lr −i)
Fig. 9. Watermarked frames (a) 50th frame of video I, (b) 100th frame of video I, (c) 50th frame of video II and (d) 100th frame of video II.
i = 0, 1, . . . , Lr .
,
(15)
The columns of matrix AE(T,M) nT also obeys a binomial distribution
PnT (j1 ) =
Ld
j1
j
× (ˇ) 1 × (1 − ˇ)
(Ld −j1 )
,
j1 = 0, 1, . . . , Ld .
T(LLSB +LMSB )/Ld P T = PLI .
j
PnT (k) × PnL (j − k),
The value of PT depends on LSB rate ˛, tampered rate ˇ and Ld . The value of PT relatives to ˛ and ˇ is shown in Fig. 7.
E
j = 0, 1, . . . Ld .
k=0
So the probability of all columns of pendent is as follows: T PLI
=
Lr Ld
AE(T,M) AEL
Pv (i) × PnTL (j) [1 − q(i, j)] .
(19)
(16)
Denote the columns of matrix AE(T,M) AL as nTL , and PnTL (j) =
All blocks in one frame can be restored with the probability
4. Experimental results (17)
being linearly inde-
The test videos used in experiment are two color surveillance videos both with 150 frames sized 384 × 288. Two frames of each test video are shown in Fig. 8. The frame index are modulated to binary sequence sized 9 bits.
(18)
i=0 j=0
Fig. 8. Original frames (a) 50th frame of video I, (b) 100th frame of video I, (c) 50th frame of video II and (d) 100th frame of video II.
Fig. 10. Imperceptibility evaluation results. (a) PSNR of video I and (b) PSNR of video II.
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model IJLEO-52914; No. of Pages 8
ARTICLE IN PRESS Y. Shi et al. / Optik xxx (2013) xxx–xxx
6
Fig. 11. (a) The ˛ values of each frame in video I. (b) The ˛ values of each frame in video II.
4.1. Imperceptibility evaluation Peak Signal to Noise Ratio (PSNR) is used to evaluate the perceptual quality. The watermarked frames corresponding to Fig. 8 are shown in Fig. 9. There is no visible artifact that can be observed in all of the test video sequences. Fig. 10 shows the PSNR values of all frames. It can be seen that the PSNR values of video I and video II are from 37.84 dB to 37.93 dB and 37.67 dB to 37.76 dB, respectively, which are acceptable for the imperceptibility requirement.
Fig. 13. (a) Tampered version of 100th frame in video I by replacing parts of background. (b) Location of tampered regions for (a). (c) Restored version of (a). (d) Difference magnified by a factor 30 of original 100th frame and (c). (e) Tampered version of 100th frame in video II by deleting moving object. (f) Location of tampered regions for (e). (g) Restored version of (e). (h) Difference magnified by a factor 30 of original 100th frame and (g).
4.2. The recovering ability evaluation in the un-tamper case In the un-tamper case, the moving objects can be recovered losslessly. Fig. 11 shows the ˛ values of every frame in each test video. We can see that the ␣ values of all frames are from 1.54% to 3.30% in video I and are from 0.05% to 1.06% in video II. According to Fig. 6, it can be got that the moving objects in both test videos can be successfully recovered with probability 1.0. The recovered frames and the difference between original frames are shown in Fig. 12.
Table 1 The related parameters of spatial tampering test.
Fig. 12. (a) Recovered 100th frame of video I. (b) Difference between (a) and corresponding original frame. (c) Recovered 100th frame of video II. (b) Difference between (c) and corresponding original frame.
Video I Video II
LSB rate (%)
Tampering rate (%)
PSNR after recovering (dB)
2.92 0.79
2.89 2.08
38.03 37.78
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model IJLEO-52914; No. of Pages 8
ARTICLE IN PRESS Y. Shi et al. / Optik xxx (2013) xxx–xxx
7
Fig. 15. (a) Original 40th frame. (b) Spatial tampered version of (a). (c) Location of tampered regions for (b). (d) Restored version of (a). (e) Difference magnified by a factor 30 of (a) and (d).
4.3. Spatial tampering In order to illustrate the ability of recovering both moving objects and the background, two types of spatial tampering are designed. One is to tamper the moving object pixels and the other one is to tamper the background. For video I, the regions of background are replaced by two other images (shown as Fig. 13(a)). And the moving object is deleted for video II (shown as Fig. 13(e)). The results of tamper location are shown in Fig. 13(b) and (f) where black regions are tampered. Fig. 13(c) and (g) are the recovered version for Fig. 13(a) and (e), respectively. Fig. 13(d) and (h) are the differences between original frame and restored one from which we can see that the moving objects have been restored completely. The related parameters in this test are shown in Table 1.
Fig. 14. Results of temporal tampering detection: (a) without tampering, (b) frame replacing, (c) frame dropping and (d) frame exchanging.
Fig. 16. Results of temporal tampering detection under spatial tampering case.
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078
G Model IJLEO-52914; No. of Pages 8
ARTICLE IN PRESS Y. Shi et al. / Optik xxx (2013) xxx–xxx
8
4.4. Temporal tampering
Acknowledgments
In this experiment, frame replacing, frame dropping and frame exchanging are performed to video I. Without any temporal tampering manipulation, the extracted frame number versus the observed frame number is shown in Fig. 14(a). In the frame replacing test, the frames 51–100 are replaced by frames 1–50 sequentially. The authentication result is shown in Fig. 14(b). It illustrates that the observed frame indexes are consistent with the extracted ones for frames 1–50 and 101–150, but not for frames 51–100. That is to say, there is temporal tampering to frames 51–100. In the frame dropping test, frames 51–100 are dropped. From Fig. 14(c), it can be seen that there are 100 frames in the current video, but the maximum extracted frame index is 150 and there is a jump between frames 50 and 101. That demonstrates frames 51–100 have been tampered in temporal domain. In the frame exchanging test, the first 30 frames and last 30 frames are exchanged. Fig. 14(d) shows the results of authentication. From Fig. 14 we can see that proposed method can detect the temporal tampering and tampering type properly. Furthermore, this method can re-range the tampered frames by the extracted frame indexes, that is, it can recover the frame-exchanging tampered video to the original one.
This work is supported by the Young Scientific Research Foundation of Jilin Province Science and Technology Development Project (No. 201201070, No. 201201063), the Jilin Provincial Natural Science Foundation (No. 201115003), the Fund of Jilin Provincial Science & Technology Department (No. 20111804, NO. 20110364), the Science Foundation for Post-doctor of Jilin Province (No. 2011274), and the Program for Changjiang Scholars and Innovative Research Team in University (PCSIRT).
4.5. Spatio-temporal tampering In this test, the test video I is tampered in spatial domain firstly. The tempering style is to delete all moving object pixels and add two images for all frames (shown as Fig. 15(b)). Then, in temporal domain, the frames 51–100 and 101–150 are replaced by frames 1–50 simultaneously. Spatial authentication and restoration are performed first and all frames can be restored in spatial domain. The results are shown in Fig. 15(c)–(e). In temporal authentication, the frames 51–100 can be judged as tempered and tampering type can be got also. So the frames 1–50 are the trusted frames. The result of temporal authentication is shown in Fig. 16. It illustrates that the proposed method can detect the temporal tampering even after the video has suffered spatial tampering. 5. Conclusions A novel video authentication method taking the moving objects into consideration has been proposed in this paper. The synthesized frame method has been proposed to ensure the same moving objects are obtained in embedding end and authentication end. By embedding dual watermark to the host video, the proposed method can locate spatial, temporal and spatio-temporal tampering accurately. When there is spatial tempering and the tampering area is not too extensive, the tampered regions can be recovered approximately and the moving objects can be restored completely. If there is no tempering, the moving objects can be restored losslessly. At the embedding of the first watermark, for the pixels that with pixel value 0 or 255, that is the underflow or overflow case, the distortion cannot be avoided. Fortunately, such case is rare. So it can be overlooked. Besides, the compression versions of the digital videos are abounded around, so a video authentication method that can resist video compression is more practical and will be further studied in the future.
References [1] F.A.P. Petitcolas, R.J. Anderson, M.G. Kuhn, Information hiding-a survey, Proc. IEEE 87 (1999) 1062–1078. [2] D. Kundur, D. Hatzinaoks, Digital watermarking for telltale tamper proofing and authentication, Proc. IEEE 87 (1999) 1167–1180. [3] J. Fridrich, Methods for tamper detection in digital images, in: Multimedia and Security Workshop at ACM Multimedia 99, Orlando, FL, USA, 1999. [4] M. Celik, G. Sharma, E. Saber, A.M. Tekalp, Hierarchical watermarking for secure image authentication with localization, IEEE Trans. Image Process. 11 (2002) 585–595. [5] W.C. Chen, M.S. Wang, A fuzzy c-means clustering-based fragile watermarking scheme for image authentication, Expert Syst. Appl. 36 (2009) 1300–1307. [6] S.H. Liu, H.X. Yao, W. Gao, Y.L. Liu, An image fragile watermark scheme based on chaotic image pattern and pixel-pairs, Appl. Math. Comput. 185 (2007) 869–882. [7] H.J. He, J.S. Zhang, F. Chen, Adjacent-block based statistical detection method for self-embedding watermarking techniques, Signal Process. 89 (2009) 1557–1566. [8] B.G. Mobasseri, M.J. Sieffert, Content authentication and tamper detection in digital video, in: Proceedings of the IEEE International Conference on Image Processing, Vancouver, BC, Canada, 2000. [9] Q.B. Sun, D.J. He, Z.S. Zhang, Q. Tian, A secure and robust approach to scalable video authentication, in: Proceedings of 2003 International Conference on Multimedia and Expo, vol. 2, 2003, pp. 209–212. [10] P.C. Su, C.S. Wu, I.F. Chen, C.Y. Wu, Y.C. Wu, A practical design of digital video watermarking in H.264/AVC for content authentication, Signal Process Image Commun. 26 (2011) 413–426. [11] X.L. Chen, H.M. Zhao, A novel video content authentication algorithm combined semi-fragile watermarking with compressive sensing, in: Second International Conference on Intelligent Systems Design and Engineering Application, Sanya, Hainan, 2012. [12] D.W. Xu, R.D. Wang, J.C. Wang, A novel watermarking scheme for H.264/AVC video authentication, Signal Process Image Commun. 26 (2011) 267–279. [13] S.Y. Chen, H. Leung, Chaotic watermarking for video authentication in surveillance applications, IEEE Trans. Circuits Syst. Video Technol. 18 (2008) 704–709. [14] J. Fridrich, M. Goljan, Images with self-correcting capabilities, in: Proceedings of the IEEE International Conference on Image Processing, Kobe, Japan, 1999. [15] C.Y. Lin, S.F. Chang, SARI: self-authentication and -recover image watermarking system, in: Proceedings of the 9th ACM International Conference on Multimedia, Ottawa, Canada, 2001. [16] X. Zhu, A. Ho, P. Marziliano, A new semi-fragile image watermarking with robust tampering restoration using irregular sampling, Signal Process Image Commun. 22 (2007) 515–528. [17] M.U. Celik, G. Sharma, A.M. Tekalp, E. Saber, Video authentication with self recovery, in: Proceedings of Security and Watermarking of Multimedia Contents IV, vol. 4314, 2002, pp. 531–541. [18] A.M. Hassan, A. Al-Hamadi, Y.M.Y. Hasan, M.A.A. Wahab, B. Michaelis, Secure Block-based video authentication with localization and self-recovery, Proceedings of World Academy of Science, Eng. Technol. (2009) 69–74. [19] X.P. Zhang, S.Z. Wang, Fragile watermarking with error-free restoration capability, IEEE Trans. Multimedia 10 (2008) 1490–1499. [20] X.P. Zhang, S.Z. Wang, Z.X. Qian, G.R. Feng, Reference sharing mechanism for watermark self-Embedding, IEEE Trans. Image Process. 20 (2011) 485–495. [21] J. Tian, Reversible data embedding using a difference expansion, IEEE Trans. Circuits Syst. Video Technol. 13 (2003) 890–896.
Please cite this article in press as: Y. Shi, et al., Object based dual watermarking for video authentication, Optik - Int. J. Light Electron Opt. (2013), http://dx.doi.org/10.1016/j.ijleo.2012.11.078