- Email: [email protected]
Object watermarks for digital images and video
Image and Vision Computing 22 (2004) 583–595 www.elsevier.com/locate/imavis
Object watermarks for digital images and video Xiangwei Kong*, Yu Liu, Huajian Liu, Deli Yang Department of Electronic Engineering, School of Electronics & Information Engineering, Dalian University of Technology, Dalian 116023, China Received 10 May 2001; received in revised form 19 August 2003; accepted 15 September 2003
Abstract The growth of new image technologies has created a need for techniques that can be used for copyright protection of digital images and video. One approach for copyright protection is to introduce an invisible signal, known as a digital watermark, into an image or video sequence. With the development of MPEG-4, frame-based approach has been migrating to object-based approach. Therefore, object-based watermarking schemes are needed. In this article, we propose a novel blind object watermarking scheme for images and video using shape adaptive-discrete wavelet transform (SA-DWT). To make the watermark robust and perceptual invisible, we embed it in the weighting mean of the wavelet blocks using the quantisation visual model based on the human visual system. Watermark detection is accomplished without the original, unwatermarked object by using statistical detection technique. Experimental results demonstrate that the proposed watermarking scheme is perceptual invisible and robust against many attacks such as lossy image/video compression (e.g. JPEG, JPEG2000 and MPEG-4), scaling, adding noise, filtering, D/A and A/D conversion, etc. q 2003 Elsevier B.V. All rights reserved. Keywords: Copyright protection; Object watermark; Security; Shape adaptive-discrete wavelet transform
1. Introduction With the growth of multimedia systems in distributed environments, the research of multimedia security as well as multimedia copyright protection becomes an important issue. As a potential and effective way to solve this problem, digital watermarking becomes a very active research area of signal and information processing. A digital watermark is a signal that is embedded in a digital image or video sequence that allows one to establish ownership, identify a buyer, or provide some additional information about the digital content. Digital watermark can be classified into two types: invisible and visible watermark. This article focuses on invisible watermarks. In general, an invisible digital watermark technique must satisfy the following two properties [1 –3]. (1) The embedded watermark should be statistically and perceptually invisible. * Corresponding author. Tel.: þ 86-411-4708470; fax: þ 86-4114708116. E-mail addresses: [email protected] (X. Kong); yuvliu@yahoo. com.cn (Y. Liu). 0262-8856/$ - see front matter q 2003 Elsevier B.V. All rights reserved. doi:10.1016/j.imavis.2003.09.016
(2) The watermark must be difficult to remove. It should also be robust to common signal processing and geometric distortion, such as compression, adding noise and scaling. Many watermark algorithms have been proposed to address this issue of ownership identification [3 – 6]. Cox et al. [3] propose a DCT based spread spectrum watermarking technique. A pseudo-random sequence is embedded into the significant DCT coefficients and is retrieved by calculating the similarity function of the original watermark and extracted watermark. Su et al. [4] proposes a wavelet-based watermark algorithm. Based on the principle of multithreshold wavelet codec (MTWC), the method searches the significant wavelet coefficients to embed the watermark in order to increase the robustness. The embedding strength in each subband is determined by the threshold of the subband. Polilchuk and Zeng [5] propose two kinds of adaptive watermarking methods. One is based on discrete cosine transform (IA-DCT), the other is based on discrete wavelet transform (IA-W). The watermark is embedded according to the JND threshold. Kaewkamnerd et al. [6] propose a wavelet based adaptive watermarking scheme. The human visual system (HVS) is employed to
584
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
determine the weighting function Tðx; yÞ to control the watermark casting process. With the development of MPEG-4, this standard is very attractive for a large range of applications such as video editing, Internet, video distribution, wireless video communications. MPEG-4 standard enables the production of content, and is able to access and manipulate directly objects within a video sequence. Frame-based approach has been migrating to object-based approach. Therefore, object watermarking schemes are needed in the MPEG-4 environment. Several object watermarking methods have been proposed [7 – 9]. Wu et al. [7] propose a multiresolution object watermarking approach based on the 2D and 3D shape adaptive wavelet transforms. The advantage of the multiresolution watermarking method is its robust against image/video compression and computational saving. However, the main disadvantage is that original image/video object is required for watermark detection. Kim et al. [8] propose an object-based video watermarking method using the shape adaptive-discrete cosine transforms (SA-DCT). The SA-DCT method is superior to all other padding methods in terms of the robustness against the image deformations. But the watermark can be damaged by a wavelet-based image codec in the quantization stage. Therefore, this method limits their applications in the context of JPEG2000 and MPEG-4 due to the fact that the wavelet transform is playing an important role in JPEG2000 and MPEG-4. Piva et al. [9] propose an object watermarking system for MPEG-4 streams. Since this method applies the discrete wavelet transform (DWT) to the whole image and the watermark is embedded in the all wavelet coefficients belonging to the three detail bands at level 0, this may lead to loss of the watermark which is embedded in the region outside the object. Barni et al. proposed a method that consists in embedding the watermark in each video object. They modify some predefined pair of quantized DCT coefficients in the luminance blocks of pseudo-randomly selected MBs. The first step of the recovery scheme consists in summing
the differences of each pair; the value of the embedded bits is then given by the sign of the sum. In this article, we propose an approach to blind watermarking of objects of images and video based on SA-DWT [10]. Unlike most watermarking methods, the watermark is not embedded by modulating individual wavelet coefficient but by modulating the weighting mean of coefficients in the wavelet blocks. HVS is employed to achieve the best tradeoff between perceptual invisibility and robustness to signal processing. Watermark detection is accomplished without the original, unwatermarked object by using statistical detection technique. Experimental results demonstrate that the proposed watermarking scheme is perceptual invisible and robust against unintentional and intentional attacks such as lossy image/video compression (e.g. JPEG, JPEG2000 and MPEG-4), scaling, adding noise, filtering, D/A and A/D conversion, etc. In the following, we first briefly describe SA-DWT, then focus on the proposed object watermarking scheme.
2. Shape adaptive-discrete wavelet transform Given an arbitrarily shaped object with shape mask information, the SA-DWT transforms the samples in the arbitrarily shaped region into the same number of coefficients as in the subband domain, while keeping the spatial correlation, locality, and self-similarity across subbands. Fig. 1 illustrates the result of a two-level wavelet decomposition of an arbitrarily shaped object. In the SA-DWT decomposition, the shape mask is decomposed into a pyramid of subbands in the same way as the SA-DWT so that each subband has a corresponding shape mask associated with it to specify the locations of the coefficients in that subband. Fig. 2(a) shows the parent – child relation of SA-DWT tree descending from a coefficient in the subband LL3. As shown in the figure, there are two types of nodes in a tree: in-nodes and outnodes. The major task is to extend the conventional watermarking methods to the case with out-nodes. A simple way is to set all those out-nodes, values to zeros
Fig. 1. Multi-resolution decomposition of an arbitrarily shaped object.
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
585
Fig. 2. (a) Parent–child relation of wavelet trees in SA-DWT. (b) Reorganization of a wavelet tree into a wavelet block.
and then apply the conventional watermarking methods. However, this may lead to loss of the watermark. To avoid this problem, we reorganized the coefficients of each wavelet tree to form a wavelet block as shown in Fig. 2(b). In this way, the wavelet blocks are classified into two types: care blocks that have in-nodes and don’t care blocks that don’t have in-nodes. By doing so, all the don’t care blocks need not be watermarked and will be skipped, while these out-nodes values in care blocks will be set to zeros and later we will adopt a method to ensure that the watermark is not embedded in these out-nodes in care blocks.
If the magnitudes of all Ii ðkÞ’s are changed by V due to some distortions, then I^0 ðkÞ ¼
N21 X
Pi ðlIi ðkÞl þ OÞ
i¼1
¼
N21 X
Pi lIi ðkÞl þ
i¼1 |fflfflffl ffl{zfflfflfflffl}
S1
N21 X
i¼1 |ffl ffl{zfflffl}
S2
Since Pi is a binary random sequence with uniform distribution, S2 is approximately zero. Then the weighting mean becomes. I^0 ðkÞ ¼ S1 þ S2 <
3. Proposed watermarking scheme
ð2Þ Pi O
N21 X
^ Pi lIi ðkÞl ¼ IðkÞ
ð3Þ
i¼1
The block diagrams of watermark embedding and detection are shown in Fig. 3. 3.1. Watermark embedding Given a set of wavelet coefficients, it has been observed that the population mean has a smaller variance than that of individual coefficient. Thus, the watermark embedded in the weighting mean of the wavelet blocks is more robust than in the individual coefficient. The basic concept of the weighting mean is similar to that of Ref. [11], but is developed independently. The weighting mean of the wavelet block is defined as ^ ¼ IðkÞ
N21 X
Pi lIi ðkÞl
ð1Þ
i¼1
where Ii ðkÞ is the ith wavelet coefficient in the kth wavelet block, i ¼ 1; …; N 2 1 and N is the number of coefficients in the wavelet block. I0 ðkÞ denotes the DC coefficient in the kth wavelet block. Pi is a binary random sequence with uniform distribution, Pi [ { 2 1; 1}:
So the weighting mean has the advantage of preserving small variance when some distortions are encountered. The watermark W; consisting of a binary pseudo random sequence, WðkÞ [ { 2 1; 1}; is embedded by modifying the weighting mean of wavelet blocks in this way ^ þ aTðkÞWðkÞ I^0 ðkÞ ¼ IðkÞ
ð4Þ
^ where IðkÞ is the weighting mean of the kth wavelet block. a is a scaling factor. To adapt the watermark sequence to the local properties of the wavelet block, we use the quantisation model based on HVS [12] in the watermark system. The visual model takes into account the brightness sensitivity and texture sensitivity of the wavelet block to noise. The weighting function TðkÞ is defined as TðkÞ ¼ brightnessðkÞ·textureðkÞb where 8 > < brightnessðkÞ ¼ 3 þ 1 I ðkÞ 256 0 > : textureðkÞ ¼ Var{I ðkÞ : i ¼ 1; …; N 2 1} i
ð5Þ
586
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 3. Block diagrams for the proposed watermarking scheme. (a) Watermark embedding. (b) Watermark detection.
and b is set to 0.318 to control the degree of texture sensitivity. The eye is less sensitive to noise in the highly bright and the highly textured areas of the image. The function brightnessðkÞ estimates the local brightness based on the graylevel values of the low pass subband of the wavelet block and the function texture(k) is defined as the local variance of the wavelet block, except for the DC coefficient. ^ To update the weighting mean IðkÞ; each individual coefficient Ii ðkÞ in the kth wavelet block must be updated accordingly. Let D ¼ aTðkÞWðkÞ; the following rule can be applied to update Ii ðkÞ
lI ðkÞl > Di ¼ N21i D > > X > > > lIj ðkÞl > : j¼1
Proof. Let D ¼ aTðkÞWðkÞ: From Eq. (4), it is obtained: ^ þ D:Assume I^0 ðkÞ ¼ IðkÞ
Di ¼
I 0i ðkÞ ¼ Ii ðkÞ þ Pi signðIi ðkÞÞDi where 8 ( þ1; if x . 0 > > > signðxÞ ¼ > > > 21; if x , 0 > <
If the sign of Ii ðkÞ is changed after applying Eq. (6), then I 0i ðkÞ is set to zero. Di is the magnitude of update on Ii ðkÞ: If Ii ðkÞ is the out-node in the care wavelet block, then Di ¼ 0 due to the fact that the out-nodes values in care blocks have been set to zeros. So the watermark will not be embedded in these out-nodes in care blocks. The proof of the rule is presented as follows:
lIi ðkÞl N21 X
D;
lIj ðkÞl
j¼1
then I^0 ðkÞ ¼ ð6Þ
N21 X i¼1
¼
N21 X i¼1
Pi lIi ðkÞl þ
N21 X i¼1
Di ¼
N21 X
Pi ðlIi ðkÞl þ Pi Di Þ
i¼1
Pi signðIi ðkÞÞðIi ðkÞ þ Pi signðIi ðkÞÞDi Þ
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Let I 0i ðkÞ ¼ Ii ðkÞ þ Pi signðIi ðkÞÞDi ; and introduce the restraining condition: if signðI 0i ðkÞÞ – signIi ðkÞ; then I 0i ðkÞ ¼ 0: Therefore, it is obtained ^0
I ðkÞ ¼
N21 X
Pi signðI 0i ðkÞÞI 0i ðkÞ
¼
i¼1
N21 X
Thus, when Ii ðkÞ is updated to is also updated to I^0 ðkÞ:
3.3. Security
^ the weighting mean IðkÞ
3.2. Watermark detection Watermark detection is accomplished without the original, unwatermarked object by correlating the possibly watermarked weighting mean of the wavelet blocks I^0 and the watermark W as follows M21 X
c¼
YðkÞ
k¼0
pffiffiffi s M
pffiffiffi m M ¼ s
ð7Þ
where YðkÞ ¼ I^0 ðkÞWðkÞ; M is the length of the watermark sequence, m and s2 are the mean and the variance of YðkÞ; given, respectively, by M21 X
m¼
M21 X
YðkÞ
k¼0
M
; s2 ¼
c¼
^ IðkÞWðkÞ þ
k¼0
Some assumptions are made for the proposed scheme: The attacker does not have the original object as well as source codes of watermark embedding and detection. The information available to the public is only the watermarked object. Even though the attacker knows that the SA-DWT is used and understands the algorithm of watermark embedding and detection, the following information is kept to confidential to prevent the attacker from removing the watermark: (1) The number L of the wavelet transform level. (2) The wavelet filter (e.g. Haar, 9/7, 9/3, etc.). (3) The seed used to generate the random sequence with uniform distribution P: (4) The seed used to generate the watermark W: All data listed above can be used as a private key, and encrypted with the user ID so that they are unknown to the public.
ðYðkÞ 2 mÞ2
k¼0
M21
ð8Þ
If the object is watermarked with W 0 ; W – W 0 ; then the correlation c is given by M21 X
minimizes the total detection errors is Tc ¼ mc =2: In our experiments, the threshold Tc ¼ 4:
Pi lI 0i ðkÞl
i¼1
I 0i ðkÞ;
587
M21 X
aTðkÞW 0 ðkÞWðkÞ k¼0 pffiffiffi s M
ð9Þ
3.4. Comparison of the proposed method to other existing methods In this section we give a comparison of the proposed method to other existing methods. Some of the following information is already mentioned at different points in this article, but we have it summarized at this place. There are several advantages in our proposed method:
If the object is watermarked with W; the correlation c is M21 X
c¼
k¼0
^ IðkÞWðkÞ þ
M21 X
pk¼0 ffiffiffi s M
2
aTðkÞW ðkÞ ð10Þ
^ W; are independent and identically disAssume that I; tributed random variables and W has zero mean value. Under these assumptions, the mean value of c is 8 0; > > > > 0 > > < if W – W or No watermark M21 mc ¼ ð11Þ X > > a TðkÞ p ffiffiffi > > > EðaTðkÞÞ M > : pffiffiffi ; if W ¼ W 0 ø k¼0 s s M The value of the correlation sum c is then compared with a threshold Tc : The watermark W is considered to be present if c . Tc and absent if c , Tc : The threshold Tc that
(1) In order to avoid that the watermark is embedded in the region outside the object and then lead to the loss of the watermark, we adopt a don’t care wavelet block skipping method to ensure that the watermark is not embedded in the region outside the object. (2) Given a set of wavelet coefficients, it has been observed that the population mean has a smaller variance than that of individual coefficient. Thus, unlike most watermarking methods, the watermark is not embedded by modulating individual wavelet coefficient but by modulating the weighting mean of coefficients in the wavelet blocks. (3) HVS is employed to achieve the best tradeoff between perceptual invisibility and robustness to signal processing. (4) Watermark detection is accomplished without the original, unwatermarked object by using statistical detection technique.
588
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 4. (a) Original object ‘Akiyo’. (b) Watermarked object ‘Akiyo’ (PSNR ¼ 43.52 dB).
4. Experimental results We tested our scheme on a number of images and video, but only report results for ‘Akiyo’ and ‘News’ in detail. In our experiments, we fix a as a constant and choose a ¼ 2:0: In order to test the performance of the proposed watermarking scheme, 200 watermarks were randomly generated. 4.1. Experiments for image object
the response of the watermark detector to 200 randomly generated watermarks of which only one matches the watermark present in Fig. 4(b). The response to the correct watermark (i.e. number 100) is much higher than the responses to incorrect watermarks. To evaluate the robustness of our scheme against unintentional and intentional attacks, we test the watermarked object with JPEG, JPEG2000, scaling, adding noise, filtering, A/D and D/A conversion, and multiple watermarking attack.
For image object, we use the foreground object of the first frame (704 £ 480) of Akiyo sequence. The PSNR result between the original object and the watermarked object is 43.52 dB. As shown in Fig. 4, the watermark is perceptual invisible and the object with watermark appears visually identical to the object without watermark. In Fig. 5 the absolute difference between the original object and the watermarked one, magnified by a factor 32, is shown: it is evident that the watermark is mainly hidden into the regions of high texture and the edges and there is no watermark embedded in the region outside the object. Fig. 6 shows
JPEG is a widely used compression format and the watermark should be resistant to this distortion. As shown in Fig. 7, with the decreasing of the quality of the JPEG compressed object, the response of the watermark detector also decreases. We have found that the proposed watermark can survive even when quality factor is as low as 10% (see Fig. 8), although the object is visibly distorted (see Fig. 9).
Fig. 5. Absolute difference between the original object and the watermarked one, magnified by a factor 32.
Fig. 6. Detector response of the watermarked object ‘Akiyo’ for 200 randomly generated watermark.
4.1.1. JPEG compression distortion
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 7. Watermark detector response on the JPEG compressed and watermarked object ‘Akiyo’.
589
Fig. 10. JPEG2000 compression copy of the watermarked object ‘Akiyo’ with 65% quality.
4.1.2. JPEG2000 compression distortion JPEG2000 is the new generation compression standard, which is based on wavelet transform. In our experiments, we test the watermarked object with JPEG2000 compression using LuraWave Smart-Compress [13]. The detector response of the watermarked object Akiyo after the JPEG2000 compression with 75% quality is 9.20. Fig. 10 shows the object after the JPEG2000 compression with 65% quality, which results in very significant distortion. The response of the watermark detector in this case is 7.32, which is still above the threshold Tc (see Fig. 11).
Fig. 8. Detector response to a JPEG compression copy of the watermarked object ‘Akiyo’ with 10% quality.
4.1.3. Scaling Scaling is very easy to perform during the editing of digital images. So the watermarking technique must be robust to the scaling attack. We test our scheme in the case of scaling the watermarked object by 0.5 £ 0.5 using StirMark 3.1 [14,15]. The experiment results show the watermark can still be retrieved as shown in Fig. 12 with the detector response 8.67.
Fig. 9. JPEG compressed copy of the watermarked object ‘Akiyo’ with 10% quality.
Fig. 11. Detector response to a JPEG2000 compression copy of the watermarked object ‘Akiyo’ with 65% quality.
590
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 14. Watermarked object ‘Akiyo’ after 10% Gaussian noise adding. Fig. 12. Detector response to 0.5 £ 0.5 scaled copy of the watermarked object ‘Akiyo’.
4.1.4. Adding noise Noise is one of common distorts in the image processing and transmission. In the experiment, we add 20% uniform noise and 10% Gaussian noise into the watermarked object as shown in Figs. 13 and 14. The watermark can still be retrieved successfully, and the responses of the watermark detector are 8.93 and 9.39. 4.1.5. Filtering Filtering is also one of the common image processing. The watermarked object was filtered with 3 £ 3 blur filter (see Fig. 15) and 3 £ 3 median filter (see Fig. 16). The responses of the watermark detector are 9.02 and 9.24. These responses are well above the threshold Tc ; even if the objects appear degraded.
Fig. 15. Watermarked object ‘Akiyo’, blur filtered (3 £ 3).
4.1.6. D/A and A/D conversion D/A and A/D conversion is a very serious attack to the watermarked image. After D/A and A/D conversion, the image’s fidelity decreases and some usual-methods fail to retrieve the watermark. In our experiment,
the watermarked object was printed with a HP LaserJet 6L PCL printer, with a resolution of 300 dpi on standard paper; The object was then scanned using an Epson Perfection 1200 Photo scanner with a resolution of 300 dpi. The experimental result demonstrates our algorithm is robust to the D/A and A/D conversion and the response of the watermark detector is 10.01. The object after print and scan processing is shown in Fig. 17.
Fig. 13. Watermarked object ‘Akiyo’ after 20% uniform noise adding.
Fig. 16. Watermarked object ‘Akiyo’, median filtered (3 £ 3).
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
591
Fig. 17. Watermarked object ‘Akiyo’ after D/A and A/D conversion.
4.1.7. Multiple watermarking attack In some application scenarios, more than one watermark needs to be embedded in the object, and each watermark should be detected by the watermark detector. The original object is watermarked, then the watermarked object is again watermarked, and so on until the object with four different watermarks is obtained (see Fig. 18). The detector is well able to retrieve all the four watermarks embedded in the object, as shown in Fig. 19. 4.1.8. Experimental results for image object ‘Bream’ We also test our watermarking scheme on the image object ‘Bream’, we use the foreground object of the first frame (704 £ 480) of Bream sequence. The PSNR result between the original object and the watermarked object is 41.21 dB. As shown in Fig. 20, the watermark is perceptual invisible and the object with watermark appears visually identical to the object without watermark. Fig. 21 shows the response of the watermark detector to 200 randomly generated watermarks of which only one matches the watermark present in Fig. 20(b). To demonstrate the validity of the method, we also test the watermarked object Bream with JPEG, JPEG2000,
Fig. 18. The object ‘Akiyo’ with four different watermarks.
Fig. 19. Detector response of the multiple watermarked object ‘Akiyo’ (including the four specific watermarks).
scaling, adding noise, filtering, A/D and D/A conversion. Some test results about the objects Akiyo and Bream are shown in Table 1. 4.2. Experiments for video object In previous sections, we discussed watermark scheme for image object and gave some experimental result: next we will give a watermark scheme for video object which is based on previous image object watermark scheme. Video object watermark scheme can be classified as two cases: one is the watermark scheme for uncompressed video object sequence; another is the watermark scheme for compressed MPEG-4 video object stream. As far as the former is concerned, video object watermark casting and detecting scheme has no difference with image object watermark scheme because video object is composed of many image objects. So we do not discuss this scheme. We will discuss the latter in detail. The watermark scheme for compressed MPEG-4 video object stream is shown in Fig. 22. The procedure of watermark embedding in MPEG-4 video object stream is as following: (1) Decode the MPEG-4 video stream using MPEG-4 decoder to obtain two different objects and shape information. (2) In order to embed watermark into video objects, invoke Image Object Watermark Casting Scheme using each video object and shape information as input parameters. (3) After watermark embedding, encode the two video objects into the MPEG-4 video stream using MPEG-4 encoder. In this way, we obtained the watermarked MPEG-4 video stream.
592
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 20. (a) Original Bream object. (b) Watermarked Bream object (PSNR ¼ 41.2l dB).
Video sequences used for this experiment are 30 frames (704 £ 480) of News sequence. Each frame is composed of two different objects, the background (Video Object 0) and the reporters (Video Object 1), as shown in Fig. 23.
4.2.1. MPEG-4 compression
Fig. 21. Detector response of the watermarked object ‘Bream’ for 200 randomly generated watermark.
In order to detect whether a MPEG-4 video stream has a watermark, we first decode the MPEG-4 video stream to obtain two different objects and shape information, and then invoke Image Object Watermark Detecting Scheme using each video object and shape information as input parameters. In this way, we obtained detection result for each video object.
First, video sequences are compressed obtaining MPEG-4 coded video bitstream using MPEG-4 VM Version 12.1 [16,17], with a rate of 500 kbits per Video Object Layer (VOL). Then the watermarks are embedded in video objects, frame by frame, using MPEG-4 video object watermarking scheme, as shown in Fig. 22(a). The MPEG-4 video stream is next decompressed obtaining two different objects, where the watermark detection process, as shown in Fig. 22(b), is applied. The watermark detector responses of the decoded foreground objects of News sequence are 8.73 (VO 0) and 12.04 (VO 1), as shown in Fig. 24. The responses are well above the threshold Tc and indicate that our proposed watermarking scheme is robust to MPEG-4 compression.
Table 1 Watermark detector responses after attacks Detector responses
Akiyo
Bream
Detector responses
Akiyo
Bream
JPEG quality 10 JPEG quality 30 JPEG quality 60 JPEG2000 quality 65 JPEG2000 quality 75 JPEG2000 quality 85 Scaling 0.5 £ 0.5 Scaling 0.5 £ 0.6 Scaling 2.0 £ 3.0
5.31 8.78 11.35 7.32 9.20 13.09 8.67 9.25 14.11
5.82 15.12 20.97 5.74 17.11 22.36 11.68 13.61 23.76
Uniform noise 10% Uniform noise 20% Uniform noise 30% Gaussian noise 10% Gaussian noise 20% Blur filtering 3 £ 3 Median filtering 3 £ 3 Gaussian filtering D/A and A/D
11.69 8.93 5.41 9.39 5.50 9.02 9.24 9.59 10.01
24.15 18.81 15.13 20.92 13.01 13.35 9.92 15.90 14.21
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
593
Fig. 22. Block diagrams for Video Object Watermarking Scheme. (a) Watermark Embedding. (b) Watermark Detection.
4.2.2. Format conversion from MPEG-4 to MPEG-2 The watermarked MPEG-4 video bitstream is decompressed obtaining frames, and then these frames are compressed obtaining MPEG-2 coded video bitstream using MPEG-2 TM5 algorithm [18]. The MPEG-2 coded
video bitstream is next decompressed, and each frame is separated obtaining different objects, where the watermark detection process is applied. As shown in Fig. 25, the two watermarks embedded in the two objects are easily detected. The correct detection of the two objects indicates that
Fig. 23. (a) A frame of the video sequence ‘News’. (b) The Video Object 0 ‘background’. (c) The Video Object I ‘reporters’.
594
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 24. Watermark detection response relating to the Video Object 0 (a) and the Video Object 1 (b) after MPEG-4 compression, with a rate of 500 kbits per VOL.
Fig. 25. Watermark detection response relating to the Video Object 0 (a) and the Video Object 1 (b) after format conversion from MPEG-4 to MPEG-2.
the proposed scheme is robust to conversion from MPEG-4 to MPEG-2. 5. Conclusions In this article, a novel blind object watermarking scheme for images and video using SA-DWT has been proposed. To make the watermark robust and perceptual invisible, we embed it in the weighting mean of the wavelet blocks using the quantisation visual model based on HVS. The visual model takes into account the brightness sensitivity and texture sensitivity. Watermark detection is accomplished without the original, unwatermarked object by using statistical detection technique. Our proposed scheme can be also applied to watermark-based object retrieving, and indexing.
Acknowledgements This work was supported by The National High Technology Research and Development Program of China (2002AA145010).
References [1] V. Solachidis, I. Pitas, Circularly symmetric watermark embedding in 2D DFT domain, Proceeding of ICASSP’99 6 (1999) 3469–3472. [2] R.B. Wolfgang, C.I. Podichuk, E.J. Delp, Perceptual watermarks for digital images and video, Processings of the IEEE 87 (7) (1999) 1108–1126. [3] I.J. Cox, J. Kilian, T. Leighton, T. Shamoon, Secure spread spectrum watermarking for multimedia, IEEE Transactions on Image Processing 6 (1997) 1673–1687.
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595 [4] P. Su, C.J. Kuo, H.M. Wang, Blind digital watermarking for cartoon and map images, IS and T/SPIE Conference on Security and Watermarking of Multimedia Contents, San Jose, California, January, 1999, pp. 296–305. [5] C.I. Podichuk, W. Zeng, Image adaptive watermarking using visual models, IEEE Journal on Selected Areas in communication, Special Issue on Copyright and Privacy Protection 16 (1998) 525–538. [6] N. Kaewkamnerd, K.R. Rao, Wavelet based image adaptive watermarking scheme, Electronics Letters 36 (2000) 312–313. [7] X. Wu, W. Zhu, Z. Xiong, Y. Zhang, Object-based multiresolution watermarking of images and video, ISCAS’2000, Geneva, Switzerland, May 23–31, 2000, pp. 212 –215. [8] G.Y. Kim, J. Lee, C.S. Won, Object-based video watermarking, ICCE’99, June 22–24, 1999, pp. 100–101. [9] A. Piva, R. Caldelli, A.D. Rosa, A DWT-based object watermarking system for MPEG-4 video streams, Proceedings of ICIP’2000, Vancouver, Canada, September 2000, vol. III, 2000, pp. 5–8. [10] S. Li, W. Li, Shape-adaptive discrete wavelet transforms for arbitrarily shaped visual object coding, IEEE Transactions Circuits and Systems for Video Technology 10 (5) (2000) 725– 743.
595
[11] G. Yu, C. Lu, H. Liao, J. Sheu, Mean quantization blind watermarking for image authentication, Proceedings of ICIP’2000, Vancouver, Canada, September 2000, vol. III, 2000, pp. 706–709. [12] N. Jayant, J. Johnston, R. Safrankek, Signal compression based on models of human perception, Proceedings of the IEEE 81 (l0) (1993) 1385– 1421. [13] LuraTech, LuraWave SmartCompress, http://www.luratech.com, June 2000. [14] M.G. Kuhn, F.A.P. Petitcolas, Stirmark, http://www.cl.cam.ac.uk/ ~fapp2/watermarking/stirmark/, November 1997. [15] F.A.P. Petitcolas, R.J. Anderson, Evaluation of copyright marking systems, Proceedings of IEEE Multimedia Systems (ICMCS’99) Florence, Italy 1 (1999) 574 –579. [16] ISO/IEC JTC1/SC29/WGll Document N2502, Information Technology—Generic coding of Audio–Visual Objects, Part 2: Visual, ISO/IEC 14496-2, Final Draft of International Standard, November 1998. [17] ISO/IEC JTC1/SC29/WGll Document N2552, MPEG-4 Video Verification Model Version 12.1, December 1998. [18] ISO/IEC 13818-2, Generic Coding of Moving Pictures and Associated Audio, Recommendation H.262(MPEG-2), 1995.
Object watermarks for digital images and video Xiangwei Kong*, Yu Liu, Huajian Liu, Deli Yang Department of Electronic Engineering, School of Electronics & Information Engineering, Dalian University of Technology, Dalian 116023, China Received 10 May 2001; received in revised form 19 August 2003; accepted 15 September 2003
Abstract The growth of new image technologies has created a need for techniques that can be used for copyright protection of digital images and video. One approach for copyright protection is to introduce an invisible signal, known as a digital watermark, into an image or video sequence. With the development of MPEG-4, frame-based approach has been migrating to object-based approach. Therefore, object-based watermarking schemes are needed. In this article, we propose a novel blind object watermarking scheme for images and video using shape adaptive-discrete wavelet transform (SA-DWT). To make the watermark robust and perceptual invisible, we embed it in the weighting mean of the wavelet blocks using the quantisation visual model based on the human visual system. Watermark detection is accomplished without the original, unwatermarked object by using statistical detection technique. Experimental results demonstrate that the proposed watermarking scheme is perceptual invisible and robust against many attacks such as lossy image/video compression (e.g. JPEG, JPEG2000 and MPEG-4), scaling, adding noise, filtering, D/A and A/D conversion, etc. q 2003 Elsevier B.V. All rights reserved. Keywords: Copyright protection; Object watermark; Security; Shape adaptive-discrete wavelet transform
1. Introduction With the growth of multimedia systems in distributed environments, the research of multimedia security as well as multimedia copyright protection becomes an important issue. As a potential and effective way to solve this problem, digital watermarking becomes a very active research area of signal and information processing. A digital watermark is a signal that is embedded in a digital image or video sequence that allows one to establish ownership, identify a buyer, or provide some additional information about the digital content. Digital watermark can be classified into two types: invisible and visible watermark. This article focuses on invisible watermarks. In general, an invisible digital watermark technique must satisfy the following two properties [1 –3]. (1) The embedded watermark should be statistically and perceptually invisible. * Corresponding author. Tel.: þ 86-411-4708470; fax: þ 86-4114708116. E-mail addresses: [email protected] (X. Kong); yuvliu@yahoo. com.cn (Y. Liu). 0262-8856/$ - see front matter q 2003 Elsevier B.V. All rights reserved. doi:10.1016/j.imavis.2003.09.016
(2) The watermark must be difficult to remove. It should also be robust to common signal processing and geometric distortion, such as compression, adding noise and scaling. Many watermark algorithms have been proposed to address this issue of ownership identification [3 – 6]. Cox et al. [3] propose a DCT based spread spectrum watermarking technique. A pseudo-random sequence is embedded into the significant DCT coefficients and is retrieved by calculating the similarity function of the original watermark and extracted watermark. Su et al. [4] proposes a wavelet-based watermark algorithm. Based on the principle of multithreshold wavelet codec (MTWC), the method searches the significant wavelet coefficients to embed the watermark in order to increase the robustness. The embedding strength in each subband is determined by the threshold of the subband. Polilchuk and Zeng [5] propose two kinds of adaptive watermarking methods. One is based on discrete cosine transform (IA-DCT), the other is based on discrete wavelet transform (IA-W). The watermark is embedded according to the JND threshold. Kaewkamnerd et al. [6] propose a wavelet based adaptive watermarking scheme. The human visual system (HVS) is employed to
584
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
determine the weighting function Tðx; yÞ to control the watermark casting process. With the development of MPEG-4, this standard is very attractive for a large range of applications such as video editing, Internet, video distribution, wireless video communications. MPEG-4 standard enables the production of content, and is able to access and manipulate directly objects within a video sequence. Frame-based approach has been migrating to object-based approach. Therefore, object watermarking schemes are needed in the MPEG-4 environment. Several object watermarking methods have been proposed [7 – 9]. Wu et al. [7] propose a multiresolution object watermarking approach based on the 2D and 3D shape adaptive wavelet transforms. The advantage of the multiresolution watermarking method is its robust against image/video compression and computational saving. However, the main disadvantage is that original image/video object is required for watermark detection. Kim et al. [8] propose an object-based video watermarking method using the shape adaptive-discrete cosine transforms (SA-DCT). The SA-DCT method is superior to all other padding methods in terms of the robustness against the image deformations. But the watermark can be damaged by a wavelet-based image codec in the quantization stage. Therefore, this method limits their applications in the context of JPEG2000 and MPEG-4 due to the fact that the wavelet transform is playing an important role in JPEG2000 and MPEG-4. Piva et al. [9] propose an object watermarking system for MPEG-4 streams. Since this method applies the discrete wavelet transform (DWT) to the whole image and the watermark is embedded in the all wavelet coefficients belonging to the three detail bands at level 0, this may lead to loss of the watermark which is embedded in the region outside the object. Barni et al. proposed a method that consists in embedding the watermark in each video object. They modify some predefined pair of quantized DCT coefficients in the luminance blocks of pseudo-randomly selected MBs. The first step of the recovery scheme consists in summing
the differences of each pair; the value of the embedded bits is then given by the sign of the sum. In this article, we propose an approach to blind watermarking of objects of images and video based on SA-DWT [10]. Unlike most watermarking methods, the watermark is not embedded by modulating individual wavelet coefficient but by modulating the weighting mean of coefficients in the wavelet blocks. HVS is employed to achieve the best tradeoff between perceptual invisibility and robustness to signal processing. Watermark detection is accomplished without the original, unwatermarked object by using statistical detection technique. Experimental results demonstrate that the proposed watermarking scheme is perceptual invisible and robust against unintentional and intentional attacks such as lossy image/video compression (e.g. JPEG, JPEG2000 and MPEG-4), scaling, adding noise, filtering, D/A and A/D conversion, etc. In the following, we first briefly describe SA-DWT, then focus on the proposed object watermarking scheme.
2. Shape adaptive-discrete wavelet transform Given an arbitrarily shaped object with shape mask information, the SA-DWT transforms the samples in the arbitrarily shaped region into the same number of coefficients as in the subband domain, while keeping the spatial correlation, locality, and self-similarity across subbands. Fig. 1 illustrates the result of a two-level wavelet decomposition of an arbitrarily shaped object. In the SA-DWT decomposition, the shape mask is decomposed into a pyramid of subbands in the same way as the SA-DWT so that each subband has a corresponding shape mask associated with it to specify the locations of the coefficients in that subband. Fig. 2(a) shows the parent – child relation of SA-DWT tree descending from a coefficient in the subband LL3. As shown in the figure, there are two types of nodes in a tree: in-nodes and outnodes. The major task is to extend the conventional watermarking methods to the case with out-nodes. A simple way is to set all those out-nodes, values to zeros
Fig. 1. Multi-resolution decomposition of an arbitrarily shaped object.
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
585
Fig. 2. (a) Parent–child relation of wavelet trees in SA-DWT. (b) Reorganization of a wavelet tree into a wavelet block.
and then apply the conventional watermarking methods. However, this may lead to loss of the watermark. To avoid this problem, we reorganized the coefficients of each wavelet tree to form a wavelet block as shown in Fig. 2(b). In this way, the wavelet blocks are classified into two types: care blocks that have in-nodes and don’t care blocks that don’t have in-nodes. By doing so, all the don’t care blocks need not be watermarked and will be skipped, while these out-nodes values in care blocks will be set to zeros and later we will adopt a method to ensure that the watermark is not embedded in these out-nodes in care blocks.
If the magnitudes of all Ii ðkÞ’s are changed by V due to some distortions, then I^0 ðkÞ ¼
N21 X
Pi ðlIi ðkÞl þ OÞ
i¼1
¼
N21 X
Pi lIi ðkÞl þ
i¼1 |fflfflffl ffl{zfflfflfflffl}
S1
N21 X
i¼1 |ffl ffl{zfflffl}
S2
Since Pi is a binary random sequence with uniform distribution, S2 is approximately zero. Then the weighting mean becomes. I^0 ðkÞ ¼ S1 þ S2 <
3. Proposed watermarking scheme
ð2Þ Pi O
N21 X
^ Pi lIi ðkÞl ¼ IðkÞ
ð3Þ
i¼1
The block diagrams of watermark embedding and detection are shown in Fig. 3. 3.1. Watermark embedding Given a set of wavelet coefficients, it has been observed that the population mean has a smaller variance than that of individual coefficient. Thus, the watermark embedded in the weighting mean of the wavelet blocks is more robust than in the individual coefficient. The basic concept of the weighting mean is similar to that of Ref. [11], but is developed independently. The weighting mean of the wavelet block is defined as ^ ¼ IðkÞ
N21 X
Pi lIi ðkÞl
ð1Þ
i¼1
where Ii ðkÞ is the ith wavelet coefficient in the kth wavelet block, i ¼ 1; …; N 2 1 and N is the number of coefficients in the wavelet block. I0 ðkÞ denotes the DC coefficient in the kth wavelet block. Pi is a binary random sequence with uniform distribution, Pi [ { 2 1; 1}:
So the weighting mean has the advantage of preserving small variance when some distortions are encountered. The watermark W; consisting of a binary pseudo random sequence, WðkÞ [ { 2 1; 1}; is embedded by modifying the weighting mean of wavelet blocks in this way ^ þ aTðkÞWðkÞ I^0 ðkÞ ¼ IðkÞ
ð4Þ
^ where IðkÞ is the weighting mean of the kth wavelet block. a is a scaling factor. To adapt the watermark sequence to the local properties of the wavelet block, we use the quantisation model based on HVS [12] in the watermark system. The visual model takes into account the brightness sensitivity and texture sensitivity of the wavelet block to noise. The weighting function TðkÞ is defined as TðkÞ ¼ brightnessðkÞ·textureðkÞb where 8 > < brightnessðkÞ ¼ 3 þ 1 I ðkÞ 256 0 > : textureðkÞ ¼ Var{I ðkÞ : i ¼ 1; …; N 2 1} i
ð5Þ
586
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 3. Block diagrams for the proposed watermarking scheme. (a) Watermark embedding. (b) Watermark detection.
and b is set to 0.318 to control the degree of texture sensitivity. The eye is less sensitive to noise in the highly bright and the highly textured areas of the image. The function brightnessðkÞ estimates the local brightness based on the graylevel values of the low pass subband of the wavelet block and the function texture(k) is defined as the local variance of the wavelet block, except for the DC coefficient. ^ To update the weighting mean IðkÞ; each individual coefficient Ii ðkÞ in the kth wavelet block must be updated accordingly. Let D ¼ aTðkÞWðkÞ; the following rule can be applied to update Ii ðkÞ
lI ðkÞl > Di ¼ N21i D > > X > > > lIj ðkÞl > : j¼1
Proof. Let D ¼ aTðkÞWðkÞ: From Eq. (4), it is obtained: ^ þ D:Assume I^0 ðkÞ ¼ IðkÞ
Di ¼
I 0i ðkÞ ¼ Ii ðkÞ þ Pi signðIi ðkÞÞDi where 8 ( þ1; if x . 0 > > > signðxÞ ¼ > > > 21; if x , 0 > <
If the sign of Ii ðkÞ is changed after applying Eq. (6), then I 0i ðkÞ is set to zero. Di is the magnitude of update on Ii ðkÞ: If Ii ðkÞ is the out-node in the care wavelet block, then Di ¼ 0 due to the fact that the out-nodes values in care blocks have been set to zeros. So the watermark will not be embedded in these out-nodes in care blocks. The proof of the rule is presented as follows:
lIi ðkÞl N21 X
D;
lIj ðkÞl
j¼1
then I^0 ðkÞ ¼ ð6Þ
N21 X i¼1
¼
N21 X i¼1
Pi lIi ðkÞl þ
N21 X i¼1
Di ¼
N21 X
Pi ðlIi ðkÞl þ Pi Di Þ
i¼1
Pi signðIi ðkÞÞðIi ðkÞ þ Pi signðIi ðkÞÞDi Þ
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Let I 0i ðkÞ ¼ Ii ðkÞ þ Pi signðIi ðkÞÞDi ; and introduce the restraining condition: if signðI 0i ðkÞÞ – signIi ðkÞ; then I 0i ðkÞ ¼ 0: Therefore, it is obtained ^0
I ðkÞ ¼
N21 X
Pi signðI 0i ðkÞÞI 0i ðkÞ
¼
i¼1
N21 X
Thus, when Ii ðkÞ is updated to is also updated to I^0 ðkÞ:
3.3. Security
^ the weighting mean IðkÞ
3.2. Watermark detection Watermark detection is accomplished without the original, unwatermarked object by correlating the possibly watermarked weighting mean of the wavelet blocks I^0 and the watermark W as follows M21 X
c¼
YðkÞ
k¼0
pffiffiffi s M
pffiffiffi m M ¼ s
ð7Þ
where YðkÞ ¼ I^0 ðkÞWðkÞ; M is the length of the watermark sequence, m and s2 are the mean and the variance of YðkÞ; given, respectively, by M21 X
m¼
M21 X
YðkÞ
k¼0
M
; s2 ¼
c¼
^ IðkÞWðkÞ þ
k¼0
Some assumptions are made for the proposed scheme: The attacker does not have the original object as well as source codes of watermark embedding and detection. The information available to the public is only the watermarked object. Even though the attacker knows that the SA-DWT is used and understands the algorithm of watermark embedding and detection, the following information is kept to confidential to prevent the attacker from removing the watermark: (1) The number L of the wavelet transform level. (2) The wavelet filter (e.g. Haar, 9/7, 9/3, etc.). (3) The seed used to generate the random sequence with uniform distribution P: (4) The seed used to generate the watermark W: All data listed above can be used as a private key, and encrypted with the user ID so that they are unknown to the public.
ðYðkÞ 2 mÞ2
k¼0
M21
ð8Þ
If the object is watermarked with W 0 ; W – W 0 ; then the correlation c is given by M21 X
minimizes the total detection errors is Tc ¼ mc =2: In our experiments, the threshold Tc ¼ 4:
Pi lI 0i ðkÞl
i¼1
I 0i ðkÞ;
587
M21 X
aTðkÞW 0 ðkÞWðkÞ k¼0 pffiffiffi s M
ð9Þ
3.4. Comparison of the proposed method to other existing methods In this section we give a comparison of the proposed method to other existing methods. Some of the following information is already mentioned at different points in this article, but we have it summarized at this place. There are several advantages in our proposed method:
If the object is watermarked with W; the correlation c is M21 X
c¼
k¼0
^ IðkÞWðkÞ þ
M21 X
pk¼0 ffiffiffi s M
2
aTðkÞW ðkÞ ð10Þ
^ W; are independent and identically disAssume that I; tributed random variables and W has zero mean value. Under these assumptions, the mean value of c is 8 0; > > > > 0 > > < if W – W or No watermark M21 mc ¼ ð11Þ X > > a TðkÞ p ffiffiffi > > > EðaTðkÞÞ M > : pffiffiffi ; if W ¼ W 0 ø k¼0 s s M The value of the correlation sum c is then compared with a threshold Tc : The watermark W is considered to be present if c . Tc and absent if c , Tc : The threshold Tc that
(1) In order to avoid that the watermark is embedded in the region outside the object and then lead to the loss of the watermark, we adopt a don’t care wavelet block skipping method to ensure that the watermark is not embedded in the region outside the object. (2) Given a set of wavelet coefficients, it has been observed that the population mean has a smaller variance than that of individual coefficient. Thus, unlike most watermarking methods, the watermark is not embedded by modulating individual wavelet coefficient but by modulating the weighting mean of coefficients in the wavelet blocks. (3) HVS is employed to achieve the best tradeoff between perceptual invisibility and robustness to signal processing. (4) Watermark detection is accomplished without the original, unwatermarked object by using statistical detection technique.
588
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 4. (a) Original object ‘Akiyo’. (b) Watermarked object ‘Akiyo’ (PSNR ¼ 43.52 dB).
4. Experimental results We tested our scheme on a number of images and video, but only report results for ‘Akiyo’ and ‘News’ in detail. In our experiments, we fix a as a constant and choose a ¼ 2:0: In order to test the performance of the proposed watermarking scheme, 200 watermarks were randomly generated. 4.1. Experiments for image object
the response of the watermark detector to 200 randomly generated watermarks of which only one matches the watermark present in Fig. 4(b). The response to the correct watermark (i.e. number 100) is much higher than the responses to incorrect watermarks. To evaluate the robustness of our scheme against unintentional and intentional attacks, we test the watermarked object with JPEG, JPEG2000, scaling, adding noise, filtering, A/D and D/A conversion, and multiple watermarking attack.
For image object, we use the foreground object of the first frame (704 £ 480) of Akiyo sequence. The PSNR result between the original object and the watermarked object is 43.52 dB. As shown in Fig. 4, the watermark is perceptual invisible and the object with watermark appears visually identical to the object without watermark. In Fig. 5 the absolute difference between the original object and the watermarked one, magnified by a factor 32, is shown: it is evident that the watermark is mainly hidden into the regions of high texture and the edges and there is no watermark embedded in the region outside the object. Fig. 6 shows
JPEG is a widely used compression format and the watermark should be resistant to this distortion. As shown in Fig. 7, with the decreasing of the quality of the JPEG compressed object, the response of the watermark detector also decreases. We have found that the proposed watermark can survive even when quality factor is as low as 10% (see Fig. 8), although the object is visibly distorted (see Fig. 9).
Fig. 5. Absolute difference between the original object and the watermarked one, magnified by a factor 32.
Fig. 6. Detector response of the watermarked object ‘Akiyo’ for 200 randomly generated watermark.
4.1.1. JPEG compression distortion
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 7. Watermark detector response on the JPEG compressed and watermarked object ‘Akiyo’.
589
Fig. 10. JPEG2000 compression copy of the watermarked object ‘Akiyo’ with 65% quality.
4.1.2. JPEG2000 compression distortion JPEG2000 is the new generation compression standard, which is based on wavelet transform. In our experiments, we test the watermarked object with JPEG2000 compression using LuraWave Smart-Compress [13]. The detector response of the watermarked object Akiyo after the JPEG2000 compression with 75% quality is 9.20. Fig. 10 shows the object after the JPEG2000 compression with 65% quality, which results in very significant distortion. The response of the watermark detector in this case is 7.32, which is still above the threshold Tc (see Fig. 11).
Fig. 8. Detector response to a JPEG compression copy of the watermarked object ‘Akiyo’ with 10% quality.
4.1.3. Scaling Scaling is very easy to perform during the editing of digital images. So the watermarking technique must be robust to the scaling attack. We test our scheme in the case of scaling the watermarked object by 0.5 £ 0.5 using StirMark 3.1 [14,15]. The experiment results show the watermark can still be retrieved as shown in Fig. 12 with the detector response 8.67.
Fig. 9. JPEG compressed copy of the watermarked object ‘Akiyo’ with 10% quality.
Fig. 11. Detector response to a JPEG2000 compression copy of the watermarked object ‘Akiyo’ with 65% quality.
590
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 14. Watermarked object ‘Akiyo’ after 10% Gaussian noise adding. Fig. 12. Detector response to 0.5 £ 0.5 scaled copy of the watermarked object ‘Akiyo’.
4.1.4. Adding noise Noise is one of common distorts in the image processing and transmission. In the experiment, we add 20% uniform noise and 10% Gaussian noise into the watermarked object as shown in Figs. 13 and 14. The watermark can still be retrieved successfully, and the responses of the watermark detector are 8.93 and 9.39. 4.1.5. Filtering Filtering is also one of the common image processing. The watermarked object was filtered with 3 £ 3 blur filter (see Fig. 15) and 3 £ 3 median filter (see Fig. 16). The responses of the watermark detector are 9.02 and 9.24. These responses are well above the threshold Tc ; even if the objects appear degraded.
Fig. 15. Watermarked object ‘Akiyo’, blur filtered (3 £ 3).
4.1.6. D/A and A/D conversion D/A and A/D conversion is a very serious attack to the watermarked image. After D/A and A/D conversion, the image’s fidelity decreases and some usual-methods fail to retrieve the watermark. In our experiment,
the watermarked object was printed with a HP LaserJet 6L PCL printer, with a resolution of 300 dpi on standard paper; The object was then scanned using an Epson Perfection 1200 Photo scanner with a resolution of 300 dpi. The experimental result demonstrates our algorithm is robust to the D/A and A/D conversion and the response of the watermark detector is 10.01. The object after print and scan processing is shown in Fig. 17.
Fig. 13. Watermarked object ‘Akiyo’ after 20% uniform noise adding.
Fig. 16. Watermarked object ‘Akiyo’, median filtered (3 £ 3).
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
591
Fig. 17. Watermarked object ‘Akiyo’ after D/A and A/D conversion.
4.1.7. Multiple watermarking attack In some application scenarios, more than one watermark needs to be embedded in the object, and each watermark should be detected by the watermark detector. The original object is watermarked, then the watermarked object is again watermarked, and so on until the object with four different watermarks is obtained (see Fig. 18). The detector is well able to retrieve all the four watermarks embedded in the object, as shown in Fig. 19. 4.1.8. Experimental results for image object ‘Bream’ We also test our watermarking scheme on the image object ‘Bream’, we use the foreground object of the first frame (704 £ 480) of Bream sequence. The PSNR result between the original object and the watermarked object is 41.21 dB. As shown in Fig. 20, the watermark is perceptual invisible and the object with watermark appears visually identical to the object without watermark. Fig. 21 shows the response of the watermark detector to 200 randomly generated watermarks of which only one matches the watermark present in Fig. 20(b). To demonstrate the validity of the method, we also test the watermarked object Bream with JPEG, JPEG2000,
Fig. 18. The object ‘Akiyo’ with four different watermarks.
Fig. 19. Detector response of the multiple watermarked object ‘Akiyo’ (including the four specific watermarks).
scaling, adding noise, filtering, A/D and D/A conversion. Some test results about the objects Akiyo and Bream are shown in Table 1. 4.2. Experiments for video object In previous sections, we discussed watermark scheme for image object and gave some experimental result: next we will give a watermark scheme for video object which is based on previous image object watermark scheme. Video object watermark scheme can be classified as two cases: one is the watermark scheme for uncompressed video object sequence; another is the watermark scheme for compressed MPEG-4 video object stream. As far as the former is concerned, video object watermark casting and detecting scheme has no difference with image object watermark scheme because video object is composed of many image objects. So we do not discuss this scheme. We will discuss the latter in detail. The watermark scheme for compressed MPEG-4 video object stream is shown in Fig. 22. The procedure of watermark embedding in MPEG-4 video object stream is as following: (1) Decode the MPEG-4 video stream using MPEG-4 decoder to obtain two different objects and shape information. (2) In order to embed watermark into video objects, invoke Image Object Watermark Casting Scheme using each video object and shape information as input parameters. (3) After watermark embedding, encode the two video objects into the MPEG-4 video stream using MPEG-4 encoder. In this way, we obtained the watermarked MPEG-4 video stream.
592
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 20. (a) Original Bream object. (b) Watermarked Bream object (PSNR ¼ 41.2l dB).
Video sequences used for this experiment are 30 frames (704 £ 480) of News sequence. Each frame is composed of two different objects, the background (Video Object 0) and the reporters (Video Object 1), as shown in Fig. 23.
4.2.1. MPEG-4 compression
Fig. 21. Detector response of the watermarked object ‘Bream’ for 200 randomly generated watermark.
In order to detect whether a MPEG-4 video stream has a watermark, we first decode the MPEG-4 video stream to obtain two different objects and shape information, and then invoke Image Object Watermark Detecting Scheme using each video object and shape information as input parameters. In this way, we obtained detection result for each video object.
First, video sequences are compressed obtaining MPEG-4 coded video bitstream using MPEG-4 VM Version 12.1 [16,17], with a rate of 500 kbits per Video Object Layer (VOL). Then the watermarks are embedded in video objects, frame by frame, using MPEG-4 video object watermarking scheme, as shown in Fig. 22(a). The MPEG-4 video stream is next decompressed obtaining two different objects, where the watermark detection process, as shown in Fig. 22(b), is applied. The watermark detector responses of the decoded foreground objects of News sequence are 8.73 (VO 0) and 12.04 (VO 1), as shown in Fig. 24. The responses are well above the threshold Tc and indicate that our proposed watermarking scheme is robust to MPEG-4 compression.
Table 1 Watermark detector responses after attacks Detector responses
Akiyo
Bream
Detector responses
Akiyo
Bream
JPEG quality 10 JPEG quality 30 JPEG quality 60 JPEG2000 quality 65 JPEG2000 quality 75 JPEG2000 quality 85 Scaling 0.5 £ 0.5 Scaling 0.5 £ 0.6 Scaling 2.0 £ 3.0
5.31 8.78 11.35 7.32 9.20 13.09 8.67 9.25 14.11
5.82 15.12 20.97 5.74 17.11 22.36 11.68 13.61 23.76
Uniform noise 10% Uniform noise 20% Uniform noise 30% Gaussian noise 10% Gaussian noise 20% Blur filtering 3 £ 3 Median filtering 3 £ 3 Gaussian filtering D/A and A/D
11.69 8.93 5.41 9.39 5.50 9.02 9.24 9.59 10.01
24.15 18.81 15.13 20.92 13.01 13.35 9.92 15.90 14.21
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
593
Fig. 22. Block diagrams for Video Object Watermarking Scheme. (a) Watermark Embedding. (b) Watermark Detection.
4.2.2. Format conversion from MPEG-4 to MPEG-2 The watermarked MPEG-4 video bitstream is decompressed obtaining frames, and then these frames are compressed obtaining MPEG-2 coded video bitstream using MPEG-2 TM5 algorithm [18]. The MPEG-2 coded
video bitstream is next decompressed, and each frame is separated obtaining different objects, where the watermark detection process is applied. As shown in Fig. 25, the two watermarks embedded in the two objects are easily detected. The correct detection of the two objects indicates that
Fig. 23. (a) A frame of the video sequence ‘News’. (b) The Video Object 0 ‘background’. (c) The Video Object I ‘reporters’.
594
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595
Fig. 24. Watermark detection response relating to the Video Object 0 (a) and the Video Object 1 (b) after MPEG-4 compression, with a rate of 500 kbits per VOL.
Fig. 25. Watermark detection response relating to the Video Object 0 (a) and the Video Object 1 (b) after format conversion from MPEG-4 to MPEG-2.
the proposed scheme is robust to conversion from MPEG-4 to MPEG-2. 5. Conclusions In this article, a novel blind object watermarking scheme for images and video using SA-DWT has been proposed. To make the watermark robust and perceptual invisible, we embed it in the weighting mean of the wavelet blocks using the quantisation visual model based on HVS. The visual model takes into account the brightness sensitivity and texture sensitivity. Watermark detection is accomplished without the original, unwatermarked object by using statistical detection technique. Our proposed scheme can be also applied to watermark-based object retrieving, and indexing.
Acknowledgements This work was supported by The National High Technology Research and Development Program of China (2002AA145010).
References [1] V. Solachidis, I. Pitas, Circularly symmetric watermark embedding in 2D DFT domain, Proceeding of ICASSP’99 6 (1999) 3469–3472. [2] R.B. Wolfgang, C.I. Podichuk, E.J. Delp, Perceptual watermarks for digital images and video, Processings of the IEEE 87 (7) (1999) 1108–1126. [3] I.J. Cox, J. Kilian, T. Leighton, T. Shamoon, Secure spread spectrum watermarking for multimedia, IEEE Transactions on Image Processing 6 (1997) 1673–1687.
X. Kong et al. / Image and Vision Computing 22 (2004) 583–595 [4] P. Su, C.J. Kuo, H.M. Wang, Blind digital watermarking for cartoon and map images, IS and T/SPIE Conference on Security and Watermarking of Multimedia Contents, San Jose, California, January, 1999, pp. 296–305. [5] C.I. Podichuk, W. Zeng, Image adaptive watermarking using visual models, IEEE Journal on Selected Areas in communication, Special Issue on Copyright and Privacy Protection 16 (1998) 525–538. [6] N. Kaewkamnerd, K.R. Rao, Wavelet based image adaptive watermarking scheme, Electronics Letters 36 (2000) 312–313. [7] X. Wu, W. Zhu, Z. Xiong, Y. Zhang, Object-based multiresolution watermarking of images and video, ISCAS’2000, Geneva, Switzerland, May 23–31, 2000, pp. 212 –215. [8] G.Y. Kim, J. Lee, C.S. Won, Object-based video watermarking, ICCE’99, June 22–24, 1999, pp. 100–101. [9] A. Piva, R. Caldelli, A.D. Rosa, A DWT-based object watermarking system for MPEG-4 video streams, Proceedings of ICIP’2000, Vancouver, Canada, September 2000, vol. III, 2000, pp. 5–8. [10] S. Li, W. Li, Shape-adaptive discrete wavelet transforms for arbitrarily shaped visual object coding, IEEE Transactions Circuits and Systems for Video Technology 10 (5) (2000) 725– 743.
595
[11] G. Yu, C. Lu, H. Liao, J. Sheu, Mean quantization blind watermarking for image authentication, Proceedings of ICIP’2000, Vancouver, Canada, September 2000, vol. III, 2000, pp. 706–709. [12] N. Jayant, J. Johnston, R. Safrankek, Signal compression based on models of human perception, Proceedings of the IEEE 81 (l0) (1993) 1385– 1421. [13] LuraTech, LuraWave SmartCompress, http://www.luratech.com, June 2000. [14] M.G. Kuhn, F.A.P. Petitcolas, Stirmark, http://www.cl.cam.ac.uk/ ~fapp2/watermarking/stirmark/, November 1997. [15] F.A.P. Petitcolas, R.J. Anderson, Evaluation of copyright marking systems, Proceedings of IEEE Multimedia Systems (ICMCS’99) Florence, Italy 1 (1999) 574 –579. [16] ISO/IEC JTC1/SC29/WGll Document N2502, Information Technology—Generic coding of Audio–Visual Objects, Part 2: Visual, ISO/IEC 14496-2, Final Draft of International Standard, November 1998. [17] ISO/IEC JTC1/SC29/WGll Document N2552, MPEG-4 Video Verification Model Version 12.1, December 1998. [18] ISO/IEC 13818-2, Generic Coding of Moving Pictures and Associated Audio, Recommendation H.262(MPEG-2), 1995.