Robust blind image watermarking using crisscross inter-block prediction in the DCT domain

Accepted Manuscript Robust blind image watermarking using crisscross inter-block prediction in the DCT domain Ling-Yuan Hsu, Hwai-Tsu Hu PII: DOI: Reference:

S1047-3203(17)30070-6 http://dx.doi.org/10.1016/j.jvcir.2017.03.009 YJVCI 1979

To appear in:

J. Vis. Commun. Image R.

Received Date: Revised Date: Accepted Date:

5 April 2016 31 December 2016 3 March 2017

Please cite this article as: L-Y. Hsu, H-T. Hu, Robust blind image watermarking using crisscross inter-block prediction in the DCT domain, J. Vis. Commun. Image R. (2017), doi: http://dx.doi.org/10.1016/j.jvcir.2017.03.009

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Robust blind image watermarking using crisscross inter-block prediction in the DCT domain Ling-Yuan Hsua and Hwai-Tsu Hub a

Department of Information Management, St. Mary’s Junior College of Medicine, Nursing and Management, I-Lan, Taiwan b Department of Electronic Engineering, National I-Lan University, I-Lan, Taiwan

Abstract Watermarking has been proposed as a solution to the problem protecting copyrighted multimedia in networked environments. This paper presents a simple but effective blind watermarking scheme capable of satisfying requirements pertaining to imperceptibility as well as robustness, while maintaining a sufficient payload capacity. In the proposed scheme, partly sign‐altered mean modulation and mixed modulation are introduced to the crisscross discrete cosine transform (DCT)-based inter-block. Substituting a set of coefficients for a single coefficient enhances robustness against malign attacks. The inclusion of mixed modulation enables control over the parameters required to provide resistance against commonly encountered attacks while maintaining a high peak signal-to-noise ratio. Experiment results demonstrate that the proposed algorithm exceeds the performance of the seven other schemes in providing robust resistance to variety of attacks, particularly those associated with Gaussian noise and speckle noise.

Keywords: blind image watermarking, discrete cosine transform, partly sign‐altered, mixed modulation, crisscross inter-block prediction Figure-1

I. Introduction

The past few decades have seen an explosion in the use of digital multimedia. This is exemplified by the transformation of much of the world’s audio, image, and video archives into an electronic form, much of which has been stored as personal collections on networked information systems throughout the internet. The availability and convenience of software tools means that tampering with multimedia data has also become very easy. Numerous methods have been devised to deal with this issue, ranging from copyright protection to content authentication and steganography, wherein the hiding of data plays a crucial role. Digital watermarking is a data-hiding method used in authentication, copyright protection, and secretive communications. All digital image watermarking techniques must satisfy five requirements: imperceptivity, non-detectability, security, robustness, and capacity. Imperceptivity and non-detectability refer to the requirement that watermarks be invisible to the human eye in order to prevent their removal and ensure that non-designated algorithms be prevented from initiating watermark retrieval. Security refers to the need for systems to prevent the decryption of embedded information, whereas robustness is the ability of the system to maintain a valid watermark after geometrical or non-geometrical attacks. The capacity of a watermark largely determines the robustness of the system; however, excessive capacity can undermine imperceptibility. Thus, there is usually a tradeoff between the two. Blind digital watermarking generally refers to the spatial or transform domain. Various Watermarking techniques can be selected according to the requirements of the application. Many recent advances in the area of digital watermarking have been inspired by the manipulation of the domain transform of multimedia objects. Watermarks embedded in the transform domain are generally more robust and less perceptible; however, computational requirements tend to be higher than those in the spatial domain. The domains used for digital

Figure-2

watermarking include discrete cosine transform (DCT) [1-7], discrete Fourier transform (DFT) [8-11], discrete wavelet transform (DWT) [12-16], and singular value decomposition (SVD) [17-22]. Hybrid transform domains have also been employed for digital watermarking, in methods

such

as

DWT-DFT[23],

DWT-DCT[24,

25],

DWT-SVD[26,

27],

and

DWT-SVD-DCT[28, 29]. DCT has proven particularly effective with regard to energy compaction and the ability to incorporate characteristics of the human visual system (HVS) [30, 31]. DWT has been more widely used in digital image watermarking, due to its excellent multi-resolution characteristics and spatial localization, which are similar to the theoretical models of the human visual system [22, 32-34]. Chen et al. presented a robust digital image watermarking scheme based on DWT (wmDWT), in which a watermark is embedded in the middle frequency sub-band to achieve high PSNR [34]. Guo et al. proposed a hybrid DWT and DCT (DWTDCT) method to enhance the robustness of the encrypted domain watermarking scheme [22] for use in plaintext as well as encrypted documents. In [32], Moghaddam and Nemati used the neighborhood concept to identify suitable locations for watermark insertion in the spatial domain using a modified imperialistic competition algorithm (MICA). Despite the fact that MICA is robust against a variety of attacks, such as noise, blurring, and sharpening, it lacks the ability to defend against JEPG attacks and requires considerable computational time to identify suitable locations for the insertion of a watermark. Al-Haj and Amer implemented robust watermarking in the region-of-noninterest (RONI) using a blind scheme in the DWT and SVD (DWTSVD) domain [27]. DWTSVD can help to guarantee the authenticity of watermarked images as well as the confidentiality of the owners, while enabling localized detection of regions that have been tampered with. Nonetheless, the resulting images tend to appear distorted and the robustness of the watermark is relatively low. A number of algorithms in the DCT domain have been proposed to facilitate the Figure-3

embedding of binary watermarks in images. The first type of technique involves changing the designated range of a given coefficient [3]. In [3], Patra et al. proposed a Chinese remainder theorem (CRT) for image digital watermarking in the DCT domain, in which CRT properties are used to determine whether the watermark bit is embedded on the DC or on the low frequency AC coefficients of the DCT. CRT has been shown to withstand JPEG compression while achieving high PSNR values under many types of attack; however, this method is largely ineffectual with regard to scaling, rotation, and filtering attacks. The second type is based on the absolute modulation of individual coefficients of an image. Chen and Wornell [35] proposed the most common of these methods, quantization index modulation (QIM), to map selected coefficients to a designated range according to binary values [36, 37]. In [38], Das et al. presented inter-block coefficient correlation (IBCC) in an attempt to explore the correlation between DCT coefficients drawn from adjacent DCT blocks. In that study, a pair of DCT coefficients is selected from two neighboring blocks, whereupon one of the coefficients is adjusted according to the other. This approach is particularly effective with regard to imperceptibility, due the fact that DCT coefficients are modified only moderately; however, the resulting watermark is not necessarily able to withstand severe attacks. The third type is based on the relative modulation of a pair of elements. In [5], Wang and Pearmain proposed a data hiding technique via the self-reference (SR) of specific coefficients in the DCT domain. The DC coefficients in a 3×3 grid of DCT blocks are formulated in a manner that renders an estimate of an AC coefficient in the central block. The magnitude of the corresponding AC coefficient is adaptively increased or decreased according to whether the watermark bit is a ‘0’ or ‘1’, based on relative modulation (RM). This type of reference scheme is highly efficient; however, it can result in considerable distortions in the image resulting from estimates of insufficient accuracy. In [7], Hsu and Hu proposed the blind watermarking of images using a backward-propagation neural network (BPNN) to exploit inter-block prediction and visibility thresholds in the DCT domain (referred to as wmNN). Figure-4

The relative modulation method is used to adjust the relationship between the intended coefficients and their BPNN predictions subject to just-noticeable difference (JND) in the formulation of blind image watermarks. Most of the methods described above are based on relative modulation (RM), which was developed to improve robustness against malign attacks in blind image watermarking. Embedding a watermark bit involves the selection and manipulation of a target coefficient and estimate (or prediction) with the aim of maintaining a desired inequality relationship. Quantization index modulation (QIM) provides good robustness while achieving image quality superior to that of RM [39]. Mixed modulation (MM) is a novel blind watermarking method intended to ameliorate the problem of quality degradation resulting from excessive coefficient modulation in the DCT domain [31]. The image quality obtained using MM is similar to that achieved using QIM; however, MM is generally superior in terms of bit error rate (BER) due to the use of commensurate embedding strength. In cases of image compression, the BER values obtained using MM are even smaller than those obtained using RM. MM is a combination of QIM and RM, which makes it possible to achieve adequate robustness against commonly encountered attacks without sacrificing PSNR. All of the above binary watermarking schemes are based on the adjustment of a single coefficient. In this paper, we propose a novel scheme in which a partly sign‐altered mean modulation (PSAM) DCT coefficient is implemented in conjunction with mixed modulation (MM) technique (referred to as PSAM-MM) for blind image watermarking. The proposed PSAM-MM substituting a set of coefficients for a single coefficient enhances robustness against malign attacks. Table 1 compares the specifications of the proposed PSAM-MM with wmNN with regard to six essential aspects: coefficient, technical complexity, relationship among coefficients, time complexity, modulation, and model. The proposed PSAM-MM is a fast and simple method, which provides notable advantages with regard to robustness and imperceptibility. Figure-5

The remaining sections of this paper are organized as follows. Section II outlines the application of partly sign‐altered mean modulation DCT coefficients to mixed modulation in the DCT domain. In Section III, we introduce the proposed blind image watermarking scheme in crisscross DCT-based inter-block prediction. In Section IV, we present our experimental results and the contributions of this paper are outlined in Section V.

II. Application of partly sign‐altered mean modulation (PSAM) DCT coefficients to mixed modulation (MM) The proposed partly sign‐altered mean modulation (PSAM) DCT coefficients are applied to mixed modulation (MM) and then incorporated into block-wise DCT-based scheme for blind image watermarking.

A.

Partly sign‐altered mean modulation DCT coefficients Most watermarking schemes involve the modification of a single coefficient to enable

the embedding of a watermark in the DCT domain. However, this can lead to focusing effects centered on the upper‐left pixel in each image block. We sought to overcome this situation by altering the sign of some selected DCT coefficients before taking the average. Here, the term “partly” refers to a set of coefficients that is used as a substitute for a single coefficient, the term “sign-altered” refers to a perturbation in the sign with the aim of dispersing the energy to all pixels in the DCT domain, and the term “mean modulation” refers to the process of modulating the mean value obtained from selected DCT coefficients. The proposed PSAM strategy provides robust resistance to attacks by providing multiple coefficients that would need to be altered in order to damage the embedded watermark. Following the application of a 2‐D DCT to an image block, the coefficients in the block can be represented as watermarking of PSAM value

C  i, j

88

. The

 is achieved by modulating the mean value gathered from Figure-6

selected DCT coefficients as



1 7 7 i, j i, j C  ,  i , j  1,0, 1 ; i  0,1,2,...,7; j  0,1,2,...,7;  p i 0 j 0

(1)



(2)

and



   i, j  i, j  0

where C i, j represents the (i, j )th DCT coefficient in the frequency domain.  i , j denotes the (i, j )th sign in the DCT block. It can be observed that  i , j  0 means that the (i, j )th coefficient was not selected,  i, j  1 means that the (i, j )th coefficient has been selected but positively sign-altered, and  i , j  1 means that the (i, j )th coefficient has been selected but negatively sign-altered. Situations in which the signs of  i , j in a DCT block are i, j not equal to zero are referred to as partly sign‐altered (PSA) and  1, 1 . The symbol

 denotes a PSA set, and p is the number of  i , j in  . The DCT coefficients C i, j of selected PSAs are modified in the final step of embedding. Figure 1 lists the AC coefficients recommended for use in PSAM.

B.

PSAM-MM RM has been used in the development of blind image watermarking schemes, wherein

the intended coefficient is adjusted relative to its predicted coefficient in the DCT domain. When the coefficient is closer to its predicted coefficient, the use of RM generally reduces distortion and enhances the robustness of the watermarked image. However, when the coefficient is far from its estimated coefficient, QIM provides noticeably better image quality than that obtained using RM. This situation has led to the development of MM scheme to obtain the advantages of RM as well as QIM. Thus, MM is able to achieve image quality on par with that of QIM while inheriting robust attributes from RM, which can help it to withstand common digital signal processing attacks. We also employed PSAM strategy as an Figure-7

alternative to a coefficient in order to prevent the focusing effect. The proposed MM comprising RM and QIM can be expressed in mathematic terms, as follows: If w = 1   min    , max  ,   max  ,   ,

(3)

  d       2   2     , if d        d         0.5  2   , if d     2      , otherwise  

(4)

elseif w = 0   max    , min  ,   max  ,   ,

(5)

  d       2  0.5  2   , if d        d        2     , if d     2     , otherwise  

(6)

end

  , if        ; ˆ     , otherwise. where



(7)

 represents the threshold separating RM and QIM, w denotes the watermark bit,

denotes the adjustment factor for embedding strength,  is a quantization step, and 

stands for the ground level in the upward direction. The symbols  and  respectively represent the upper bound and lower bound of RM. PSAM of

 , and

 is the PSAM,  is the predicted

d     denotes the gap between

 and  .

 

and  represent

the floor function and absolute function, respectively. The symbols  and  respectively Figure-8

denote the results of modulation on

 via RM and QIM. As shown in Figure 2, in

accordance with gap d, we assume that   is the demarcation between the two available strategies. In the event that gap d falls within   , then RM is activated. In this situation, the image that is watermarked using MM inherits rudimentary robust attributes from the RM. Eqs. (3) and (5) present the mathematic forms of RM, which make it possible to obtain an RM-estimated  for the intended value. In contrast, when gap d falls outside   , QIM is used to deal with PSAM values that present excessive drift. Eqs. (4) and (6) can be used to obtain a QIM-estimated  for the other intended value. Eqs. (4) and (6) can then be used with MM to achieve image quality comparable with that obtained using QIM. The final step of MM involves the use of Eq. (7) to identify which solution results in less alteration to the image; i.e., the result that is similar to the original image and provides higher PSNR. After obtaining the embed estimate value ˆ using Eq. (7), the coefficients in DCT are modified using the embed adjustment function in Eq. (8). Note that only selected DCT coefficients in the PSA set  are modified. C i , j  C i , j  ( ˆ   )  i , j , i  0,1,2,...,7 and j  0,1,2,...,7

(8)

The embedded watermark bit is extracted as follows:

 1,   0,  d    w    +0.5 %2,     d    +0.5 %2,    

if d '   and      if d '   and      if d   

,

(9)

if d    

where   represents a PSAM acquired from a watermarked image,   is a predicted PSAM of   , and d        denotes the gap between   and   . w is the extracted watermark. During watermark extraction using Eq. (9),

 is the only setting required. For

the inner field characterized by d '   , RM is used to determine whether   is greater than Figure-9

  . In the case where d ' falls in the outer field, QIM is used to determine whether the quantized integer is odd.

III. Crisscross inter-block prediction When embedding a watermark in each DCT block, PSAM-MM can be used to obtain a balance between robustness and imperceptibility. MM involves two main values: PSAM value

 and predicted PSAM value  . Previous schemes used heterogeneous methods to

obtain predictions of higher accuracy. In other words, this coefficient can be predicted using nearby coefficients at different locations. Figure 3 illustrates the application of the 1,1 heterogeneous prediction method in DCT. In Figure 3, the predicted coefficient Cx , y is

0,0 0,0 0,0 0,0 predicting from Cx 1, y 1 , Cx 1, y 1 , Cx 1, y 1 and Cx 1, y 1 [5]. However, the pixels in the

image are usually very similar. Generally, coefficients at the same locations as the nearby block are highly correlated with one another, which makes the homogeneous prediction method a reasonable approach. Figure 4 illustrates the application of the homogeneous 1,1 1,1 prediction method in DCT, in which the predicted coefficient Cx , y is predicted from Cx 1, y ,

1,1 1,1 C1,1 x , y 1 , C x , y 1 , and C x 1, y . However, the above method (i.e., homogeneous prediction) has

one particular shortcoming (Figure 4). When the watermark is embedded in the current DCT block, the reference value used in previous blocks is modified. This can lead to mutual interference and subsequent errors in watermark extraction. Two sets of PSA (i.e.,  ) crisscross inter-block predictions are required to overcome this deficiency and they must be entirely different to avoid mutual interference. This approach to prediction is used in PSAM-MM, wherein predicted value  is the mean PSAM value obtained from the twelve blocks in a 5  5 grid, as follows:

Figure-10

 x, y

( x  1, y  2), ( x  1, y  2), ( x  2, y  1), ( x, y  1),  1      x, y , ( x, y ')  ( x  2, y  1), ( x  1, y), ( x  1, y), ( x  2, y  1),  , 12 ( x, y  1), ( x  2, y  1), ( x  1, y  2), ( x  1, y  2)   

(10)

where  x , y represents the ( x, y)th PSAM of the DCT block in the image. This type of crisscross inter-block prediction can be used to obtain the PSAM predicted value  without the threat of mutual interference. Figure 5 illustrates the relationship between the central and surrounding blocks involved in PSAM-MM. Preserving the visual quality of embedded images requires that the selected PSAs be reliable. We employed a quantization table (QT) based on a model of human perception that takes into account sensitivity to frequency as well as contrast masking. The QT

Q88

provides a very good indication of the relative sensitivity of each DCT coefficient, which has led to its being adopted as the “default” in JPEG compression, as follows: Q88  Q i , j i  0,1,2,...,7;j  0,1,2,...,7 16 12  14  14 = 18   24  49   72

11 12 13 17 22 35 64 92

10 14 16 22 37 55 78 95

16 19 24 29 56 64 87 98

24 26 40 51 68 81 103 112

40 58 57 87 109 104 121 100

51 60 69 80 103 113 120 103

61  55  56   62  77   92  101  99 

(11)

QT is an optimum allocation of a given budget of bits based on the coefficient statistics; however, each element in the table was acquired from a series of psycho-visual experiments used to determine the visibility thresholds for DCT basis functions. In each QT, low frequency terms occur in the top left corner and high frequency terms are in the bottom right corner. In the case of image compression, smaller values in the QT correspond to greater energy in the reconstruction of transformed coefficients from their quantized values. Here we gather the two sets of PSA  a   0,2 , 1,1 , 2,0 , 2,2  and b   0,1 , 1,0 , 1,2 , 2,1

Figure-11

according to QT. The summation of quantized values in PSA set Q0,2  Q1,1  Q2,0 +Q2,2 ) and the summation of quantized values in PSA set

 a is 52 (i.e.,  b is 50 (i.e.,

Q0,1  Q1,0  Q1,2 +Q2,1 ). The results obtained using these two PSA sets (i.e., 52 and 50) are similar;

therefore, we designated these two PSA sets as “partial” participation. We selected the two PSA sets with the aim of achieving balance between high and low frequencies as well as in the vertical and horizontal directions in the DCT frequency domain. In determining the “sign-altered” PSA, the results of the DCT coefficients are also selected according to QT. The summation of quantized values of Q0,2 and Q1,1 is equal to Q 2,0 and Q2,2 (i.e., 14+12=10+16). Similarly, the summation of quantized values of

Q0,1 and Q1,2 is

equal to

Q1,0 and Q 2,1 (i.e., 12+13=11+14). According to the above description, it would be

reasonable to set the signs in the PSAs as follows:  0,2   1,1  +1 ,  2,0   2,2 =  1 ,

 0,1   1,2  +1 and  1,0   2,1 = 1 . The two sets of PSA used in this paper are presented in Figure 5. For the block locations ( x, y) and ((x  y)%2  0) , we use  a ; i.e., the 0,2 1,1 2,0 2,2 coefficients Cx, y , Cx, y , Cx, y and Cx, y are selected and modified for this location of the block.

0,1 1,0 1,2 2,1 Otherwise,  b is employed and the coefficients Cx, y , Cx, y , Cx, y and Cx, y are selected and

modified. The procedural flow of the proposed PSAM-MM watermarking scheme is presented in Figure 6 as a summary of the discussion in this section. For the embedding of the watermark, we follow the steps outlined in Figure 6. First, a host image is partitioned into non-overlapping blocks of size 8 × 8. Second, DCT is applied to each block separately. Third, determine which PSA set to be used according to the block location. Fourth, the PSAM value

 x , y and its predicted PSAM value  x , y are obtained using the PSAM strategy (Eqs. (1) and (10)) and the chosen PSA set  . Fifth, MM is activated to archive the embedded

Figure-12

estimate value ˆ x , y according to Eqs. (3)-(7). Sixth, using Eq. (8), the watermark is embedded into the coefficients corresponding to selected PSA set  . Finally, the inverse DCT is used to restore each individual block and then create the watermarked image by gathering the blocks together. The first four steps in the extraction of the watermark are similar to those used in embedding the watermark, wherein a host image is partitioned into non-overlapping blocks of size 8 × 8, to which is applied the DCT. We then identify which PSA set to use based on the location of the block. The PSAM strategy (Eqs. (1) and (10)) is then used to obtain the PSAM value   and its predicted value   in accordance with the selected PSA set  . Finally, the MM in the form of Eq. (9) in the watermarking process is used to extract the watermark. To enhance security, the watermark has been scrambled using chaotic mixtures prior to embedding. We applied the mixed skew tent map and Arnold transformation (MSA) to the encryption and decryption of the watermark image during embedding and extraction procedures. The MSA key comprises two parts: K1 and K2. K1 is a positive number, which refers to the number of iterations used in Arnold scrambling, whereas K2 is a real number used as the initial value s0  (0,1) in the skew tent map [7, 40].

IV. Performance evaluation As shown in Figure 7, the test materials included 12 different 512×512 8-bit grayscale images, namely “Cameraman”, “House”, “Jetplane”, “Lake”, “Lena”, “Livingroom”, “Mandril”, “Pappers”, “Pirate”, “Walkbridge”, “Womanblonde”, and “Womandarkhair” , which were selected from [41]. Figure 8 presents the watermarks and their scrambled versions. Two forms of binary logos (64×64) were used as watermarks: one with an equal number of “1s” and “0s” and one with an unequal number. As illustrated in Figure 8, not only the final

Figure-13

appearances but also the amounts of "1s" and "0s" in the scrambled outputs are completely different from the original ones after processed using the MSA method. As shown in Eqs. (3) -(9), the embedding and extraction of the watermark involves modifying a selected PSAM-MM subject to the controlling parameters

 ,  , ,  , 

quantization step  to enable the dynamic adjustment of parameters Discrete threshold

and  . Here we use

,  , 

and  .

 and ground level  are controlled by the upper bound  ,

whereupon sets     0.5 and     1.0 are applied in MM. In this manner, the six parameters were reduced to four with the parameters of the PSAM-MM set as follows:

  0.05 ,   18 ,   0.5 ,   0.9 , in accordance with [34]. The metrics used to measure the amount of image distortion introduced by watermark casting included peak signal-to-noise ratio (PSNR), mean structural similarity (mSSIM) and bit error rate (BER). The PSNR refers to numerical differences between images. mSSIM is a well-known quality metric used to measure similarity between two images, whereas BER is used to describe the number of incorrectly extracted watermarks divided by the total number of watermarks. The definitions of PSNR, mSSIM, and BER are as follows:     2 255   PSNR( I , I )  10  log10  1 P Q 2   P  Q  ( I p ,q  I p ,q )  p 1 q 1  

(12)

mSSIM( I , I ) 

1 R S  SSIM( Br ,s , Br ,s ) R  S r 1 s 1

(13)

BER(W ,W ) 

1 X Y  wx, y  wx, y X  Y x 1 y 1

(14)

where P  Q signifies the image size, R  S equals the total number of non‐overlapped blocks in each image and X  Y represents the size of the watermark image. The symbols

 

I  I x , y  and I  I x , y

respectively represent the original image and watermarked Figure-14

image. The function SSIM( Br ,s , Br ,s ) is used compute the degree of similarity between the original block Br , s and watermarked image block Br , s [5]. Symbols W  wx , y  and W   wx , y  respectively represent the original watermark and extracted watermark.

To evaluate the robustness of the proposed model in the DCT-based watermarking framework, we conducted experiments with a particular focus on image compression, noise corruption, filtering, geometrical correction and luminance adjustment. The consequences of applying sixteen different image attacks (i.e., a1~a19) are illustrated in Figure 9. a1. JPEG compression applied to the image with the quality factor (QF) selected from

20,

40, 60, 80 .

a2. JPEG2000 compression applied to the image with the compression ratio (CR) selected from

2,

4, 6, 8 .

a3. Gaussian noise (0.001) added to the image with the variance set as 0.001 of the full scale. a4. Salt and pepper (S&P) noise (1%) added to the image with 1% intensity. a5. Speckle noise (1%) added to the image with 1% intensity. a6. Median filter (3×3) a7. Gaussian lowpass filter (3×3) a8. 2D adaptive wiener filter (3×3) a9. Scaling correction (25%): image shrinking from 512×512 to 256×256 pixels with subsequent enlargement from 256×256 to 512×512 pixels a10. Image rotation I: counterclockwise by 45 and then 45 back to the original position a11. Image rotation II: counterclockwise by 32 and then 32 back to the original position a12. Cropping I: Removal of 25% of the image from the left side a13. Cropping II: Removal of 25% of the image from the top left corner a14. Cropping III: Removal of a box 78×111 in the middle of the image a15. Cropping IV: Cropped down to a box of 78×111 in the middle of the image a16. Brighten (+20): Increase in the value of each pixel by 20 Figure-15

a17. Darken (-20): Decrease in the value of each pixel by 20 a18. Spotlight I: Increased pixel values from center to border of image a19. Spotlight II: Increased pixel values from the top left corner of image This stage of the experiments deals with robustness. We conducted comparisons with seven other schemes: 1) Chinese remainder theorem (CRT) for image digital watermarking in the DCT domain proposed by Patra et al. [3], 2) self-reference (SR) proposed by Wang and Pearmain [5], 3) inter-block coefficient correlation (IBCC) proposed by Das et al. [38], 4) watermarking based on neural network prediction (wmNN) proposed by Hsu and Hu [7], 5) the robust blind image watermarking method proposed by Chen et al. [34], which uses DWT (wmDWT) in the middle frequency sub-band, 6) the hybrid DWT-DCT (DWTDCT) method proposed by Guo et al. [22], and 7) the insertion of robust watermarks in the DWT and SVD (DWTSVD) domain, as proposed by Al-Haj and Amer for use in the RONI of images [27]. CRT [3] is used to modify the DCT coefficient, based on the coefficients of the internal block. Three of the schemes (SR [5], IBCC [38] and wmNN [7]) modify the DCT coefficient based on cross-block coefficients. We selected these four schemes (CRT [3], SR [5], IBCC [38] and wmNN [7]) because they exploit the correlation among DCT blocks. The other three schemes (wmDWT [34], DWTDCT [22] and DWTSVD [27]) are based on DWT domain. As for the implementation of the CRT scheme [3], 38 and 107 served as the pair of co-prime numbers in the DC coefficient, and 38 and 55 were used for the pair of co-prime numbers in the other AC coefficients. In our implementation of the SR scheme[5], watermark bits can be embedded in 1,1

only one position; i.e., C x , y . The reference threshold was specified as 8 and 5% of the absolute value of the selected AC coefficients. In the implementation of the IBCC [38], the size of the embedded watermark was just 64×63; therefore, BER calculations were based on 64×63 bits. In the implementation of the wmNN[7], the minimum clearance was set at 12 and the offset value was set at 32. In our implementation of wmDWT[34], the quantization size was set to 90. In our implementation of the DWTDCT [22], we set the embedding strength to 6. In our Figure-16

implementation of DWTSVD [27], the watermark was embedded in the LH sub-band. The remaining parameter settings were adopted in accordance with those found in the literature [3, 5, 7, 22, 27, 34 , 38]. Table 2 presents the average and standard deviation of PSNRs obtained by applying these eight schemes to 12 images. We can see that the values in Table 2(a) are very similar to those in Table 2(b). As shown in Table 2, the DWTSVD achieved PSNR of 62 dB, and the proposed PSAM-MM achieved an average PSNR level close to 40 dB, both of which are superior to the other six schemes. Figure 10 presents the eight watermarking results when applied to the image “Lena”, wherein the CRT image (c) presents noticeable blocking, and the wmDWT image (f) presents noticeable noise. The edges of the SR, IBCC, and DWTDCT images present slight atomization, whereas the images obtained using DWTSVD, wmNN, and the proposed PSAM-MM retain strong similarity to the original image. Figure 11 presents the images of “Lena” with the corresponding BERs as well as the watermarks extracted after various attacks. Clearly, the DWTSVD scheme was the least effective in all resisting attacks except S&P noise (a4), cropping I (a12) cropping II (a13), cropping III (a14), brighten (a16) and darken (a17) attacks, resulting in BER values exceeding 40%. The DWTDCT scheme was unable to provide effective defense against any of the attacks, especially in JPEG compression (a1), JPEG2000 compression (a2/CF 8), noise (a3, a4 and a5), median filter (a6), adaptive wiener filter (a8), scaling correction (a9) and rotation correction (a10) attacks., wmDWT proved more effective than DWTDCT in resisting compression attacks (a1 and a2). The CRT scheme resulted in BER values exceeding 30% after JPEG compression (a1/QF 20), median filter (a6), adaptive wiener filter (a8), scaling correction (a9) and rotation correction (a10 and a11) attacks. The SR scheme outperformed CRT. All BER values obtained using the SR scheme are below 15%, with the exceptions of JPEG compression (a1/QF 20) and speckle noise attack (a5). The IBCC scheme achieved results similar to those of the SR scheme; however, it was vulnerable to the S&P noise attack (a4). wmNN was resistant to all attacks except S&P noise (a4) and speckle noise (a5). The Figure-17

proposed PSAM-MM scheme enabled the extraction of the entire watermarked image with a BER value of less than 15%, except in the case of cropping attack IV (a15). Table 3 lists the average BERs obtained from 12 watermarked images using the eight schemes. Clearly, there is little difference between the values in Table 3(a) and those in Table 3(b). As expected, the BER values indicate that the watermarks can be extracted without error using any of the schemes except DWTSVD. As shown in Table 3, the average BER values for cropping attacks (a11 and a12) are higher than 12%; however, it does not appear that the proposed scheme is to be blamed. Note that 25% cropping refers to a thorough obliteration of one-fourth of an image. Any BER value near 12.5% should be considered reasonably acceptable, due to the fact that it is simply the result of pure guessing. This kind of situation also occurs in a cropping attack III (a14), wherein the image loses 3.42% of its data. Considering that in an image with a total size of 512×512 cropped down to a box of 78×111, there are 10 (= 78 / 8 with  denoting the ceiling function) × 14 (= 111 / 8 ) blocks with incorrect data. Consequently, the BER value is expected to approach one-half of 3.42%. In contrast, the cropping attack IV (a15) obliterates a large portion of the watermarked image. Only 9×13 (= 78 / 8 × 111 / 8 ) blocks maintain correct information. As a result, the BER values are close to 49%; i.e., (100% - 2.86%) / 2. The fact that the proposed PSAM-MM must refer to other blocks further weakens the resistance against cropping attacks (a12-a15), especially when the missing part is relatively large. By contrast, no significant difference has been found between a10 and a11 in both DCT- and DWT-based schemes in the case of rotation attacks. In addition to cropping attacks (a12-a15), our concern involves the performance results achieved by the examined schemes under other forms of attack. As shown in Table 3, the DWTSVD scheme rendered the worst performance in resisting all attacks except brighten (a16) and darken (a17), which resulted in BER values exceeding 40%. The DWTDCT scheme Figure-18

outperformed CRT in resisting JPEG2000 (a2/CR2 and a2/CR4), filtering (a6, a7 and a8), scaling correction (a9), rotation correction (a10 and a11), brighten (a16), darken (a17), spotlight (a18 and a19) attacks. CRT withstood attacks a12 and a13 as well as the other schemes but performed poorly against the other types of attacks. The SR scheme outperformed the other methods in resisting the S&P noise attack (a4). IBCC outperformed CRT and SR in a1/QF20, a1/QF80, a2/CR4, a2, a3, a5 a6, a7, a8, a9, and a17. The wmDWT resisting for compression attacks (a1 and a2) has a superior performance than CRT, SR, IBCC. wmNN outperformed CRT, SR and IBCC in resisting attacks based on JPEG compression (a1/QF20) and rotation correction I (a10). DCT makes it possible for PSAM-MM to provide greater resistance to these types of attack than is possible using non-DCT methods, such as wmDWT, DWTDCT, and DWTSVD. As PSAM involves multiple DCT coefficients, the proposed PSAM-MM thus holds the potential to manifest superior robustness. In our design, MM is used to achieve a suitable balance between image quality and robustness, wherein the selective tuning of parameters can help to maintain image quality higher than that of DCT-based methods, such as CRT, SR, IBCC and wmNN. The combination of PSAM with MM in crisscross inter-block prediction makes it possible for PSAM-MM to significantly outperform CRT, SR, and IBCC, especially in providing resistance against the following attacks: JEPG (a1/QF20), JPEG2000 (a2/CR8), Gaussian noise (a3), speckle noise (a5), median filter (a6), adaptive wiener filter (a8) and scaling correction (a9). In general, the BER values have been improved by 5% to 20%. Moreover, the proposed PSAM-MM provides defensive capabilities superior to those of wmNN in dealing with all types of attacks except a1/QF20, rotation correction (a10 and a11), and spotlight I (a18). As shown in Table 3, PSAM-MM appears impervious to JPEG2000 compression attacks at a compression ratio of 2 (a2/CR2), resulting in a BER of 0. The proposed PSAM-MM achieved the lowest BER in many of the tests, while wmNN proved effective in restoring the watermark; however, the CRT, SR, IBCC, DWTDCT and DWTSVD methods resulted in obvious deterioration. Figure-19

To provide further support for this argument, we employed 374 images embedded with our logo (Figure 8(a)) drawn from the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) dataset [42], the results of which are presented in Tables 4 and 5. Clearly, there is little difference between the data in Tables 2 and 4 as well as between Tables 3 and 5. However, unlike the case shown in Table 3, the proposed PSAM-MM cannot guarantee the BER to be zero for the a2/CR2 attack as in Table 5. Such a discrepancy originates from the fact that few images selected from the ILSVRC dataset possess blocks filled with the minimum or maximum values allowed by grey images. For example, in an image displaying pure white text on a pure black background, the pixel values in many image blocks are either 0 or 255. For a pure black or white image block, there is no room for embedding an arbitrary binary bit because the pixel values are not allowed to exceed above 255 or below 0. To overcome this drawback, we may examine the DC level of each block in advance and then modify the DC value if necessary. Although the DC modification enables the expansion of the available range for binary embedding via PSAM-MM, the resulting PSNR will inevitably be decreased. Despite of such a drawback, the proposed PSAM-MM still outperforms other methods in defense against JPEG2000 compression attacks (a2). Overall, the proposed PSAM-MM scheme demonstrates excellent performance under most types of attack.

V. Conclusions This paper reports a novel scheme for watermarking images in which partly sign‐altered mean modulation (PSAM) DCT coefficients is combined with mixed modulation (MM) for use in crisscross inter-block prediction. PSAM embeds a set of coefficients (rather than a single coefficient) to enhance robustness against attacks. MM is used to achieve balance between image quality and robustness, wherein selective tuning of the parameters can help to maintain image quality (PSNR) of 40 dB or higher with low BER. Crisscross inter-block prediction makes it possible for the proposed PSAM-MM to prevent mutual interference Figure-20

while enhancing the reliability of PSAM values. By maintaining a careful balance between robustness and imperceptivity, the watermarks embedded using PSAM-MM provide a high degree of protection against a wide range of attacks, exceeding the performance of the CRT, SR, IBCC, wmNN, wmDWT, DWTDCT and DWTSVD schemes.

Acknowledgments This research work was supported by the Ministry of Science and Technology, Taiwan, ROC under Grants MOST 105-2221-E-562-003 and MOST 104-2221-E-197-023.

References

[1]

M. Barni, F. Bartolini, V. Cappellini, and A. Piva, "A DCT-domain system for robust image watermarking," Signal Processing, vol. 66, pp. 357-372, 1998.

[2]

S. D. Lin and C. F. Chen, "A robust DCT-based watermarking for copyright protection," IEEE Trans. Consumer Electronics, vol. 46, pp. 415-421, 2000. J. C. Patra, J. E. Phua, and C. Bornand, "A novel DCT domain CRT-based watermarking scheme for image authentication surviving JPEG compression," Digital Signal Processing, vol. 20, pp. 1597-1611, 2010.

[3]

[4]

[5] [6] [7]

[8] [9]

S. D. Lin, S. C. Shie, and J. Y. Guo, "Improving the robustness of DCT-based image watermarking against JPEG compression," Computer Standards & Interfaces, vol. 32, pp. 54-60, 2010. Y. Wang and A. Pearmain, "Blind image data hiding based on self reference," Pattern Recognition Letters, vol. 25, pp. 1681-1689, 2004. W. C. Chu, "DCT-based image watermarking using subsampling," IEEE Trans. Multimedia, vol. 5, pp. 34-38, 2003. L.-Y. Hsu and H.-T. Hu, "Blind image watermarking via exploitation of inter-block prediction and visibility threshold in DCT domain," Journal of Visual Communication and Image Representation, vol. 32, pp. 130-143, 10// 2015. P. Tao and A. M. Eskicioglu, "An Adaptive Method for Image Recovery in the DFT Domain," Journal of Multimedia, pp. 36-45, 2006. T. K. Tsui, X. P. Zhang, and D. Androutsos, "Color Image Watermarking Using Multidimensional Fourier Transforms," IEEE Trans. Information Forensics and Security, vol. 3, pp. 16-28, 2008. Figure-21

[10] [11] [12]

[13] [14]

[15] [16] [17] [18]

[19]

[20]

[21]

[22]

[23]

V. Solachidis and L. Pitas, "Circularly symmetric watermark embedding in 2-D DFT domain," IEEE Trans. Image Processing, vol. 10, pp. 1741-1753, 2001. J. Lang and Z.-g. Zhang, "Blind digital watermarking method in the fractional Fourier transform domain," Optics and Lasers in Engineering, vol. 53, pp. 112-121, 2// 2014. Y. Wang, J. F. Doherty, and R. E. Van Dyck, "A wavelet-based watermarking algorithm for ownership verification of digital images," IEEE Trans. Image Processing, vol. 11, pp. 77-88, 2002. Z. H. Wei, P. Qin, and Y. Q. Fu, "Perceptual digital watermark of images using wavelet transform," IEEE Trans. Consumer Electronics, vol. 44, pp. 1267-1272, 1998. M. S. Hsieh, D. C. Tseng, and Y. H. Huang, "Hiding digital watermarks using multiresolution wavelet transform," IEEE Trans. Industrial Electronics, vol. 48, pp. 875-882, 2001. G. Zhang, S. Wang, and Q. Wen, "An adaptive block-based blind watermarking algorithm," presented at the 7th Int. Conf. on Signal Processing (ICSP) 2004. N. Liu, H. Li, H. Dai, D. Guo, and D. Chen, "Robust blind image watermarking based on chaotic mixtures," Nonlinear Dynamics, vol. 80, pp. 1329-1355, 2015. C. C. Chang, P. Y. Tsai, and C. C. Lin, "SVD-based digital image watermarking scheme," Pattern Recognition Letters, vol. 26, pp. 1577-1586, 2005. K. L. Chung, W. N. Yang, Y. H. Huang, S. T. Wu, and Y. C. Hsu, "On SVD-based watermarking algorithm," Applied Mathematics and Computation, vol. 188, pp. 54-57, 2007. Q. Su, Y. Niu, Y. Zhao, S. Pang, and X. Liu, "A dual color images watermarking scheme based on the optimized compensation of singular value decomposition," AEU - International Journal of Electronics and Communications, vol. 67, pp. 652-664, 2013. P. Bao and X. Ma, "Image adaptive watermarking using wavelet domain singular value decomposition," IEEE Trans. Circuits and Systems for Video Technology, vol. 15, pp. 96-102, 2005. C. C. Chang, Y. S. Hu, and C. C. Lin, "A Digital Watermarking Scheme Based on Singular Value Decomposition," in Combinatorics, Algorithms, Probabilistic and Experimental Methodologies. vol. 4614, B. Chen, M. Paterson, and G. Zhang, Eds., ed: Springer Berlin Heidelberg, 2007, pp. 82-93. J. Guo, P. Zheng, and J. Huang, "Secure watermarking scheme against watermark attacks in the encrypted domain," Journal of Visual Communication and Image Representation, vol. 30, pp. 125-135, 7// 2015. X. Kang, J. Huang, Y. Q. Shi, and Y. Lin, "A DWT-DFT composite watermarking scheme robust to both affine transform and JPEG compression," IEEE Trans. Circuits and Systems for Video Technology, vol. 13, pp. 776-786, 2003. Figure-22

[24]

H.-T. Hu and L.-Y. Hsu, "Collective blind image watermarking in DWT-DCT domain with adaptive embedding strength governed by quality metrics," Multimedia Tools and Applications, pp. 1-20, 2016.

[25]

N. Wang, Y. Wang, and X. Li, "A Novel Robust Watermarking Algorithm based on DWT and DCT,"

[26]

[27] [28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

presented at the Int. Conf. on Computational Intelligence and Security, 2009. M. Zhang, Q. Zhang, and C. Zhou, "Robust Digital Image Watermarking in DWT-SVD Domain," in Artificial Intelligence and Computational Intelligence: Third International Conference, AICI 2011, Taiyuan, China, September 24-25, 2011, Proceedings, Part II, H. Deng, D. Miao, J. Lei, and F. L. Wang, Eds., ed Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 75-84. A. Al-Haj and A. Amer, "Secured Telemedicine Using Region-Based Watermarking with Tamper Localization," Journal of Digital Imaging, vol. 27, pp. 737-750, 2014. H.-T. Hu and L.-Y. Hsu, "Exploring DWT–SVD–DCT feature parameters for robust multiple watermarking against JPEG and JPEG2000 compression," Computers & Electrical Engineering, vol. 41, pp. 52-63, 1// 2015. S. Murty and P. R. Kumar, "A robust digital image watermarking scheme using hybrid DWT-DCT-SVD technique," International Journal of Computer Science and Network Security, vol. 10, pp. 185-192, 2010. J. R. Hernandez, M. Amado, and F. Perez-Gonzalez, "DCT-domain watermarking techniques for still images: detector performance analysis and a new structure," IEEE Trans. Image Processing, vol. 9, pp. 55-68, 2000. C. Agarwal, A. Mishra, and A. Sharma, "Gray-scale image watermarking using GA-BPN hybrid network," Journal of Visual Communication and Image Representation, vol. 24, pp. 1135-1146, 10// 2013. M. E. Moghaddam and N. Nemati, "A robust color image watermarking technique using modified Imperialist Competitive Algorithm," Forensic Science International, vol. 233, pp. 193-200, 12/10/ 2013. T.-S. Nguyen, C.-C. Chang, and X.-Q. Yang, "A reversible image authentication scheme based on fragile watermarking in discrete wavelet transform domain," AEU International Journal of Electronics and Communications, vol. 70, pp. 1055-1061, 8// 2016. S.-T. Chen, H.-N. Huang, W.-M. Kung, and C.-Y. Hsu, "Optimization-based image watermarking with integrated quantization embedding in the wavelet-domain," Multimedia Tools and Applications, vol. 75, pp. 5493-5511, 2016. B. Chen and G. W. Wornell, "Quantization Index Modulation: A Class of Provably Good Methods for Digital Watermarking and Information Embedding," IEEE Trans. Information Theory, vol. 47, p. 1423, 2001. J. P. Boyer, P. Duhamel, and J. Blanc-Talon, "Scalar DC-QIM for Semifragile Figure-23

[37]

[38]

[39]

[40] [41] [42]

Authentication," IEEE Trans. Information Forensics and Security, vol. 3, pp. 776-782, 2008. N. K. Kalantari and S. M. Ahadi, "A Logarithmic Quantization Index Modulation for Perceptually Better Data Hiding," IEEE Trans. Image Processing, vol. 19, pp. 1504-1517, 2010. C. Das, S. Panigrahi, V. K. Sharma, and K. K. Mahapatra, "A novel blind robust image watermarking in DCT domain using inter-block coefficient correlation," AEU International Journal of Electronics and Communications, vol. 68, pp. 244-253, 2014. A. Phadikar, S. P. Maity, and M. Mandal, "Novel wavelet-based QIM data hiding technique for tamper detection and correction of digital images," Journal of Visual Communication and Image Representation, vol. 23, pp. 454-466, 4// 2012. S. S. Jamal, T. Shah, and I. Hussain, "An efficient scheme for digital watermarking using chaotic map," Nonlinear Dynamics, vol. 73, pp. 1469-1474, 2013. SIPI Image Database Available: http://sipi.usc.edu/database/ O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, et al., "ImageNet Large Scale Visual Recognition Challenge," International Journal of Computer Vision, vol. 115, pp. 211-252, 2015.

Figure-24

Figures:

(a) (b) Figure 1. Illustration of the AC coefficients suggested for PSAM (a) DCT coefficients (b) recommended AC associated with gray area that can be +1 or -1.

Figure-25

QIM

Transitive zone

RM

Transitive zone

QIM

Figure 2. Illustration of mixed modulation technique

Figure-26

Figure 3. Heterogeneous estimate method in DCT.

Figure-27

Figure 4. Homogeneous estimate method in DCT

Figure-28

(a)

(b) Figure 5. Illustration of the relationship between central and surrounding blocks used in PSAM-MM (a)  a : ((x  y)%2  0) and (c) b : ((x  y)%2  1) .

Figure-29

Watermark Embedding

Watermark Extraction

Host image

Watermarked image

Block partition

Block partition

DCT

DCT

(x + y)%2=0

(x + y)%2=0

Yes

Yes

No

a

b

b

a

Watermark

PSAM

PSAM MSA

No

MM

MM

Binary embedding No

End of Blocks

Inverse DCT No

Yes

End of Blocks

MSA

Yes Watermarked image

Watermark

Figure 6. Embedding and extraction procedures used in the proposed watermarking scheme.

Figure-30

Figure 7. Test images collected from USC‐SIPI image database.

Figure-31

(a)

MSA

(c)

(b)

(d)

Figure 8. (a) Our binary image logo (64×64), (b) (a)’s scrambled version (64×64), (c) Cameramen binary image logo (64×64), and (c)’s scrambled version (64×64).

Figure-32

(a1/QF 20)

(a2/CF 8)

(a3)

(a4)

(a5)

(a6)

(a7)

(a8)

(a9)

(a10)

(a11)

(al2)

(a13)

(a14)

(a15)

(al6)

(a17) (a18) (a19) Figure 9. The consequences of applying sixteen different image attacks. (i.e., a1~a19)

Figure-33

(a)

(b)

(c)

(d)

(e)

(f)

(g) (h) (i) Figure 10. (a) The original “Lena” images, and the version watermarked using (b) CRT (37.48 dB), (c) SR (36.84 dB), (d) IBCC (36.01 dB), (e) wmNN (40.52 dB), (f) wmDWT (39.00 dB), (g) DWTDCT (38.98 dB), (h) DWTSVD (62.69 dB) and (i) the proposed PSAM-MM (39.56 dB).

Figure-34

a1/ a1/ a1/ a1/ QF 20 QF 40 QF 60 QF 80

a2/ CF 2

a2/ CF 4

a2/ CF 6

a2/ CF 8

a3

a4

a5

a6

a7

a8

a9

a10

a11

a12

a13

a14

a15

a16

a17

a18

a19

35.65

15.70

7.86

3.71

3.00

5.84

15.09

21.51

18.48

16.99

23.58

38.48

21.48

39.53

35.77

43.36

43.58

12.48

12.74

1.64

49.68

16.77

17.41

21.46

5.98

34.57

2.37

0.00

0.00

0.00

0.00

2.78

7.32

13.11

12.06

19.95

11.57

1.78

9.96

8.35

13.87

14.03

12.45

13.01

2.17

49.71

0.05

0.12

1.61

0.05

27.80

5.66

0.37

0.05

0.05

0.05

0.40

1.98

10.71

17.91

18.65

8.83

1.14

7.14

6.70

13.62

13.69

12.38

12.23

1.89

46.08

0.10

0.03

1.54

0.10

1.95

0.20

0.00

0.00

0.00

0.00

0.12

1.66

9.72

15.45

20.80

4.13

0.07

2.95

2.44

7.74

7.69

12.67

13.09

1.83

49.68

0.00

0.00

1.29

0.22

11.30

8.42

1.73

0.00

0.00

0.00

1.54

7.98

15.55

27.34

34.38

32.28

1.15

38.79

31.47

39.21

43.41

12.48

12.74

1.66

49.56

0.00

0.00

2.05

0.32

48.29

43.31

34.03

16.65

0.00

1.59

13.33

21.73

29.69

30.18

34.84

23.32

5.66

22.00

17.36

29.88

29.93

12.94

12.50

1.81

49.71

0.10

0.07

2.00

0.17

51.03

51.64

52.81

51.29

48.10

50.44

49.17

51.76

50.29

23.83

51.05

48.32

48.73

50.78

49.73

51.10

49.41

12.48

12.74

1.61

49.61

0.00

0.00

48.54

45.87

13.21

0.02

0.00

0.00

0.00

0.00

0.00

0.00

2.17

11.65

11.04

3.05

0.07

1.32

1.83

12.23

12.31

13.23

12.31

1.81

49.07

0.00

0.00

1.54

0.05

(a) BER

(b) BER

(c) BER

(d) BER

(e) BER

(f) BER

(g) BER

(h) BER

Figure 11. Resulting “Lena” images with BERs under various attacks along with watermarks extracted using various schemes. (a) CRT, (b) SR, (c) IBCC, (d) wmNN, (e) wmDWT, (f) DWTDCT, (g) DWTSVD, (h) the proposed PSAM-MM.

35/41

Tables: Table 1. Specifications of wmNN and PSAM-MM. wmNN PSAM-MM Embed in single coefficient. Embed in multiple coefficients. It is complicated, because it uses the It is simple, because it uses mean 2. Technical complexity BPNN. operation. 3. Coefficients relationship Heterogeneous Homogeneous. 4. Time complexity High time complexity. Low time complexity. 5. Modulation RM MM 6. Model NN model is needed. Do not need any model. 1. Coefficient

36/41

Table 2. Average PSNRs and mSSIMs under various attacks: the eight schemes embedded with (a) our logo and (b) cameramen logo (the twelve well-known images shown in Figure 7).

(a)

(b)

mean PSNR standard deviation mean mSSIM standard deviation mean PSNR standard deviation mean mSSIM standard deviation

CRT 37.70 0.19 0.948 0.019 37.76 0.18 0.949 0.018

SR 37.02 2.27 0.973 0.004 37.09 2.18 0.973 0.004

IBCC wmNN wmDWT DWTDCT DWTSVD PSAM-MM 36.05 38.71 39.43 39.12 62.82 39.49 1.44 1.77 0.00 2.90 0.42 0.24 0.962 0.971 0.963 0.974 1.000 0.962 0.006 0.007 0.013 0.006 0.000 0.012 36.00 38.87 39.29 38.14 62.72 39.53 1.54 1.74 0.37 2.90 0.13 0.26 0.962 0.972 0.957 0.974 1.000 0.962 0.006 0.007 0.019 0.006 0.000 0.011

37/41

Table 3. Statistical BER results obtained from the eight methods while under threat from various attacks embedded with (a) our logo and (b) Cameramen logo (twelve well known images shown in Figure 7).

(a)

(b)

Attack type None a1/QF20 a1/QF40 a1/QF60 a1/QF80 a2/CR2 a2/CR4 a2/CR6 a2/CR8 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19 a1/QF20 a1/QF40 a1/QF60 a1/QF80 a2/CR2 a2/CR4 a2/CR6 a2/CR8 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19

CRT 0.00 34.62 16.82 8.68 4.16 3.41 8.22 16.03 23.56 20.17 16.12 24.42 37.28 22.20 38.36 34.17 41.58 42.23 12.48 12.74 1.66 49.69 18.16 18.99 20.96 6.46 33.86 16.36 8.90 4.25 3.34 8.11 15.83 23.38 19.90 16.31 24.88 37.18 21.74 37.77 33.76 41.11 41.05 13.04 12.72 1.73 48.68 18.31 18.91 20.80 6.39

SR 0.00 34.58 2.68 0.21 0.18 0.14 1.01 5.11 11.56 13.07 11.97 20.01 13.45 2.02 12.02 9.73 16.14 16.16 12.43 12.91 1.93 49.65 0.32 1.42 1.97 0.62 34.77 2.60 0.22 0.18 0.15 1.03 5.00 11.33 12.78 11.73 19.79 13.59 2.02 11.99 9.84 16.22 16.16 13.03 13.02 1.93 48.71 0.35 1.42 2.12 0.64

IBCC 0.00 25.63 5.43 0.40 0.17 0.14 0.49 2.53 6.72 10.67 17.56 19.10 11.82 1.40 9.65 8.35 16.99 17.09 12.38 12.19 1.77 46.20 0.34 1.25 2.14 0.66 25.55 5.48 0.40 0.16 0.14 0.45 2.51 6.94 10.68 17.11 19.07 12.02 1.46 9.83 8.38 17.01 17.14 11.76 12.23 1.68 47.22 0.31 1.28 2.12 0.60

wmNN 0.00 2.79 0.30 0.10 0.06 0.06 0.20 1.65 5.41 8.56 13.82 19.31 7.42 0.39 5.32 4.27 12.17 12.42 12.71 13.13 1.90 49.74 0.26 1.19 1.88 0.55 2.73 0.27 0.08 0.03 0.03 0.22 1.73 5.40 8.68 14.07 19.18 7.27 0.33 5.38 4.07 12.36 12.45 13.36 13.08 1.97 48.66 0.25 1.14 1.85 0.58

38/41

wmDWT DWTDCT DWTSVD PSAM-MM 0.00 0.00 1.07 0.00 15.87 46.25 50.72 11.31 8.00 41.51 50.09 0.02 1.67 33.47 50.28 0.01 0.04 17.83 50.04 0.01 0.01 0.17 49.44 0.00 0.32 6.58 50.15 0.01 3.72 17.75 50.24 0.12 10.40 26.32 50.41 1.03 15.45 30.21 50.00 1.80 26.29 29.86 24.34 12.08 34.62 34.69 50.35 12.83 29.35 26.89 49.90 6.28 4.09 6.94 51.41 0.09 40.09 25.66 50.28 3.78 30.29 21.20 50.41 3.11 38.89 31.91 50.04 15.91 43.29 29.95 50.11 16.04 12.48 13.14 12.99 12.82 12.74 12.65 13.29 12.63 2.71 1.78 1.72 1.95 49.60 49.61 49.68 49.17 0.31 0.35 1.64 0.17 1.40 1.57 3.57 1.18 2.52 2.37 48.00 1.97 0.66 0.75 46.45 0.49 14.27 46.74 49.64 11.42 7.48 41.68 49.79 0.01 1.56 33.64 50.35 0.01 0.04 17.58 50.24 0.01 0.01 0.17 49.35 0.00 0.34 6.62 49.96 0.01 3.95 17.80 50.22 0.14 11.02 26.07 49.98 1.12 15.28 30.17 50.08 1.98 25.90 29.59 23.47 12.14 33.75 34.82 49.84 12.65 34.74 26.65 48.85 6.26 5.73 6.84 50.29 0.08 46.70 25.55 49.64 3.86 36.27 21.08 50.28 3.15 42.01 31.39 49.93 16.02 41.87 29.33 49.99 16.15 13.04 12.60 13.04 12.73 12.72 13.06 12.72 12.74 1.75 1.71 1.70 2.01 48.54 48.82 48.63 49.06 0.37 0.35 0.56 0.19 1.64 1.55 2.47 1.08 2.67 2.39 48.11 2.03 0.78 0.78 46.41 0.46

Table 4. Average PSNRs and mSSIMs for eight schemes tested under various attacks (374 images drawn from ILSVRC dataset [42]). mean standard deviation mean mSSIM standard deviation PSNR

CRT SR IBCC wmNN wmDWT DWTDCT DWTSVD 37.83 38.06 36.59 38.83 39.70 38.80 63.52 0.25 3.51 2.81 2.04 0.82 4.39 1.06 0.946 0.975 0.962 0.967 0.955 0.976 1.000 0.023 0.006 0.008 0.012 0.031 0.009 0.000

39/41

PSAM-MM 39.58 0.39 0.960 0.014

Table 5. Statistical BER results obtained from the eight methods while under threat from various attacks (374 images drawn from ILSVRC dataset [42]). Attack type None a1/QF20 a1/QF40 a1/QF60 a1/QF80 a2/CR2 a2/CR4 a2/CR6 a2/CR8 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 a17 a18 a19

CRT 0.00 33.95 19.34 11.89 5.66 4.84 8.72 15.81 22.40 21.34 16.85 24.32 35.97 24.41 37.35 33.66 40.17 41.21 12.45 12.01 1.67 49.69 19.57 20.08 22.33 8.53

SR 0.00 36.45 3.24 1.23 0.42 0.49 1.63 5.93 11.12 13.59 12.09 18.72 12.25 1.88 10.85 8.74 15.12 15.12 12.44 12.78 1.92 49.64 2.32 2.53 3.20 1.20

IBCC 0.00 26.30 6.13 0.70 0.32 0.27 0.92 3.97 8.05 11.26 17.32 17.71 11.72 1.62 9.91 8.34 16.59 16.55 12.38 12.33 1.77 46.22 2.14 2.27 3.18 1.14

wmNN 0.00 3.11 0.58 0.56 0.26 0.43 1.03 3.50 6.99 9.16 14.14 17.72 7.27 0.45 5.61 4.08 11.98 11.97 12.70 13.01 1.89 49.75 2.24 2.40 3.20 1.11

40/41

wmDWT 0.07 16.89 9.65 2.36 0.95 0.86 1.94 6.52 12.70 15.89 26.10 30.89 25.15 5.59 40.53 31.32 38.73 42.96 12.54 12.81 1.66 49.60 2.79 3.10 4.09 1.78

DWTDCT 0.00 47.60 41.64 34.73 28.42 0.59 6.94 16.91 24.70 30.29 29.19 32.53 22.80 6.26 23.70 19.80 26.48 26.67 12.94 12.50 1.74 49.59 2.63 2.92 3.90 1.41

DWTSVD PSAM-MM 3.91 0.00 49.59 13.17 49.92 0.66 49.84 0.10 49.95 0.06 49.39 0.04 49.85 0.16 49.92 1.04 49.97 2.80 50.04 2.32 25.52 12.18 49.96 11.98 49.14 7.07 47.80 0.31 50.00 4.02 50.04 3.61 49.99 14.50 49.99 14.52 15.32 12.59 14.81 12.63 1.77 1.90 49.69 48.67 7.26 2.07 8.94 1.99 48.84 3.06 47.72 0.94

Highlights: 1. 2. 3. 4.

PSAM and MM are introduced to the crisscross discrete cosine transform (DCT)-based inter-block. PSAM embeds a set of coefficients (rather than a single coefficient) to enhance robustness against attacks. MM is used to achieve balance between image quality and robustness. Crisscross inter-block prediction makes it possible for the proposed PSAM-MM to prevent mutual interference.

41/41