Online Sequential Extreme Learning Machine for watermarking in DWT domain

Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

Contents lists available at ScienceDirect

Neurocomputing journal homepage: www.elsevier.com/locate/neucom

Online Sequential Extreme Learning Machine for watermarking in DWT domain Ram Pal Singh a,n, Neelam Dabas b, Vikash Chaudhary c, Nagendra c a

Department of Computer Science, DDUC, University of Delhi, Delhi, India Department of Computer Science, University of Delhi, Delhi, India c Department of Computer Science, BNC, University of Delhi, Delhi, India b

art ic l e i nf o

a b s t r a c t

Article history: Received 21 September 2014 Received in revised form 16 March 2015 Accepted 17 March 2015

Protecting and securing an information of digital media is very crucial due to illegal reproduction and modification of media has become an acute problem for copyright protection now a day. A Discrete Wavelet Transform (DWT) domain based robust watermarking scheme with Extreme Learning Machine (ELM), Online Sequential Extreme Learning Machine (OSELM) and Weighted Extreme Learning Machine (WELM) have been implemented on different color images. The proposed scheme which combine DWT with ELM, OSELM and WELM machine learning methods and a watermark or a tag or a sequence is embedded as an ownership information. Experimental results demonstrate that the proposed watermarking scheme is imperceptible/transparent and robust against image processing and attacks such as blurring, cropping, JPEG, noise addition, rotation, scaling, scaling–cropping, and sharpening. Performance and efficacy of algorithms of watermarking scheme is determined by measuring Peak Signal to Noise Ratio (PSNR), Bit Error Rate (BER) and Similarity parameter SIMðX; X n Þ and calibrated results are compared with other existing machine learning methods. As a watermark detector, machine learning techniques are used to learn neighbors relationship among pixels in a natural image has high relevance to its neighbors, so this relationship can be predicted by its neighbors using machine learning methods and watermark image can be extracted and detected and thereby ownership can be verified. & 2015 Elsevier B.V. All rights reserved.

Keywords: BER ELM OSELM PSNR SIMðX; X n ) WELM

1. Introduction In recent years, there has been a rapid development of multimedia including images, audio and video, are reprinted, duplicated and easily redistributed over internet which has become a very serious problem to protect intellectual property right (IPR) of digital media. Implementation of image processing applications must be completed within required time constraints. One such application is digital watermarking of color image in which embedding and extraction of tag or sequence or watermark must complete in minimum time complexity [5–9]. Although there are many other mechanisms like cryptography used for protection of digital data but this is a weak method to decrypt. The robust and imperceptible digital watermark schemes were developed to remove the drawbacks of cryptography techniques. Embedding and extraction processes should be optimized without loss of visual quality of image. A number of soft computing learning methods used to develop robust and imperceptible watermarking techniques [1–3]. In recent years,

n

Corresponding author. E-mail address: [email protected] (R.P. Singh).

digital watermarking has received considerable attention for finding unauthorized use of digital media. In digital watermarking, a watermark or a trademark or a sequence is embedded into the image for copyright protection and embedded watermark can be extracted from the media in order to prove ownership. The number of color images are used for digital watermarking either in spatial domain or in frequency domain. In spatial domain, an intensity value of pixel is modified but watermarking in this domain is not robust while in frequency domain the coefficients of image are modulated by adding additional information and scheme becomes more imperceptible. There are number of methods based on frequency domain [2–11] used for digital watermarking for colored images in which watermark logo embedded into blue channel as human vision system (HSV) [42] as it is insensitive to blue channel. In image authentication techniques, where visually recognizable pattern is embedded as watermark in low frequency sub-band in DWT, gives a trade off between imperceptibility and robustness. A digital watermarking scheme must have following requirements (i) imperceptible or transparency (ii) difficult to extract without affecting quality of an image (iii) should have robustness against image processing, conventional and geometrical attacks. Therefore, developing a computational algorithm which exhibits these requirements is not an easy

http://dx.doi.org/10.1016/j.neucom.2015.03.115 0925-2312/& 2015 Elsevier B.V. All rights reserved.

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i

R.P. Singh et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

2

job. A number of information hiding schemes have been used and reported in the literature [13]. Recently, computational intelligencetechniques have been widely used and neural networks and support vector machine (SVMs) [14,15] have been dominant. However, these techniques have numbers of unavoidable drawbacks such as (i) slow learning speed due to increase of local minima's (ii) poor computational capability (iii) particularly standard variant of SVM training involves a quadratic programming problem, thereby computational complexity is unusually intense. SVM is used as regression problem for watermarking scheme of the image [16,18] for watermark embedding and extraction. The number of machine learning methods are used for digital watermarking of image in spatial as well as frequency domains. Most of the drawbacks in neural networks based back propagation and support vector machine, ELM based learning algorithm tries to address them. We have explored the possibility of ELM based methods for information hiding in natural images where every pixel has high relevance to its neighbors, it can be predicted by its neighbors pixel relationship [16] using ELM, OSELM and WELM and this pixels relevance help in extraction of watermark. The relationship among pixels can be memorized by the training process and it was found that this relationship found intact even though the natural images are subjected to some attacks such as image processing, blurring, cropping, JPEG compression, rotation, noise addition, scaling, scaling–cropping, and sharpening. So this feature can be used for robust digital watermark embedding and extraction combined with machine learning techniques used as relationship is intact despite of number of attacks and using this relationship, watermark can be extracted by predicting this relationship. Rest of the paper is organized as follows. We briefly give theories about ELM, OSELM and Weighted ELM in Section 2. Watermarking schemes are described in Section 3. Experimental results are discussed in Section 4. Finally, concluded in Section 5.

2. Brief theories of learning machines 2.1. ELM Over last decades, batch learning algorithms for machine learning have wide range of application in different kind of research areas, starting from pattern recognition [19,27], text classification [30], time series analysis [31,32] to watermarking and information hiding [16,29], etc. Huang et al. [20–26] proposed new machine learning algorithm, Extreme Learning Machine for single hidden layer feed forward neural networks (SLFNs). In this algorithm, input layer weights need not to be tuned iteratively and to be generated randomly, however, the output weights are determined analytically using least-squares method. This algorithm has fast learning speed and high learning accuracy with good generalization ability. ELM is a batch learning type of algorithm having single hidden layer feed forward neural networks (SLFNs). The nature of ELM has been investigated by using the interpolation and universal approximation capabilities [27]. Given N arbitrary distinct data samples ðxi ; t i ÞN i , where xi ¼ ½xi1 ; …; xin  and t i ¼ ½t i1 ; …; t im . The output function of SLFNs with L^ number of hidden nodes can approximate N input samples with zero error then βi, ai and bi hold such that f L ðxÞ ¼

L^ X i¼1

βi gi ðxÞ ¼

L^ X

βi Gðai ; bi ; xÞ;

ai A Rn ; bi A R; βi A Rm

ð1Þ

i¼1

where, gi, denotes the output function Gðai ; bi ; xÞ of ith hidden node and ai ¼ ½ai1 ; …; ain  is the weight vector connecting ith hidden neuron and input neuron and bi is the threshold of ith hidden neuron, are learning parameters of hidden nodes and β i ¼ ½βi1 ; …; β im  is the weight vector connecting ith hidden node to output neuron. For additive nodes, the activation function gðxÞ : R-R for ith hidden node,

gi, is defined as g i ¼ Gðai ; bi ; xÞ ¼ gðai  x þ bi Þ;

ai A Rn ; bi A R

ð2Þ

The above two equations can be written in the matrix form as Hβ ¼ T

ð3Þ

where H is called the hidden layer output matrix of the SLFNs [22] and β is the weight vector connecting the hidden node to the output node and T is the target vector written as 2 3 Gða1 ; b1 ; x1 Þ … GðaL^ ; bL^ ; x1 Þ 6 7 ⋮ … ⋮ H NL^ ¼ 4 5 Gða1 ; b1 ; xN Þ … GðaL^ ; bL^ ; xN Þ 0

βT

1

B 1C B ⋮ C B C βLm ¼ ^ B TC @ β L^ A

0 and

t T1

1

B C ⋮C T Nm ¼ B @ A t TL^

ð4Þ

As described in [25], the parameters of hidden layer nodes as weights ai and bias bi need not be adjusted again and again but these are randomly generated, assigned and fixed. Therefore, for known values of hidden layer output matrix H and output matrix T, the solution of output parameter, β, can be obtained as

β^ ¼ H† T

ð5Þ †

where H is the Moore–Penrose generalized pseudo inverse [36] of the hidden layer output. The orthogonal projection method can be used for determining Moore–Penrose generalized inverse as H † ¼ ðH T HÞ  1 H T provided HTH is non-singular in nature. However, HTH may sometimes tends to be singular and in that case the orthogonal projection method may not be used for inverse calculation. It is important to note that singular value decomposition (SVD) [36] is always used for calculation of Moore–Penrose inverse and implementation of ELM. When the number of training data samples ^ still input is higher than the number of hidden nodes, that is, N 4 L, weights and biases can be generated randomly, assigned and fixed, an output weights by using same Moore-Penrose inverse of hidden nodes matrix H with a small nonzero training error. Therefore, it should be noted in conclusion about the theoretical aspect of ELM that the hidden node parameters ai and bi (input weights and biases or centres) of SLFNs need not be tuned during training and may simply be assigned with random values. One of the typical features about the implementation of ELMs is addition of random computational nodes in hidden nodes and hidden layer of SLFNs need not be tuned again and again as it is independent of input training data samples. Unlike traditional neural networks, ELM not only tends to reach the smallest training error but also the smallest norm of output weights that leads to better generalization performance of learning networks. Therefore, the output weight can be resolved using least-square method as ELM hidden layer parameters are fixed. Accordingly output estimation function (1) may be determined for ELM by f ðELMÞ ðxÞ ¼

L^ X

βi gi ðxÞ;

ai A Rn ; bi A R; βi A Rm

i¼1

¼

L^ X

βi gðai  x; bi Þ

ð6Þ

i¼1

The essence of ELM can be summarized as follows: (1) The parameters of hidden layer of ELM need not be iteratively tuned [20,38]. (2) The training error J H β  T J and norm of output weight J β J need to be minimized [20,37,38].

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i

R.P. Singh et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

(3) Universal approximation condition inevitable for hidden layer feature mapping [22–24]. It must be noted that unlike other standard neural networks where it is necessary that hidden-layer parameters to be tuned again and again. However, in ELM these parameters can be randomly generated and fixed rather tuned. And output weights are determined analytically using least-squares method. 2.2. OSELM ELM algorithm described in previous subsection learn training data set only after all training samples are available and is essentially batch or offline learning method. As entire data samples are used for training process so all data should be loaded in large memory and memory may run out when data samples are very large. Therefore, in real time applications standard ELM algorithm has this limitation and cannot be applied in many industrial applications of importance where data is available one-by-one or chunk by chunk. For this type of application, sequential learning algorithms are preferred. There are numbers of sequential machine learning algorithms but the number of parameters are optimized which is very time consuming to tune those parameters. As discussed in [39], the said algorithm is known as Online Sequential Extreme Learning Machine (OSELM) and data may be learned one-by-one or trunk-by trunk or a block of data and not using data which has already been used for training the model. It is a simple and efficient online sequential learning algorithm that can easily handle additive node and radial basis function (RBF) node in unified environment [25]. Hence, sequential learning algorithm may be summarized as follows: (1) Training data is sequential in one-by-one or block-by-block with fixed or variable size presented to the learning algorithm. (2) In this learning algorithm, all the data is not supplied for training again and again but only newly arrived data samples is used for training. (3) Training data sample is discarded once it is used for training of the machine. (4) Learning machine has no advance knowledge that how much training data samples to be presented to the learning algorithm.

3

Therefore, the new model's updated parameter due to the arrival of new chunk of data sample is !T ! H0 T0 ð1Þ ð0Þ ð0Þ 1 β ¼ C1 ¼ C 1 1 ðC 1 β  H T1 H 1 β þ H T1 T 1 Þ H1 T1 ¼β

ð0Þ

ð0Þ

þ C 1 1 H T1 ðT 1  H 1 β Þ

ð10Þ

The effect of newly arrived data is reflected by perturbation of β as Δβ and new model based βn is obtained from old model βð0Þ and governed by equation as

βn ¼ βð0Þ þ ΔβðX n Þ

ð11Þ

With generalization and recursive approach, as new data sample arrives, a recursive least-squares algorithm [40] solution can be written as C k þ 1 ¼ C k þ H Tk þ 1 H k þ 1

ð12Þ

βðk þ 1Þ ¼ βðkÞ þ C kþ11 HTk þ 1 ðT k þ 1  Hk þ 1 βðkÞ Þ

ð13Þ ðk þ 1Þ

is used for computation of β From (13), it is clear that ðkÞ from β in (13), using Woodbury formula [41] an updated formula for C kþ11 can be used as C kþ11

C kþ11 ¼ ðC k þH Tk þ 1 H k þ 1 Þ  1 ¼ C k 1 þC k 1 H Tk þ 1 ðI þ H k þ 1 C k 1 H Tk þ 1 Þ  1  H Tk þ 1 C k 1

ð14Þ

C kþ11 ,

Let K k þ 1 ¼ an updating equation for machine learning for newly arrived data sample can be written as K k þ 1 ¼ K k þ K k H Tk þ 1 ðI þ H k þ 1 K k H Tk þ 1 Þ  1  H Tk þ 1 K k

ð15Þ

βðk þ 1Þ ¼ βðkÞ þ K k þ 1 HTk þ 1 ðT k þ 1  Hk þ 1 βðkÞ Þ

ð16Þ

ðk þ 1Þ

The value of an updated weight β for new data arrival at ðk þ 1Þth trunk can be determined iteratively with recursive ðk þ 1Þ formula (15) and for updated weight β at ðk þ 1Þth trunk, an output estimation function for OSELM is given by f ðOSELMÞ ðxÞ ¼

L^ X

þ 1Þ βðk g i ðxÞ; i

ai A Rn ; bi A R; βi A Rm

i¼1

¼

L^ X

þ 1Þ βðk Gðai ; bi ; xÞ i

i¼1

Batch learning based algorithm, ELM, may be formulated for this case so as to make it online sequential. The output weight matrix, β^ , is a least-squares solution of (5). The orthogonal projection method can be used for determining Moore–Penrose generalized inverse as H † ¼ ðH T HÞ  1 H T provided that HTH is nonsingular in nature. The term H † ¼ ðH T HÞ  1 H T is also called left pseudo-inverse of H as it holds H † H ¼ I L^ . Substituting the value H † ¼ ðH T HÞ  1 H T into (5), sequential implementation of the least-squares solution is possible in OSELM. Therefore, (5) becomes

β^ ¼ ðHT HÞ  1 H T T

ð7Þ

For online sequential ELM, consider initial training data sample ^ for batch learning ELM, the solution ðxi ; t i Þ, 8 i ¼ 1; …; L0 and L0 Z L, is obtained by minimizing ‖H 0 β  T 0 ‖ as

β0 ¼ C 0 1 HT0 T 0 H T0 H 0 .

ð8Þ

As detailed analysis of OSELM is given in [39], where C 0 ¼ on new chunk of data samples arrive as ðxi ; t i Þ, 8 i ¼ L0 þ 1; … ; L0 þ L1 , then !T ! !   H0 H0 H0 T T C1 ¼ ð9Þ ¼ H0 H1 ¼ C 0 þ H T1 H 1 H1 H1 H1

¼

L^ X

ðkÞ T βðkÞ i þ K k þ 1 H k þ 1 ðT k þ 1  H k þ 1 β Þgðai  x; bi Þ

ð17Þ

i¼1

The relationship among the pixels in natural images can be remembered or learned by training process of ELM and OSELM application for regression purpose. It is also found that the relationship among the pixels remain unchanged despite the images are subjected to different kinds of geometrical and conventional attacks. So this inherent feature of pixels relationship can be used for robust watermark embedding and extraction using ELM, OSELM and WELM machine learning methods. It is clear that implementation of OSELM solution is possible from recursive least-squares algorithm as in [40]. After detailed analysis of OSELM [39], if number of initialization data ^ ELM and sample L0 is not less than number of hidden nodes L, OSELM both learning algorithms can achieve same training error, testing error and generalization accuracy. After initialization phase, ELM learning phase is based on batch learning and OSELM learning phase realizes either one-by-one or trunk-by-trunk. In OSELM, once data sample is used for training, it is discarded and not used in subsequent phases [28]. In OSELM algorithm, when data sample is initially received one-by-one rather than trunk-by-trunk, Eqs. (15) and (16) reduce

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i

R.P. Singh et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

4

to the following format with the help of SMW [41] identity: T K k hk þ 1 hk þ 1 K k Kkþ 1 ¼ Kk  T 1 þhk þ 1 K k hk þ 1

ð18Þ

βðk þ 1Þ ¼ βðkÞ þK k þ 1 hk þ 1 ðt Tk þ 1 hTk þ 1 βðkÞ Þ

ð19Þ

where hk þ 1 ¼ Gða1 ; b1 ; xðk þ 1Þ Þ; …; GðaL^ ; bL^ ; xðk þ 1Þ Þ. The value of an ðk þ 1Þ updated weight β for new data arrival at ðk þ 1Þth trunk can be determined iteratively with recursive formula (19) and accordingly output estimation function (1) may be determined for ELM for ðk þ 1Þ updated weight β at ðk þ 1Þth trunk, an output estimation function for OSELM is given by f ðOSELMÞ ðxÞ ¼

L^ X

þ 1Þ βðk Gðai ; bi ; xÞ; i

ai A Rn ; bi A R; βi A Rm

¼

ðkÞ T βðkÞ i þ K k þ 1 H k þ 1 ðT k þ 1  H k þ 1 β Þgðai  x; bi Þ

ð26Þ

where ΩELM ¼ HH T : ΩELMij ¼ hðxi Þ  hðxj Þ ¼ Kðxi ; xj Þ. Actually hidden layer output vector maps the data from input space to hidden layer ~ This feature mapping is possible with the space with dimension L. help of wide range of functions which include

 Sigmoid function: Gða; b; xÞ ¼

1 1 þ expð  ða  x þbÞÞ

ð27Þ

 Gaussian function:

i¼1 L^ X

Kernel version of WELM output function is given by 0 1 Kðx; x1 Þ  1 B C I ⋮ f ðkernelÞ ðxÞ ¼ @ þ W ΩELM H T WT A C Kðx; xL~ Þ

ð20Þ

Gða; b; xÞ ¼ expð  b‖x  a‖2 Þ

ð28Þ

i¼1

 Hardlimit function: 

2.3. Weighted Extreme Learning Machine

Gða; b; xÞ ¼

Weighted regularized ELM [33] was initially proposed in [34,35] used for deterministic neural classification of data set. In this paper, we proposed the use of WELM for digital watermarking scheme for color images. Other variants of ELM has been used for robust and imperceptible digital watermarking schemes described in the literature [4,5]. Mathematical formulation of Weighted ELM as an optimization problem is written as min : ‖H β  T‖

2

and

JβJ

ð21Þ

1 0

if a  x b Z 0 otherwise

 ð29Þ

 Multiquadrics function: 2

Gða; b; xÞ ¼ ð‖x  a‖2 þb Þ1=2

ð30Þ

Therefore, with the inclusion of kernel functions or feature mapping, the given weighted ELM provides a unified solution for SLFNs.

where T is target vector and H is hidden layer matrix. More precisely the problem can be written as L~ 1 1X min : ‖β‖2 þ CW ‖ξ ‖2 ; 2 2i¼1 i

3. Watermarking scheme in DWT domain Subject to

i ¼ 1; …; L~

hðxi Þβ

¼ t Ti

ξ

T i ;

ð22Þ

Again hðxi Þ is the feature mapping vector corresponds to input sample xi in the hidden layer and β denotes output weight vector connecting hidden layer and output layer. The penalty parameter, ξi, represents training error of input data sample xi is due to the difference between actual output hðxi Þβ and the desired output ti. C is a trade off parameter and W is the weight parameter which influences the difference between actual value and desired value. As detailed mathematics of the optimization problem and solution has been discussed in [33], two versions of output vector β between hidden layer and output layer can be derived depends ~ If number of hidden layer node is small upon hidden layer size L. as compared to input data sample, than output weight vector β is  1 I β ¼ HT þ WHHT WT ð23Þ C Otherwise, output vector β will be  1 I β ¼ þ WHHT H T WT C

ð24Þ

Therefore, estimation function for WELM for all input data samples 0 1  1 T I T WT C B HH C þ WHH B C B C if↦L~ ¼ small B C 1 ð25Þ f ðWELMÞ ðxÞ ¼ B  C B H I þ WHH T H T WT C B C C @ A if ↦L~ ¼ large

In natural image analysis, it is found that every pixel has relationship to its neighbor, therefore, it can be predicted by its neighbors [11]. The said relationship among the pixels can be remembered or learned by training process of ELM application for regression. So this inherent feature of pixels relationship can be used for robust watermark embedding and extraction using ELM, OSELM and WELM. 3.1. ELM based watermark embedding Let us assume that there is a color image I ¼ ½R; G; B of size m  n, where R, G and B are the red, green and blue channel component of color image, respectively. An image to be embedded as watermark, w, is a binary image of the size p  q. Here watermark logo is embedded into blue channel as human vision system (HSV) [42] is insensitive to blue channel. The embedding algorithm is 1. The logo watermark is permuted and reshaped into line w of size p  q, as w ¼ fwi gi ¼ 1;…;pq : 2. The blue channel components, B, of the color image, I, to be watermarked transferred through DWT and its low frequency sub-band denoted as B. 3. For training of ELM, OSELM and WELM for regression and reference pixel position pt ¼ fit ; jt g, the training data set may be extracted from every 3  3 active window obtained after applying DWT and training data set is obtained as S ¼ dt j dt ¼ ½Bðit  1 ; jt  1 Þ; Bðit  1 ; jt Þ; Bðit  1 ; jt þ 1 Þ; Bðit ; jt  1 Þ; Bðit ; jt þ 1 Þ

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i

R.P. Singh et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

; Bðit þ 1 ; jt  1 Þ; Bðit þ 1 ; jt Þ; Bðit þ 1 ; jt þ 1 Þ

ð31Þ

where Bð; Þ is the pixel value in the B-channel of color image. And the pixels set Bði; jÞ are the training objective of ELM, OSELM and WELM and their respective regression output is given by following functions: X βi gðai y ¼ f ðELMÞ ðdt Þ ¼ i A mn

ai A Rn ; bi A R; β i A Rm

 dt ; bi Þ;

X

y ¼ f ðOSELMÞ ðdt Þ ¼

ai A Rn ; bi A R; β i A Rm

i A mn

X

¼

þ 1Þ βðk g i ðdt Þ; i

ð32Þ

β

ðk þ 1Þ Gðai ; bi ; dt Þ i

i A mn

X

¼

ðkÞ T βðkÞ i þK k þ 1 H k þ 1 ðT k þ 1  H k þ 1 β Þgðai  dt ; bi Þ

ð33Þ

i A mn

0

1  1 T I T HH þ WHH WT C B C B C B C if ↦L~ ¼ small B C 1 y ¼ f ðWELMÞ ðdt Þ ¼ B  C T T B H I þ WHH C H WT B C C @ A if ↦L~ ¼ large 0 1 Kðx; d1t Þ  1 B C I ⋮ þ W ΩELM ¼@ H T WT A C Kðx; dLt ~ Þ where

ð34Þ

βi is output weight vector in ith iteration for ELM and

0

0 ¼ f ðELMÞ ðdt Þ;

¼

L^ X

βi gðai 

ai A R ; bi A R; βi A R n

ð36Þ

i¼1

0

y0t ¼ f ðOSELMÞ ðdt Þ ¼ ¼

X

þ 1Þ βðk g i ðdt Þ; i

ai A Rn ; bi A R; βi A Rm

i A mn þ 1Þ βðk Gðai ; bi ; dt Þ i

i A mn

¼

X

T βðkÞ i þ K k þ 1 H k þ 1 ðT k þ 1

i A mn ðkÞ

0

ð38Þ

0

After comparing predicted value y0t ¼ f ðdt Þ and actual value Bðit ; jt Þ, one can embed watermark according to Eqs. (36)–(38) as 0 1 0 maxððBðit ; jt Þ; f ðELMÞ ðdt Þð1 þ ηt ÞÞ B C B C if↦wt ¼ 1 C Bn ðit ; jt Þ ¼ B ð39Þ 0 B minððBðit ; j Þ; f C t ðELMÞ ðdt Þð1  ηt ÞÞ A @ if↦wt ¼ 0

ð40Þ

0

1 0 maxððBðit ; jt Þ; f ðWSELMÞ ðdt Þð1 þ ηt ÞÞ B C B C if↦wt ¼ 1 C ð41Þ Bn ðit ; jt Þ ¼ B 0 B minððBðit ; j Þ; f C t ðWSELMÞ ðdt Þð1  ηt ÞÞ A @ if↦wt ¼ 0 where ηt is the embedding strength and Bn ðit ; jt Þ is the modulated wavelet coefficient in low frequency sub-band domain at position ðit ; jt Þ. 5. After embedding blue channel with desire watermark, Bchannel is combined with R and G color channels and resultant image is the final watermarked image.

3.2. Watermark detection

m

0 dt ; bi Þ

X

or

ð35Þ

we obtain predicting pixel at each embedding position ðit ; jt Þ, where t ¼ 1; …; p  q and trained output from ELM, OSELM and WELM y0t

1  1 T I T WT C B HH C þ WHH B C B C if ↦L~ ¼ small B C 0 0  yt ¼ f ðWELMÞ ðdt Þ ¼ B  C B H I þ WHH T  1 H T WT C B C C @ A ~ if ↦L ¼ large 0 1 0 Kðx; d1t Þ  1 B C I ⋮ þ W ΩELM ¼@ H T WT A C 0 Þ Kðx; dLt ~

1 0 maxððBðit ; jt Þ; f ðOSELMÞ ðdt Þð1 þ ηt ÞÞ B C B C if ↦wt ¼ 1 C Bn ðit ; jt Þ ¼ B 0 B minððBðit ; j Þ; f C ðd Þð1  η ÞÞ t ðOSELMÞ t @ t A if ↦wt ¼ 0

β is output weight vector in ðk þ 1Þth trunk of new data for OSELM. 4. For each embedding position ðit ; jt Þ, eight pixels in 3  3 active window are collected to form training data set in blue channel color image as 0

0

0

or

ðk þ 1Þ

S0 ¼ dt j dt ¼ ½Bðit  1 ; jt  1 Þ; Bðit  1 ; jt Þ; Bðit  1 ; jt þ 1 Þ; Bðit ; jt  1 Þ; Bðit ; jt þ 1 Þ; Bðit þ 1 ; jt  1 Þ; Bðit þ 1 ; jt Þ; Bðit þ 1 ; jt þ 1 Þ

5

 H k þ 1 β Þgðai  dt ; bi Þ

ð37Þ

For watermark detection, the original watermark image is not necessary and extraction algorithm is 1. The blue channel B of watermarked color image is decomposed through DWT and low frequency component is designated by B0 . 2. For each pixel reference position pt ¼ fit ; jt g, data set can be constructed by collecting 3  3 active window. The formed data ~ t ; j Þ, is set S~ is based on reference pixel Bði t ~ t  1; j ~ S~ ¼ d~ t j d~ t ¼ ½Bði t  1 Þ; Bðit  1 ; jt Þ; ~ ~ ~Bðit  1 ; j t þ 1 Þ; Bðit ; jt  1 Þ; Bðit ; jt þ 1 Þ;

Fig. 1. Lena, Baboon, Peppers, Goldhill and Barbara images used for watermarking and watermark image.

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i

R.P. Singh et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

6

Fig. 2. Lena, Baboon, Peppers, Goldhill and Barbara watermarked images with no attack with (PNSR ¼ 53.06 dB, 48.5 dB, 51.06 dB, 50.09 dB, 51.05 dB) with activation function RBF.

Table 1 The comparison results PSNR(dB) values under different attacks on Lena watermarked image with ELM(RBF), OSELM and WELM. Attacks

ELM (RBF)

OSELM WELM

Chang et al. [11]

Chen et al. [12]

Blurring Cropping JPEG Noise Rotation Scaling Scaling– Cropping Sharpening

30.73 18.20 32.77 29.90 24.77 43.02 19.54

31.22 18.94 34.68 30.95 24.89 43.08 19.98

28.9849 15.8396 32.91 25.0476 22.7389 41.8991 17.289

29.36 11.68 30.89 30.29 14.95 29.80 15.93

27.75 17.27 27.25 28.78 15.80 27.13 15.29

36.02

36.32

36.7273 27.83

33.12

Table 2 The comparison results PSNR(dB) values under different attacks on Baboon watermarked image with ELM(RBF), OSELM and WELM. Attacks

Blurring Cropping JPEG Noise Rotation Scaling Scaling– Cropping Sharpening

ELM (RBF)

OSELM WELM

Chang et al. [11]

Chen et al. [12]

29.98 15.90 30.33 32.79 20.58 30.25 17.60

23.54 15.10 34.61 24.91 20.58 30.34 17.60

23.55 15.09 31.32 25.0 20.58 30.35 17.6022

21.40 11.35 23.46 30.32 13.83 21.14 15.12

25.50 15.74 31.75 33.10 14.48 28.32 15.07

19.54

19.49

19.49

17.54

24.60

Table 3 The comparison results PSNR(dB) values under different attacks on Peppers watermarked image with ELM(RBF), OSELM and WELM. Attacks

ELM (RBF)

OSELM WELM

Chang et al. [11]

Chen et al. [12]

Blurring Cropping JPEG Noise Rotation Scaling Scaling– Cropping Sharpening

28.94 15.84 31.11 32.96 22.72 40.62 17.94

28.98 15.84 34.66 25.05 22.74 41.17 17.29

28.9849 15.8396 32.10 25.0476 22.7389 41.8991 17.289

28.61 11.68 30.27 30.47 14.82 28.42 14.16

30.70 16.84 27.91 32.04 17.13 33.59 15.41

36.72

36.63

36.7273 26.41

32.52

~ ~ ~ t þ 1; j Bði t  1 Þ; Bðit þ 1 ; jt Þ; Bðit þ 1 ; jt þ 1 Þ

Table 4 The comparison results PSNR(dB) values under different attacks on Goldhill watermarked image with ELM(RBF), OSELM and WELM. Attacks

ELM(RBF)

OSELM

WELM

Blurring Cropping JPEG Noise Rotation Scaling Scaling–Cropping Sharpening

29.81 18.58 32.14 27.65 25.01 41.89 20.97 35.70

29.86 18.59 34.10 25.03 25.0 41.48 20.97 35.35

24.9964 18.5848 32.06 25.0225 24.9979 41.6991 20.9756 35.3379

Table 5 The comparison results PSNR(dB) values under different attacks on Barbara watermarked image with ELM(RBF), OSELM and WELM. Attacks

ELM(RBF)

OSELM

WELM

Blurring Cropping JPEG Noise Rotation Scaling Scaling–Cropping Sharpening

29.0667 16.6237 31.60 24.7444 23.41 41.0826 18.3137 34.8879

29.1012 16.6247 34.05 24.8886 23.4403 40.7096 18.3098 34.3563

29.067 16.6246 33.02 27.5398 23.4679 40.905 18.3125 34.8926

Table 6 The comparison results BER values under different attacks on Lena watermarked image with ELM(RBF), OSELM and WELM. Attacks

ELM (RBF)

OSELM WELM Shen [16]

Kutter et al. [17]

Yu et al. [1]

Blurring Cropping JPEG Noise Rotation Scaling Scale– Crop Sharp

0.0321 0.104 0.176 0.0179 0.0407 0.0021 0.0498

0.0301 0.101 0.147 0.0112 0.0307 0.0020 0.0365

0.0537 0.08 0.3425 0.0737 0.1225 0.08 –

0.0113 0.07 0.3512 0.0058 0.1021 0.07 –

0.0876

0.0667 0.1737 –

–

–

3. For each embedding position, by using well-trained based ELM, OSELM and WELM, we can determine corresponding output y0 as ai A Rn ; bi A R; βi A Rm

0.0 0.0605 0.36 0.0350 0.0963 0.0605 –

ð42Þ ¼

y0 ¼ f ðELMÞ ðd~ t Þ;

0.1287 0.1252 0.0827 0.1581 0.1204 0.0901 0.1572

L^ X

βi gðai  d~ t ; bi Þ

ð43Þ

i¼1

y0 ¼ f ðOSELMÞ ðd~ t Þ ¼

X

þ 1Þ βðk g i ðd~ t Þ; i

ai A Rn ; bi A R; β i A Rm

i A mn

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i

R.P. Singh et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

0 Table 7 The comparison results BER values under different attacks on Baboon watermarked image with ELM(RBF), OSELM and WELM. Attacks

ELM(RBF)

OSELM

WELM

Blurring Cropping JPEG Noise Rotation Scaling Scale–Crop Sharp

0.1994 0.2233 0.2261 0.2013 0.2408 0.2344 0.2316 0.227

0.0993 0.1213 0.0193 0.0175 0.1158 0.1232 0.091 0.1324

0.0919 0.1261 0.1066 0.127 0.1002 0.1654 0.1564 0.0395

Table 8 The comparison results BER values under different attacks on Peppers watermarked image with ELM(RBF), OSELM and WELM. Attacks

ELM(RBF)

OSELM

WELM

Blurring Noise JPEG Rotation Scaling Scale–Crop Sharp

0.2491 0.1912 0.2592 0.2022 0.2086 0.0498 0.0876

0.0533 0.1369 0.0616 0.0873 0.0983 0.0365 0.0667

0.0579 0.0625 0.0855 0.0993 0.0496 0.0901 0.0901

Table 9 The comparison results BER values under different attacks on Goldhill watermarked image with ELM(RBF), OSELM and WELM. Attacks

ELM(RBF)

OSELM

WELM

Blurring Noise JPEG Rotation Scaling Scale–Crop Sharp

0.2629 0.2362 0.2142 0.2583 0.239 0.0498 0.0876

0.091 0.0892 0.0855 0.1388 0.0579 0.0365 0.0667

0.0855 0.1234 0.0248 0.1151 0.1195 0.0901 0.0901

Table 10 The comparison results BER values under different attacks on Barbara watermarked image with ELM(RBF), OSELM and WELM. Attacks

ELM(RBF)

OSELM

WELM

Blurring Noise JPEG Rotation Scaling Scale–Crop Sharp

0.2077 0.2399 0.1811 0.2096 0.1783 0.0498 0.0876

0.0551 0.1369 0.1213 0.0735 0.0974 0.0365 0.0667

0.0662 0.127 0.1234 0.1002 0.1654 0.0901 0.0901

¼

X

þ 1Þ βðk Gðai ; bi ; d~ t Þ i

i A mn

¼

X

ðkÞ T ~ βðkÞ i þ K k þ 1 H k þ 1 ðT k þ 1  H k þ 1 β Þgðai  d t ; bi Þ

1

 1 T I T WT C B HH C þ WHH B C B C if↦L~ ¼ small B C 0 ~   y ¼ f ðWELMÞ ðd t Þ ¼ B C 1 T B H I þ WHH T H WT C B C C @ A ~ if↦L ¼ large

Kðx; d~ 1t Þ ⋮

Kðx; d~ Lt ~ Þ

 1 C I C H T WT A C þ W ΩELM

ð45Þ

The watermark can be extracted by comparing between predicted pixel value blue channel B by trained ELM, OSELM and WELM output given by (43)–(45) and actual pixel value as ! ~ ~ t; j Þ4f 1 if ↦Bði 0 t ðELMÞ ðd t Þ wt ¼ ð46Þ 0 else w0t ¼

w0t

¼

1

~ ~ t; j Þ4f if ↦Bði t ðOSELMÞ ðd t Þ

0

else

1

~ ~ t; j Þ4f if ↦Bði t ðWELMÞ ðd t Þ

0

else

! ð47Þ ! ð48Þ

4. Finally, one-dimensional watermark bit sequence w1 ; w2 ; …; wpq is converted into a two-dimensional watermark logo image w0 . The detection result may be verified by parameter BER(bit error rate)and PSNR(peak signal to noise ratio) between image I(x) and watermarked image I 0 ðxÞ as BER ¼

pq X

ðwt  w0t Þ=p  q

ð49Þ

t¼1

255  255 =ðm  nÞ PSNR ¼ 10 log 10 Pm  1 Pn  1 0 2 i¼0 j ¼ 0 ðxij  xij Þ

ð50Þ

The SIMðX; X n Þ, a statistical similarity correlation check can be determined between original and recovered watermark sequences X and X n as , n n qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi X X SIMðX; X n Þ ¼ ðX  X n Þ ð51Þ ðX  X n Þ: i¼1

i¼1

Table 11 The comparison of training time (s) of ELM(RBF), OSELM and WELM on Lena, Baboon, Peppers, Goldhill and Barbara with Chang's method [11]. Images

Time ELM(RBF) Time (OSELM)

Time (WELM)

Time(Chang et al. [11])

Lena Baboon Peppers Goldhill Barbara

0.0156 0.0 0.0312 0.0624 0.0312

0.0 0.01 0.0 0.016 0.015

13.80 19.00 20.00 – –

0.0 0.0 0.0 0.0 0.0

Table 12 Three computed time spans (s) for ELM(RBF), OSELM and WELM on Lena image with method [5] at 20 hidden neurons (N). Images

Time ELM(RBF)

Time(OSELM)

Time(WELM)

Time([5])

Training time Embedding time Extraction time

0.0156 1.620 0.0156

0.0 0.770 0.0

0.0 0.98 0.0

0.0156 1.6563 0.0156

ð44Þ Table 13 Three computed time spans (s) for ELM(RBF), OSELM and WELM on Baboon image with method [5] at 20 hidden neurons (N).

i A mn

0

B ¼B @

7

1

or

Images

Time ELM(RBF)

Time(OSELM)

Time(WELM)

Time([5])

Training time Embedding time Extraction time

0.0156 1.662 0.0156

0.0 0.881 0.0

0.0 1.010 0.0

0.0156 1.7188 0.0156

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i

R.P. Singh et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

8

Table 14 Attacked Lena image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Lena image with ELM(RBF).

Table 15 Attacked Lena image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Lena image with OSELM.

Table 16 Attacked Lena image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Lena image with WELM.

Table 17 Attacked Baboon image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Baboon image with ELM(RBF).

4. Experimental results and discussion 4.1. Experimental results The experiment is performed on original color images Lena, Baboon, Peppers, Goldhill and Barbara of size 512  512. Lena, Peppers, Goldhill and Barbara images representing low complexity as they

have smooth regions while Baboon has high complexity because it contains regions of more complex texture. A watermark image is shown in Fig. 1 of size 32  32 and it has been used as a watermark for embedding in color images for experimental analysis. All images used here for experimental purpose are decomposed using two-level wavelet transformation having sub-image LL2 of size 128  128 used for watermark embedding (Fig. 1). Since in this watermarking scheme color images are used, watermark logo is embedded into

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i

R.P. Singh et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

9

Table 18 Attacked Baboon image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Baboon image with OSELM.

Table 19 Attacked Baboon image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Baboon image with WELM.

Table 20 Attacked Peppers image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Pepper image with ELM(RBF).

Table 21 Attacked Peppers image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Peppers image with OSELM.

blue channel as human vision system (HSV) [42] is insensitive to blue channel which lead to improve imperceptibility of scheme (Fig. 2). Tables 12 and 13represent three computed time on Lena and Baboon as training time, time consumed in embedding and embedding time with ELM(RBF), OSELM and WELM at 20 number of hidden neurons and compared with existing method in [5]. It can be noted that training time for ELM(RBF) is 15.6 ms and its value is almost zero with OSELM and WELM, much better than

existing method [5]. This implies that a fast training process unlike training time of SVM based learning methods and training of a gradient based methods which are usually get strucked in presence of local minimas's and thus waste of time. This small training time feature make these algorithms become suitable for real time image processing applications including information hiding, image and video watermarking in particular. It is cleared from results that embedding time and extraction time are also

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i

10

R.P. Singh et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

Table 22 Attacked Peppers image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Pepper image with WELM.

Table 23 Attacked Goldhill image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Goldhill image with ELM(RBF).

Table 24 Attacked Goldhill image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Goldhill image with OSELM.

Table 25 Attacked Goldhill image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Goldhill image with WELM.

within suitable expected range. It can be noted here that the computed time taken in case of image Baboon is little bit higher as nature of Baboon image is complex and not smooth. The computed time span in case of different ELM based learning models for optimizing watermarking procedures under real time applications without compromising visual quality of watermarked image. In order to evaluate the robustness of proposed watermarking schemes where different learning methods are used. The watermarked

images Lena, Baboon, Peppers, Goldhill and Barbara are subjected to the number of attacks. Embedded watermark has been extracted from attacked watermarked images by learning neighboring relationship among pixels with learning models. It is found that neighboring relationship among pixels remains unchanged even though watermarked images are subjected to the number of attacks. Therefore, this pixels relationship used for prediction of pixels relationship and thereby embedded watermark is extracted with the help of these

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i

R.P. Singh et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

11

Table 26 Attacked Barbara image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Barbara image with ELM(RBF).

Table 27 Attacked Barbara image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Barbara image with OSELM.

Table 28 Attacked Barbara image, corresponding PSNR(dB) and extracted binary ownership as a watermark from attacked watermarked Barbara image with WELM.

machine learning methods. The values of PSNR, BER and SIMðX; X n Þ are calculated on Lena, Baboon and Peppers and results are compared with already existing methods [1,11,16,17,22] as shown in Tables 1–3. The comparative results are determined using ELM, OSELM and WELM on Goldhill and Barbara and results are summarized in Tables 4–10. It was found that calculated parameters show the efficacy of learning methods in the form of robustness and imperceptibility (Table 11). Experimental results show that our implemented watermarking schemes based on ELM, OSELM and WELM outperformed Yu's method [1], Chang's method [11] and Kutter's method [17] against different attacks including blurring, cropping, JPEG, rotation, scaling, scaling– cropping, and sharpening are shown in Tables 1–4.

0.154 while watermark extracted by Shen's method, Kutter's method and Yu's method, BER are 0.360, 0.3425 and 0.3512, respectively as shown in Tables 6 and 7 and, therefore, in most of cases, our methods are superior as compared to other existing methods [1,11,17]. The attacked watermarked images, corresponding PSNR values, the extracted watermark and corresponding values of BER are shown in Tables 6 and 7. As watermark scheme are robust, experimental results demonstrate that still recognizable watermark can be extracted from severely attacked watermarked image. Therefore, simulated results demonstrate that the proposed watermarking learning methods are more robust and imperceptible than that of other machine learning methods which are used for comparison.

4.2. Discussion 5. Conclusion We demonstrate several visual image results in the form of attacked Lena, Baboon, Peppers, Goldhill and Barbara watermarked images and extracted watermark using machine learning methods shown in Tables 14–28. Attacked watermark and extracted watermark images by our implemented methods are quality wise superior than Shen's method, Kutter's method and Yu's method. When watermarked images are subjected to attacks including JPEG compression, the watermark extracted by our methods has BER 0.176 and

A robust and imperceptible watermarking scheme based on learning methods proposed for copyright protection and authentication of ownership. The proposed schemes are composed of DWT with ELM, OSELM and WELM in low frequency band LL2 in level two where all energy of image concentrated used for water marking. The watermark is embedded into low frequency band of wavelet domain based on ELM, OSELM and WELM regression

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i

12

R.P. Singh et al. / Neurocomputing ∎ (∎∎∎∎) ∎∎∎–∎∎∎

training process. The concept of high correlation with its neighbor pixels is used for prediction. This relationship can be predicted by training models for regression and used for watermarking scheme. When watermarked image is subject to attacks, it is found that this relationship among pixels do remain unchanged and it can be learned by training process used models. This process can be used for watermark extraction from watermarked image. As machine learning methods used have high generalization ability, there is always possibility of correct extraction of watermark unless the watermarked image is severely attacked. Experimental results show that our implemented methods outperformed when compared to other learning methods. References [1] P.T. Yu, H.H. Tsai, D.W. Sun, Digital watermarking based on neural networks for color images, Signal Process. 81 (3) (2001) 663–671. [2] S.D. Lin, C.F. Chen, A robust DCT-based watermarking for copyright protection, IEEE Trans. Consum. Electron. 46 (3) (2000) 415–421. [3] W.C. Chu, DCT-based image watermarking using subsampling, IEEE Trans. Multimed. 5 (1) (2003) 34–38. [4] C. Agarwal, A. Mishra, A. Sharma, G. Chetty, A novel scene based robust video watermarking scheme in dwt domain using extreme learning machine, in: Extreme Learning Machines 2013: Algorithms and Applications, Springer International Publishing, Beijing, 2014, pp. 209–225. [5] A. Mishra, A. Goel, R. Singh, G. Chetty, A novel image watermarking using extreme learning machine, in: 2012 International Joint Conference on Neural Networks (IJCNN), 2012, pp. 1–6. [6] Xuezhang Zhao, Jianxi Peng, Yunjiang Xi, The research on digital watermarking scheme algorithm based on neural networks and singular value decomposition, in: Seventh International Conference on Neural Computation (ICNC 2011), 2011, pp. 293–297. [7] A. Kaur, A. Goel, H. Gupta, Digital watermarking in neural networks model, in: Recent Advances in Engineering and Computational Sciences (RAECS), 2014, pp. 1–6. [8] G. Feng, Y. Lan, X. Zhang, Z. Qian, Dynamic adjustment of hidden node parameters for extreme learning machine, IEEE Trans. Cybern. 45 (2) (2015) 279–288. [9] G. Feng, Z. Qian, N. Dai, Reversible watermarking via extreme learning machine prediction, Neurocomputing 82 (1) (2012) 62–68. [10] I.J. Cox, J. Kilian, F.T. Leighton, T. Shamoon, Secure spread spectrum watermarking for multimedia, IEEE Trans. Image Process. 6 (12) (1997) 1673–1687. [11] C.Y. Chang, H.J. Wang, S.W. Pan, A robust DWT-based copyright verification scheme with Fuzzy ART, J. Syst. Softw. 82 (2009) 1906–1915. [12] T.H. Chen, G. Horng, W.B. Lee., A publicity verifiable copyright-proving scheme resistant to mallicious attacks, IEEE Trans. Ind. Electron. 51 (1) (2005) 327–334. [13] S.P. Mohanty, Digital Watermarking: A Tutorial Review, 1999. [14] V.N. Vapnik, Statistical Learning Theory, John Wiley & Sons, New York, 1998. [15] V.N. Vapnik, An overview of statistical learning theory, IEEE Trans. Neural Netw. 10 (5) (1999) 988–999. [16] R. Shen, Y.G. Fu, H.T. Lu, A novel image watermarking scheme based on support vector regression, J. Syst. Softw. 78 (2005) 1–8. [17] M. Kutter, F. Jordan, F. Bossen, Digital signature of color images using amplitude modulation, J. Electron. Imag. 7 (2) (1998) 326–332. [18] H. Peng, J. Wang, W. Wang, Image watermarking method in multiwavelet domain based on support vector machines, J. Syst. Softw. 83 (2010) 1470–1477. [19] A.J. Smola, B. Scholkopf, A Tutorial on Support Vector Regression, NeuroCOLT Technical Report NC-TR-98-030, Royal Holloway College, University of London, UK, 1998. [20] G.B. Huang, Q.Y. Zhu, C.K. Siew, Extreme learning machine: theory and applications, Neurocomputing 70 (2006) 489–501. [21] Q.Y. Zhu, A. Qin, P. Suganthan, G.B. Huang, Evolutionary extreme Learning Machine, Neurocomputing 70 (2005) 1759–1763. [22] G.B. Huang, L. Chen, C.K. Siew, Universal approximation using incremental constructive feed forward networks with random hidden nodes, IEEE Trans. Neural Netw. 17 (4) (2006) 879–892. [23] G.B. Huang, L. Chen, Enhanced random search based incremental extreme learning machine, Neurocomputing 71 (16–18) (2008) 3460–3468. [24] G.B. Huang, L. Chen, Convex incremental extreme learning machine, Neurocomputing 70 (2007) 3056–3062. [25] G.B. Huang, D.H. Wang, Y. Lan, Extreme learning machines: a survey, Int. J. Mach. Learn. Cybern. 2 (2011) 1107–1122. [26] G. Feng, G.B. Huang, Q. Lin, R. Ray, Error minimized extreme learning machine with growth of hidden nodes and incremental learning, IEEE Trans. Neural Netw. 20 (8) (2009) 1352–1357. [27] G.B. Huang, H. Zhou, X. Ding, Rui. Zhang, Extreme learning machine for regression and multiclass classification, IEEE Trans. Syst. Man Cybern. 42 (2) (2012) 513–529. [28] Y. Ye, S. Squartini, F. Piazza, Online sequential extreme learning machine in nonstationary environments, Neurocomputing 116 (2013) 94–101. [29] H.H. Tsai, D.W. Sun, Color image watermark extraction based on support vector machine, Inf. Sci. 177 (2007) 550–569. [30] F. Sebastiani, Machine learning in automated text categorization, ACM Comput. Surv. (CSUR) 34 (1) (2002) 1–47.

[31] L.J. Cao, Support vector machines experts for time series forecasting, Neurocomputing 51 (2003) 321–339. [32] R. Singh, S. Balasundaram, Application of Extreme Learning Machine method for time series analysis, Int. J. Intell. Technol. 2 (4) (2007) 256–262. [33] W. Zong, G.B. Huang, Y. Chen, Weighted extreme learning machine for imbalance learning, Neurocomputing 101 (2013) 229–242. [34] K.A. Toh, Deterministic neural classification, Neural Comput. 20 (6) (2008) 1565–1595. [35] W. Deng, Q. Zheng, L. Chen, Regularized extreme learning machine, in: IEEE Symposium on Computational Intelligence and Data Mining (CIDM 2009), 2009, pp. 389–395. [36] C.R. Rao, S.K. Mitra, Generalized Inverse of Matrices and its Applications, Wiley, New York, 1971. [37] P.L. Bartlett, The sample complexity of patten classification with neural networks: the size of weights is more important than the size of the network, IEEE Trans. Inf. Theory 44 (2006) 525–536. [38] G.B. Huang, L. Chen, C.K. Siew, Extreme learning machine: a new learning scheme of Feedforward networks, in: Proceeding of International Conference on Neural Networks (IJCNN 2004), vol. 2, 2004, pp. 985–990. [39] N.Y. Liang, G.B. Huang, P. Saratchandran, N. Sundarajan, A fast and accurate online sequential learning algorithm for feedforward networks, IEEE Trans. Neural Netw. 17 (6) (2006) 1411–1423. [40] E.K.P. Chong, S.H. Zak, An Introduction to Optimization, Wiley, New York, 2001. [41] G.H. Golub, C.F. van Loan, Matrix Computation, third ed., The Johns Hopkins University Press, Baltimore, Maryland, 1996. [42] C.C. Liu, H.H. Tsai, Wavelet-based image watermarking with visibility range estimation based on HSV and neural networks, Pattern Recognit 44 (4) (2011) 751–763.

Ram Pal Singh received the B.Sc. degree in Electronic Science with Basic Sciences and the M.Sc. degree in Electronic Sciences from University of Delhi, Delhi, India in 1990 and 1992, respectively, and the M.Tech degree in Computer Science and Engineering in 1998. He has received the Ph.D. in Computer Science from the School of Computer & Systems Sciences, Jawaharlal Nehru University, New Delhi, India. He joined in 2009 at Institute for Informatics, Freie University, Berlin, Germany for Post Doctorate. He is currently an Associate Professor in Department of Computer Science, DDUC, University of Delhi, Delhi, India. His current research interests include Support Vector Machine (SVM), extreme learning machine, pattern recognition, signal processing, information hiding, machine learning. Neelam Dabas received the B.Sc. degree in Computer Science from University of Delhi, Delhi, India in 2004 and the Master in Computer Application (MCA) from Banasthali Vidyapeeth University, India in 2007. She was then associated with HCL Technologies Inc. as Lead and Design Engineer for software development from 2007 to 2011. Currently she is a research scholar in Department of Computer Science, University of Delhi and doing her Ph.D. Her current research interests include, extreme learning machine, information hiding, machine learning.

Vikash Chaudhary received M.Tech degree in Computer Science and Engineering in 1998 and currently an Assistant Professor in department of computer science, BN College, University of Delhi, Delhi. He is currently pursuing Ph.D. in Computer Science and his research interests include multi media digital watermarking/ information hiding, image processing.

Nagendra received Master in Information Technology (MIT) degree in Computer Science in 1998 and currently an Assistant Professor in Department of Computer Science, BN College, University of Delhi, Delhi. He is currently associated with research activities in computer science and his research interests include pattern recognition, time series analysis, information hiding, image processing.

Please cite this article as: R.P. Singh, et al., Online Sequential Extreme Learning Machine for watermarking in DWT domain, Neurocomputing (2015), http://dx.doi.org/10.1016/j.neucom.2015.03.115i