- Email: [email protected]
Stacked Fisher autoencoder for SAR change detection
Stacked Fisher Autoencoder for SAR Change Detection
Accepted Manuscript
Stacked Fisher Autoencoder for SAR Change Detection Ganchao Liu, Lingling Li, Licheng Jiao, Yongsheng Dong, Xuelong Li PII: DOI: Article Number: Reference:
S0031-3203(19)30274-2 https://doi.org/10.1016/j.patcog.2019.106971 106971 PR 106971
To appear in:
Pattern Recognition
Received date: Revised date: Accepted date:
11 January 2019 17 May 2019 15 July 2019
Please cite this article as: Ganchao Liu, Lingling Li, Licheng Jiao, Yongsheng Dong, Xuelong Li, Stacked Fisher Autoencoder for SAR Change Detection, Pattern Recognition (2019), doi: https://doi.org/10.1016/j.patcog.2019.106971
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
1 Highlights • The original SAE is expanded to suit with the multiplicative noise in SAR change detection.
CR IP T
• The features extracted by SFAE are more discriminative than the original stacked autoencoder due to that Fisher discriminant criterion is incorporated into SFAE.
• Experiments on the simulated and real SAR datasets reveal that the proposed SFAE algorithm is effective on multitemporal single/multi-polarization SAR change detec-
tion. Specifically, the proposed SFAE method is obviously superior to the real-time
AN US
methods on detection accuracy and the non-realtime methods on computational
AC
CE
PT
ED
M
complexity.
ACCEPTED MANUSCRIPT
Stacked Fisher Autoencoder for SAR Change Detection Ganchao Liua,b , Lingling Lib , Licheng Jiaob , Yongsheng Dongc,∗, Xuelong Lia a Center
CR IP T
for OPTical IMagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University, Xi’an, China b School of Artificial Intelligence, Xidian University, Xi’an, China c School of Information Engineering, Henan University of Science and Technology, Luoyang, China
Abstract
Stacked autoencoder is effective in image denoising and classification when it is used for
AN US
synthetic aperture radar (SAR) change detection. However, the resulting features may
not be discriminative enough for in some sense. To alleviate this problem, in this paper we propose a stacked Fisher autoencoder (SFAE) for SAR change detection. Specifically, in the framework of SFAE, unsupervised layer-wise feature learning and supervised finetuning are jointly performed when training the network. The trained network can be used to detect the changes in both of the single and multi-polarization SAR datasets
M
in real-time. The proposed SFAE has two advantages. The first one is to expand the stacked autoencoder to suit the environment with the multiplicative noise in SAR change
ED
detection. The second is that the features extracted by SFAE are more discriminative than the original stacked autoencoder due to that Fisher discriminant criterion is incorporated into SFAE. The results on the simulated and real SAR datasets indicate that the proposed
PT
SFAE algorithm has a significant advantage on multitemporal single/multi-polarization SAR (SAR/PolSAR) change detection.
CE
Keywords: Stacked Fisher autoencoder (SFAE), synthetic aperture radar (SAR),
AC
change detection, stacked autoencoder (SAE), Fisher criterion.
∗ Corresponding
author Email addresses: [email protected] (Ganchao Liu ), [email protected] (Lingling Li ), [email protected] (Licheng Jiao ), [email protected] (Yongsheng Dong), [email protected] (Xuelong Li )
Preprint submitted to Pattern Recognition
July 19, 2019
ACCEPTED MANUSCRIPT
1 INTRODUCTION
3
1. Introduction Since the number of satellites for earth observation is on the increase, there is a growing interest in the analysis of images acquired on the same geographical area at different
CR IP T
times. As a high resolution earth observation system with the capacity of all-weather and all-time, multitemporal SAR images change detection has been used in a wide variety of applications, such as agricultural surveys, urban planning and disaster management.
However, because of the coherent nature of the scattering phenomenon, SAR images are inherently affected by speckle noise, which makes the automatic change analysis difficult. Therefore, the reduction of the speckle interference becomes an important issue in SAR
AN US
change detection.
Generally, SAR change detection tends to be carried out through the following three steps: despeckling, generating difference images, and classification. In the past few decades, many creative works have been done in these respects. In the first step, the classical despeckling methods, such as Lee filter[1], the probabilistic patch-based (PPB) filter [2] and the Pretest filter[3], are widely used. In the second step, to lower the impact
M
of speckle noise, the log-ratio operator is widely applied in difference images generation [4–7]. By measuring the similarities of the multitemporal SAR images, many methods
ED
[8–16] are proposed to generate difference images based on statistical distributions. In [10] and [9], the similarity is measured by the cumulant-based Kullback-Leibler divergence (CKLD) and bivariate Gamma distributions respectively. In [11], a test statistic based on
PT
the complex Wishart distribution is used for the change detection in Polarimetric SAR (PolSAR) data. In the third step, a threshold is usually set for the previous algorithms based on statistical distributions. On the other hand, as a binary classification prob-
CE
lem, change detection can be performed by using the common clustering or classification methods, such as k-means [17, 18], fuzzy c-means (FCM) [19, 20], spectral clustering [21]
AC
and so on. It can be easily found that the detection methods with speckle reduction algorithms [22–24] are obviously more effective than those without the pre-filter procedure [10, 11, 25]. However, those despeckling-based detection methods are time-consuming, and are not suitable in the age of big data any more. Inspired by the computational models of the biological brain, deep learning [26, 27] has become a powerful method in analyzing big data. In deep learning, the output of
ACCEPTED MANUSCRIPT
1 INTRODUCTION
4
former layer is the input of the latter layer. The most commonly used deep neural network models mainly include deep belief networks (DBNs) [26], convolutional neural networks (CNNs) [28], and stacked autoencoders (SAE) [29]. Deep learning has been applied in handwriting recognition [30], image classification [31], denoising [32], etc. More recently,
CR IP T
the deep neural network has even been used in multispectral and SAR image change detection[33–37]. In [33], the supervised labels are replaced by a novel unsupervised
discriminator network. In [34], the spectral-spatial-temporal features of the multispectral
imagery are extracted by a new recurrent convolution neural network. In [35], a novel network with restricted Boltzmann machine (RBM) has been introduced to solve the
change detection problem. However, this detection method is not robust enough for
AN US
different SAR datasets, since the performance of the networks are heavily relying on the selecting samples.
As an unsupervised neural network, stacked autoencoder is widely used in image denoising [29], classification [38] and natural language recognition [39]. Inspired by the good performance on image denoising and classification, the use of SAE in SAR change
M
detection has a promising future since the difficulty of SAR change detection is to reduce the influence of the inherent speckle interference. In this paper, we propose a stacked Fisher autoencoder by incorporating Fisher discriminant criterion for SAR change detec-
ED
tion. In the framework of SFAE, unsupervised layer-wise feature learning and supervised fine-tuning are jointly performed when training the network. The trained auto-encoder is suitable for different datasets with the different equivalent number of looks. In practice,
PT
to save the time spent in computation, the network can be trained in advance. The contributions of this paper are as follows. First, the SAE network is expanded
CE
to suit the environment with the serious multiplicative noise in SAR change detection. Second, the features extracted by SFAE are more discriminative than the original stacked autoencoder due to that Fisher discriminant criterion is incorporated into SFAE. Third,
AC
experiments on the simulated and real SAR datasets reveal that the proposed SFAE algorithm is effective on multitemporal single/multi-polarization SAR change detection. Specifically, the proposed SFAE method is obviously superior to the real-time methods on detection accuracy and the non-realtime methods on computational complexity. This paper is organized in five sections. In section II, the Polarimetric SAR speckle
ACCEPTED MANUSCRIPT
2 SAR SPECKLE STATISTICS AND MOTIVATIONS
5
statistics and the motivations of our method are briefly introduced. The methodology is discussed in Section III. In Section IV, the results and discussions of the experiment on simulated images and real SAR dataset are given. Finally, the conclusions are given in
CR IP T
the last section. 2. SAR Speckle Statistics and Motivations 2.1. SAR System Introduction
For the full polarimetric SAR system, the target is measured in two polarization modes: horizontal and vertical. Combining the linear receive and transmit polarizations, the target can be characterized using the scattering matrix[40]:
AN US
S=
Shh
Shv
Svh
Svv
,
(1)
where Shv is the scattering element of horizontal transmitting and vertical receiving polarization, and the other three elements are defined similiarly. In the case that the transmitter and receiver antennas are the same, the non-diagonal elements are reciprocal, i.e.,
M
Shv = Svh . Then, the full polarimetric SAR data can be characterized with the following covariance matrix:
D E 2 |Shh | √ ∗ C= 2Shv Shh ∗ hSvv Shh i
PT
ED
√
∗ 2Shh Shv D E 2 2 |Shv |
√ ∗ 2Svv Shv
∗ i hShh Svv
√ ∗ 2Shv Svv D E 2 |Svv |
,
(2)
where the superscript “*” denotes complex conjugation, h·i stands for the average value.
CE
The measured value of the scattering is presented in the plural form, which contains
both the amplitude and phase information. The visual single polarization SAR images is shown as the amplitude A or intensity information I. Among them, I = A2 . Due
AC
to the coherent interference of waves reflected from many elementary scatters, speckle is inevitablely appearing in SAR images. The diagonal terms of C are the intensities of linear polarizations and can be representeds by the following multiplicative noise model[41].
I = RnI ,
(3)
ACCEPTED MANUSCRIPT
2 SAR SPECKLE STATISTICS AND MOTIVATIONS
Train Part
Test Part
Labeled image2
Input image1
Difference image
Input image2
Difference image
Training the labeled dataset
CR IP T
Labeled image1
6
Trained network
Trained network
Change map
AN US
Figure 1: The change detection framework with neural network.
where I represents a measured intensity value, R is the corresponding actual intensity value, nI is the speckle noise in intensity format and independent of R. In this paper, the full-polarization SAR data is seen as three independent intensity SAR images in change detection.
M
Under the assumption of fully developed speckle, an L-look intensity and amplitude
ED
SAR images obey the Gamma and Nakagami distribution, respectively [42]. 1 p(I|R) = Γ(L)
PT
2 p(A|R) = Γ(L)
L R
L R
L
L
LI exp − R
LA2 exp − R
I L−1 ,
(4)
A2L−1 ,
(5)
where Γ(·) is the gamma function. When L=1, Eqs. (4) and (5) denote the distributions
CE
of single-look intensity and amplitude SAR data, which corresponding to the exponential and Rayleigh distributions respectively. In this paper, SAR images with amplitude format
AC
will be discussed as an example. 2.2. Problems and Motivations Since the SAR/PolSAR images suffer from the inherent speckle noise, the speckle re-
duction was pointed out as a major topic for the development of a successful detection scheme [43–45]. Therefore, it is necessary to develop a robust SAR image change detection technique against the speckle noise [46]. To reduce the influence of speckle noise
ACCEPTED MANUSCRIPT
2 SAR SPECKLE STATISTICS AND MOTIVATIONS
w(d2)
...
(2)
...
h
w(n-1) ( n-1)
w(dn-1)
...
h
J
(n)
(n)
Stacked Auto-encoder
θ ~ C
Logistic Classifier
M
C
y
( n-1)
) w (n d
...
h
y
(2)
AN US
w(n)
y
CR IP T
J
w ( 2)
(1)
...
...
(1)
y
...
w(d1)
...
h
...
w
J
(1)
...
x
7
Figure 2: The flowchart of stacked Fisher autoencoder for SAR change detection.
ED
theoretically, the techniques with log-ratio operator [4, 6], speckle reduction [23, 47] and Markov random fields [22, 48] are often used to improve the efficiency of change detection.
PT
In this paper, the "seriousness" of the speckle is associated with the equivalent number of looks (ENL). The smaller the ENL is, the more serious the speckle is. The techniques without pre-processing, such as [11, 12, 25], which are not suitable for the cases when the
CE
speckle is serious. For example, the ENLs of the datasets tested in [11, 12, 25] are 13, 59 and 109, respectively.
AC
More recently, the neural network based methods are proposed to obtain the change
maps of multitemporal remote-sensing images [35–37]. As a well-known deep architecture, stacked autoencoder has been proved to be effective in denoising [32] and feature extraction [38]. Though the white Gaussian noise is very different from the SAR speckle pattern, many SAR despeckling algorithms are expanded from the filters based on white Gaussian noise models. Thus, the ideal of the stacked autoencoder is worth exploring.
ACCEPTED MANUSCRIPT
3 METHODOLOGY
8
Inspired by these characters, a novel change detection framework with autoencoder network can be designed to deal with the massive amounts of multitemporal SAR/PolSAR images. In the age of big data, SAR image change detection faces new opportunities and
CR IP T
challenges. On the one hand, the processing of the massive volumes of data is very time-consuming. On the other hand, the big data makes the features learning in deep architecture feasible. To achieve this goal, we should tackle two important issues at least.
Firstly, the autoencoder network should be expanded to suit the environment with multiplicative speckle noise in SAR system. Secondly, the neural network should be modified
to meet with the task of change detection. In other words, the features extracted from
AN US
the SAE network should be discriminative to classify the changed and unchanged pixels in SAR images. In order to make the features extracted by the network discriminative, the concept of linear discriminant analysis has been widely used in recognition [49] and
change detection [50], which give us a great inspiration. Inspired by these works, we intend to make the features more discriminative by introducing a discriminant criterion.
M
In the next section, these issues will be discussed in details. 3. Methodology
ED
In this section, we first introduce the stacked autoencoder briefly, and then propose a stacked Fisher autoencoder (SFAE) for SAR change detection. Our proposed SFAE is learnt from the simulated or labeled dataset using the backpropagation algorithm. The
PT
change detection framework with neural network is shown in Fig.1.
CE
3.1. A brief description of Stacked Autoencoder In this subsection, a brief description of the standard stacked autoencoder framework
is illustrated. As shown in Fig.2, the input vector x of the autoencoder is encoded to a
AC
hidden vector h in the first: h(1) = f (z(1) ) = f (w(1) x + b(1) ).
(6) (1)
(1)
Then, remapping the hidden vector h to a decoded vector y(1) = f (wd h(1) + bd ). Here, the superscript
(1)
represents the first layer. w(1) and b(1) are denote the encoder (1)
weight matrix and the bias vector of the first layer. wd
is the corresponding decoder
ACCEPTED MANUSCRIPT
3 METHODOLOGY
9 (1)
weight matrix of the first layer. Here, the decoder weight matrix wd
is set as the
transposition of the encoder weight matrix w(1) . In this paper, f is the sigmoid function f (t) = 1/(1 + exp(−t)) [51]. The parameters {w, b} are optimized to minimize the
CR IP T
reconstruction error [29]:
(7)
JAE (w, b) = − log p(x|y) + λ kwk ,
where λ is the weight decay parameter. As shown in eq.(7), the definition of JAE (w, b) includes two terms. The former one denotes the reconstruction error. The latter one is
a regularization term used to prevent overfitting. A stacked autoencoder network can be
AN US
seen as a multi-layer autoencoder. In this network, the output of the former layer is the input of the latter layer.
3.2. Stacked Fisher Autoencoder for SAR Change Detection
For the co-registered images with multiplicative speckle noise, the ratio difference image patch is used as the input layer, i.e., x =
A1 A2 .
A1 and A2 are the observed amplitude
SAR images patch obtained at different times. Under the unchanged hypothesis, the ratio
M
distribution with different number of looks is given in [23, 52]
ED
p(xk |R1 = R2 ) ∝
xk2L1 −1 , 1 2 L1 +L2 (L L2 xk + 1)
(8)
where xk is the k-th pixel of the ratio image patch. R1 and R2 are the corresponding
PT
reflectivity patches without noise. L1 and L2 are the looks of the corresponding observed SAR image patches.
Stacked autoencoder for SAR:
CE
Substituting equation (8) into (7), a new cost function of autoencoder suit for SAR
AC
images can be obtained as follows:
2
JAE−SAR (w, b) = − log p(x|y) + λ kwk " ! 2 # m s (9) 1 XX L1 xk,j xk,j 2 ∝ γ log + 1 − log + λ kwk , m ˜ j=1 L2 yk,j yk,j k=1
where m ˜ = m/(2L − 1), m is the number of the train samples, s is the size of each sample
patches, xk,j denotes the k-th pixel of the j-th sample patch. γ =
L1 +L2 2L1 −1
is a constant, λ
ACCEPTED MANUSCRIPT
3 METHODOLOGY
10
is the decay parameter. For the stacked autoencoder, the features of the l -th layer (l =1...,n), i.e., the hidden
JSAE−SAR (w(l) , b(l) ) = − log p(x(l) |y(l) ) + λ ∝
m X s X
j=1 k=1
γ log L1 L2
(l−1) hj,k (l) yj,k
!2
l+1
X
(i) 2
w i=l
+ 1 − log
Fisher discrimination criterion:
CR IP T
layer h(l) , is the input of the next layer x(l+1) .
(l−1)
hj,k
(l)
yj,k
!
+λ
l+1
X
(i) 2
w . i=l
(10)
AN US
In this paper, the modified SAE network is trained to detect the changes in SAR images. In [49] and [50], the applications of linear discriminant analysis have been investigated for recognition and change detection. In order to make the features extracted by SAE network more discriminative, the concept of Fisher discriminant criterion [53] is
introduced. Based on the Fisher discrimination criterion, the discrimination capability of
M
the SAE network can be improved by minimizing the within-class scatter of h, denoted by Sw (h), and maximizing the between-class scatter of h, denoted by Sb (h). X
(hi − c1 )(hi − c1 )T +
ED
Sw (h) =
i∈N1
X
i∈N2
(hi − c2 )(hi − c2 )T
,
(11)
T
Sb (h) = (c1 − c2 ) (c1 − c2 )
PT
where c1 and c2 are the mean vector of the changed and unchanged features, respectively, P P i.e., c1 = m11 i∈N1 hi , c2 = m12 i∈N2 hi . N1 and N2 are the set of changed and
unchanged sites respectively. m1 and m2 are the number of samples in their corresponding
CE
classes. In practice, the Fisher criterion is usually defined as the minimizing of the ratio trace T r (Sw (h)) /T r (Sb (h)) [54] or the minimizing of the difference trace T r (Sw (h)) −
AC
T r (Sb (h))[55], where T r(·) is the trace of matrix. The relationship between the two
types of Fisher criterions have been discussed in [55]. In this paper, the difference trace version of the Fisher criterion will be added in the objective function, while the ratio trace version of the Fisher criterion will be used as a criterion to evaluate the network in the experiments. The experiments will verify the relationship between the two types of Fisher criterions.
ACCEPTED MANUSCRIPT
3 METHODOLOGY
11
When considering the difference trace T r (Sw (h)) − T r (Sb (h)), the term −T r (Sb (h))
will make the objective function non-convex. According to the suggestions of [55], an 2
elastic term khk2 is added in our Fisher function ϕ(h): 2
(12)
where η is a weight decay parameter.
CR IP T
ϕ(h) = tr(Sw (h) − Sb (h)) + η khk2 ,
Combining the Eqs. (10) and (12), we have the following objective function of the Stacked Fisher Autoencoder model (SFAE). JSF AE (w(l) , b(l) )
1 = m ˜
j=1 k=1
γ log L1 L2
i=l
(l−1)
hj,k
(l) yj,k
l+1
X
(i) 2
w + λ2 ϕ(h(l) ), i=l
!2
+ 1 − log
(l−1)
hj,k
(l) yj,k
!
(13)
M
+ λ1
s m X X
l+1
X
(i) 2
w + λ2 ϕ(h(l) )
AN US
= − log p(x(l) |y(l) ) + λ1
where λ1 and λ2 are the weight decay parameters. In the following description, the SFAE network will be optimized by the back-propagation
ED
algorithm.
Firstly, the activations throughout the network h(l) = f (z(l) ), l = 2, 3, ..., n (the (l)
δi
PT
output layer), are computed by the feedforward algorithm. Then, the "residual error" is computed to measure the difference between the network activations [51]. (l)
CE
According to the Appendix, the "residual error" δi
AC
(l+1) δi
in the decode layer l is
"
! # 2 (l−1) L1 h(l−1) h (l) = (l+1) γ log + 1 − log + λ2 ϕ(h ) L2 y(l) y(l) ∂zi 2 (l) (l−1) (L + L ) h + L2 yi 1 2 i −2γL1 (l+1) 0 . = 2 2 f zi L2 (l−1) (l) (l) L1 hi yi + L2 yi ∂
(14)
ACCEPTED MANUSCRIPT
3 METHODOLOGY
12
For all of the nodes in the layer l, the "residual error" vector can be rewritten as:
δ
(l)
where,
=
w
(l)
T
δ
(l+1)
(l)
(l)
+ λ2 q(h ) + 2ηh
•f
0
z(l) ,
(15)
CR IP T
h i (l) (c1 −c2 ) 1 − 2 h − c 1 − 1 i m1 m1 if, i ∈ {1, ..., m1 }; (l) q(hi ) = h i (l) 2) 2 hi − c2 1 − m12 − (c1m−c 2 if, i ∈ {m1 + 1, ..., m}.
The operator "•" denotes the element-wise product, also called the Hadamard product.
AN US
Furthermore,
m ∂ 1 X (l+1) (l) T (l) (l) + λ1 w(l) , δ · hk J(w , b ) = m ∂w(l) k=1
∂ 1 J(w, b) = δ (l+1) . m ∂b(l)
(16)
(17)
M
Then, the parameters are updated to minimize the objective function.
ED
Pm (l+1) (l) T 1 (l) (l) (l) w = w − α m k=1 δ · hk + λ1 w , b(l) = b(l) − α 1 δ (l+1) . m
(18)
The main procedure of the stacked Fisher autoencoder algorithm is presented in Al-
PT
gorithm 1.
Fine tune:
CE
For the stacked Fisher autoencoder, supervised fine-tuning is commonly used to im-
prove the performance of classification. In this paper, the supervised logistic classifier is used to classify the changed and unchanged pixels. The expression of the logistic classifier
AC
is
e Θ (h(n) ) = C
1 , 1 + e−ΘT h(n)
e is the label estimated by the logistic classifier. where Θ is the model parameter, C
(19)
Then, we need to fine-tune the model parameters (w, b, Θ) by minimizing the objec-
ACCEPTED MANUSCRIPT
13
CR IP T
3 METHODOLOGY
AN US
Algorithm 1 Stacked Fisher Auto-Encoder (SFAE) algorithm Input: Train samples x Task: Train the network by minimizing the objective function J(w(l) , b(l) ). Parameters: network parameters (w, b), learning rate α, weight decay parameters λ1 , λ2 and η Initialization: Initialize the parameters w=0, b=0. 1. Compute the activations by the feedforward algorithm: h(l) = f (z(l) ) = f (w(l) h(l−1) + b(l) ),
y(l) = f (z(l+1) ) = f (w(l+1) h(l) + b(l+1) ). 2. For the decode layer l +1, set
2 0 −2γL1 (L1 + L2 ) h(l−1) + L2 y(l) (l+1) . = 2 2 • f z L2 L1 h(l−1) y(l) + L2 y(l)
M
δ
(l+1)
ED
For the hidden layer l, set T 0 δ (l) = w(l) δ (l+1) + λ2 q(h(l) ) + 2ηh(l) • f z(l) . 3. Update the parameters:
1 (l+1) (l) T (l) δ h + λw , =w −α m 1 (l+1) (l) (l) . b =b −α δ m
PT CE
w
(l)
(l)
AC
4. Repeat the above steps until the objective function JSF AE (w(l) , b(l) ) does not change. Output: The parameters w(l) , b(l) ; The activation of the output hidden layer h(l)
ACCEPTED MANUSCRIPT
4 EXPERIMENTS AND RESULTS
14
e tive function J(w, b, Θ).
n+1
T 1 λ1 X
(i) 2 λ1 2 e e J(w, b, Θ) = − C−C h(n) Θ + kΘk ,
w + m 2 i=1 2
(20)
CR IP T
C is the ground truth label.
In conclusion, the training of the SFAE network for change detection includes two parts: 1) training the stacked Fisher autoencoder parameters w and b by minimize the
objective function JSF AE (w, b); 2) Fine-tuning the model parameters w, b and Θ by
e minimize the objective function J(w, b, Θ). The flowchart of the stacked autoencoder for SAR change detection is shown in Fig.2.
AN US
4. Experiments and Results
In this section, the proposed SFAE algorithm is trained on a synthetic dataset with supervised labels. The training dataset contains more than 200,000 image patches with the size of 5×5. In order to simulate the SAR image with speckle noise, the training dataset is corrupted by the multiplicative noise obeys Rayleigh distribution. For the
M
network with 25×36×36×25 architecture, the number of the samples in the training dataset is large enough. In this paper, for the different datasets with the similar ENL,
ED
the results of the proposed method are obtained by the same trained network. In the following subsections, the parameters selection and experiments on datasets are discussed.
PT
4.1. Parameters setting and the experiments on synthetic dataset To quantitative analysis the performance of the compared methods, three indices are introduced in this paper. The first one is the percentage correct classification (PCC)
CE
calculated by the true positives (TP, changed pixels that detected) and the true negatives
AC
(TN, unchanged pixels that detected):
P CC = (T P + T N )/N,
(21)
where N is the sum of the changed pixels (Nc) and the unchanged pixels (Nu). The second one is the Kappa coefficient: Kappa = (P CC − β)/(1 − β),
(22)
ACCEPTED MANUSCRIPT
4 EXPERIMENTS AND RESULTS
15
β = ((T P + F A) · N c + (M A + T N ) · N u) /N 2 . MA and FA are the short of missed alarms and false alarms respectively [19].
The last one is the operation time to evaluate the complexity of the compared algorithms.
CR IP T
In the training stage, the SFAE network is trained by proper parameters selection. Specifically, based on a great number of tests, the decay parameters are set with λ1 =
1 × 10−7 , λ2 = 0.02 and η = 0.1. Besides that, the number of the layers is suggested to set as three in this paper. In order to explain the superiority of the selected parameters,
the parameters λ1 , λ2 , η and the the selection of the number of the hidden layers are detailedly discussed in this subsection.
AN US
Fig. 3 presents the experiment on an L-band synthetic PolSAR data which is corrupted by speckle noise with L1 =L2 =4. Figs. 3(a), (b) and (c) are the Pauli images of the noisy simulated PolSAR dataset with the size of 128×128 and their corresponding standard change image. The hidden layers are visualized in Figs. 3(d)-(f). From the figure, we can see that as the number of layers increases, the feature becomes smoother and easier
M
to be classified. It implies that the SFAE model can effectively overcome the influence of speckle noise with the increase of layers. From Fig. 3(g) to (i) are the change images of the proposed SFAE with single hidden layer, two, and three hidden layers, respectively.
ED
From the Fig. 3(g), we can see that there are many missed alarms (MA) in the upper right changed square. While the missed alarms in the upper right square are decreased gradually in Figs. 3(h) and (i). The results on the synthetic dataset may imply that
PT
the performance of the SFAE network becomes more and more effective along with the
CE
increasing of the number of hidden layers.
In table 1, the ratio version of Fisher criteria (Ratio Fisher), i.e., T r (Sw (h)) /T r (Sb (h)),
AC
is used for assessing the detection performance of the SFAE algorithm with different number of layers. The Ratio Fisher coefficient of the features extracted by the SFAE network in each polarization channels are gradually decreased when the hidden layers increased from 1 to 3. However, the Ratio Fishers become worse when the numbers of hidden layers higher than three, which implies that the network is overfitting. From table 1, we can have the following conclusions. First, by minimizing the difference version of the Fisher
ACCEPTED MANUSCRIPT
16
(b) Noisy image 2
(d) Hidden layer 1
(e) Hidden layer 2
(c) Reference
AN US
(a) Noisy image 1
CR IP T
4 EXPERIMENTS AND RESULTS
ED
M
(f) Hidden layer 3
PT
(g) Single layer
(h) Two layers
(i) Three layers
AC
CE
Figure 3: Results on the simulated PolSAR datasets. (a) and (b) are the noisy Pauli images corrupted with L1 =L2 =4. (c) Reference image. From (d) to (f) are the visualization of the hidden layers. From (g) to (i) are the results of the proposed method with single hidden layer, two hidden layers, and three hidden layers, respectively. Table 1: Ratio Fisher of the features in each hidden layers
Layers 1 2 3 4 5
HH 0.3725 0.3211 0.2117 0.5554 8.7013
Ratio Fisher HV VV 2.5142 123.7999 2.1248 97.2205 1.3999 48.8651 3.7929 187.6981 60.8815 4034.7000
ACCEPTED MANUSCRIPT
4 EXPERIMENTS AND RESULTS
17
Table 2: Quantitative comparisons on the synthetic PolSAR data
MA 860 807 151 254 232 71 8
FA 648 1591 78 11 2 62 7
PCC(%) 90.80 85.36 98.60 98.50 98.57 99.19 99.91
Kappa 0.7496 0.5818 0.9574 0.9559 0.9611 0.9783 0.9976
Time(s) 0.22 0.41 0.69 0.31 0.38 0.49 13.75
CR IP T
Methods Wishart[11] PCD[12] SAE-3 SFAE-1 SFAE-2 SFAE-3 PPCD[23]
criteria in training the network, the features extracted by SFAE network become more
discriminative with the increasing of the hidden layers. Second, with the limited size of layer is suggested to set as three.
AN US
the training dataset used in this paper, to avoid overfitting, the number of the hidden
Table 2 presents a quantitative comparison between the proposed SFAE method and the latest algorithms on PolSAR change detection. The compared algorithms including the Wishart detector [11], the polarimetric change detector (PCD) [12], and the patch based polarimetric change detector (PPCD) [23]. In the experiments, the thresholds in
M
the Wishart detector [11] and PPCD [23] are 0.1 and 0.0002, respectively. The angle difference in the PCD algorithm is set as 40◦ , Besides that, we also made a comparison
ED
between the modified autoencoder and the unmodified one. In the experiments, both of the proposed SFAE network and the ordinary autoencoder are trained with the same parameters and training samples.
PT
From table 2, we can have the following conclusions: 1) Along with the increasing of the number of layers from 1 to 3, the performance of the SFAE network are gradually
CE
better, which has been demonstrated in Fig. 3 and table 1. 2) The proposed SFAE method can detect the changes in real-time. The operation time of the SFAE method is two orders of magnitude less than that in PPCD. 3) The performance of SFAE algorithm
AC
is obviously superior to the other real-time PolSAR change detection algorithms. 4) The results of the proposed SFAE are significantly outperformed than that of the ordinary autoencoder. It further manifests the improvement of the modification done in the SFAE method.
ACCEPTED MANUSCRIPT
4 EXPERIMENTS AND RESULTS
18
1
CR IP T
2
(b) Image obtained in March 2009
AN US
(a) Image obtained in June 2006
Figure 4: The Wuhan dataset.
4.2. Experiments on single-polarization SAR images
In this subsection, a real calibrated single-polarization SAR dataset is tested by the compared methods. The compared algorithms including the principal component analysis and RBM detector [35].
M
(PCA) detector [17], MRFFCM [19], Matching Pursuit (MP) detector [47], CKLD [10]
ED
In the dataset, there are two four look L-band SAR images obtained by ALOS PALSAR with 10-m resolution. The two images are the size of 10000×10000 pixels and sensed over the city of Wuhan, China in June 2006 and March 2009. The SAR image pair of
PT
Wuhan dataset are displayed in Fig. 4. The Wuhan dataset has a rich nature scene, including the urban area, farmland, rivers and lakes. In the experiments, two representative regions have been chosen for detailly descriptions. The selected Region 1 and 2
CE
are located around the Yangtze river and Donghu lake as marked in Fig. 4(a) respectively. The size of the marked regions are 500×500. The detection results of the marked
AC
region 1 are presented in Fig. 5. The main changes in Region 1 are the Yangtze river bridge under construction and the river bank. The enlarged images of the marked Region 1 are displayed in Figs. 5(a) and (b). To compare the performance of the mentioned algorithms, the log-ratio image is used as the reference in this paper. The corresponding log-ratio image of the marked region 1 is presented in Fig. 5(c). In the log-ratio image, the brighter the region is, the higher changed probability is. From Figs. 5(d) to (i) are
ACCEPTED MANUSCRIPT
19
AN US
CR IP T
4 EXPERIMENTS AND RESULTS
(b) March 2009
(c) Log-ratio
M
(a) June 2006
(e) MRFFCM
(f) CKLD
(h) RBM
(i) SFAE
CE
PT
ED
(d) PCA
AC
(g) MP
Figure 5: Results of Region 1. (a) Image obtained in June 2006. (b) Image obtained in March 2009. (c) Log-ratio image. From (d) to (i) are the results of PCA, MRFFCM, CKLD, MP, RBM, and our SFAE.
ACCEPTED MANUSCRIPT
20
AN US
CR IP T
4 EXPERIMENTS AND RESULTS
(b) March 2009
(c) Log-ratio
M
(a) June 2006
(e) MRFFCM
(f) CKLD
(h) RBM
(i) SFAE
CE
PT
ED
(d) PCA
AC
(g) MP
Figure 6: Results of Region 2. (a) Image obtained in June 2006. (b) Image obtained in March 2009. (c) Log-ratio image. From (d) to (i) are the results of PCA, MRFFCM, CKLD, MP, RBM, and our SFAE.
ACCEPTED MANUSCRIPT
4 EXPERIMENTS AND RESULTS
21
Table 3: Comparisons of the operation times(second) on SAR change detection
Region 1 1.97 26.01 114.42 174.15 122.91 1.10
Region 2 2.16 25.46 115.08 139.71 124.61 1.12
CR IP T
Time(s) PCA[17] MRFFCM[19] CKLD[10] MP[47] RBM[35] SFAE
the results of PCA, MRFFCM, CKLD, MP, RBM, and our SFAE, respectively. Among them, both of the results of PCA and MRFFCM shown in Figs. 5(d) and (e) have many false alarms. While the result of CKLD presented in figure 5(f) has numerous missed
AN US
alarms. As displayed in the bottom of Fig. 5, the bridge under construction and the river bank marked in the Fig. 5(b) are clearly detected by the MP, RBM and SFAE algorithms and the false alarms are obviously fewer than those in the results of PCA, MRFFCM. The similar conclusion can be got from the experiments on the marked Region 2.
The results on Region 2 are presented in figure 6. From figure 6(c), we can see that the
M
changes are mainly in the upper right of Donghu Lake, i.e., the area marked with a red rectangle in Fig. 6(b),where is the new Wuhan railway station for the high-speed trains. The results shown in Figs. 6(d) and (e) seem to have a lot of false alarms. By contrast,
ED
the other four results are comparable. The operation times (unit second) of the related algorithms are compared in table 3.
PT
All of the experiments are implemented with an Intel Core i3 3.2GHz. From table 3, we can see that the computation time of our SFAE is the least, followed by PCA, MRFFCM, CKLD, RBM and MP. The computation time of the SFAE method is less than those of
CE
CKLD, RBM and MP with two orders of magnitude. 4.3. Experiments on PolSAR images
AC
In this subsection, a real L-band PALSAR PolSAR dataset is tested. The PolSAR
dataset contains two PolSAR images with four-look processing which are sensed by ALOS at the area of Tokyo, Japan. The former one is obtained in July 2006 and the latter one is obtained in April 2009. In Fig. 7, the corresponding Pauli images are displayed. The size of the two images are 2290×1050 pixels. Two representative regions with the size of
300×200 are marked in Fig. 8(a).
ACCEPTED MANUSCRIPT
22
M
3
AN US
CR IP T
4 EXPERIMENTS AND RESULTS
AC
CE
PT
ED
4
(a) July 2006
(b) April 2009
Figure 7: The Tokyo dataset.
ACCEPTED MANUSCRIPT
5 CONCLUSIONS
23
Table 4: Comparisons of the operation times(second) on PolSAR change detection
Region 3 0.41 1.30 42.83 0.80
Region 4 0.26 1.33 43.13 0.81
CR IP T
Time(s) Wishart[11] PCD[12] PPCD[23] SFAE
The enlarged images of the Region 3 are presented in Figs. 8(a) and (b). In the
pseudo-color Pauli images, the urban areas are colored in pink or green. Because of the seasonal changes, the summer plants (Fig. 8(a)) are shown in red while the spring
plants (Fig. 8(b)) are shown in purple. To compare the performance of the mentioned
AN US
algorithms, the alpha and entropy ratio images are used as the reference in this paper. The corresponding alpha and entropy ratio images of the marked region 3 are presented in figures 8(c) and (d). In the ratio images, the brighter the region is, the higher changed probability is. The results of Wishart and PCD displayed in figures 8(e) and (f) have a lot of false alarms. In contrast, the false alarms in Figs. 8(g) and (h) are much smaller
than that of the others. The similar conclusions can be reached in the experiments as
M
shown in Fig. 9. From Fig. 9, we can see that the changed areas are clearly reflected in the ratio images. The changed areas have been detected by the Wishart detector and
ED
PCD with lots of false alarms. From the results presented in Figs. 8 and 9, we can see that the change detection results of the PPCD and the proposed SFAE are significantly better than that of the Wishart detector and PCD.
PT
Table 4 compares the computation times (s) of the related algorithms. From table 4, we can find that all of the algorithms except PPCD have a fast computational speed.
CE
The computation time of the PPCD is about 50 times more than that of the proposed SFAE. Above all, when compared with the Wishart detector, PCD and the PPCD, the
AC
proposed SFAE has a significant advantage on PolSAR change detection. 5. Conclusions In this paper, a new SFAE change detection algorithm is proposed for multitempo-
ral SAR systems. In the proposed SFAE method, the network is trained on the labeled synthetic dataset with supervision and tested on the unlabeled dataset without supervision in real-time. In this paper, the original stacked autoencoder is expanded to suit
ACCEPTED MANUSCRIPT
24
(b) April 2009
(c) Alpha Ratio
(d) Entropy Ratio
(g) PPCD
(h) SFAE
PT
ED
M
(a) July 2006
AN US
CR IP T
5 CONCLUSIONS
(e) Wishart
(f) PCD
AC
CE
Figure 8: Results of Region 3. (a) Image obtained in July 2006. (b) Image obtained in April 2009. (c) Alpha ratio image. (d) Entropy ratio image. From (e) to (h) are the results of Wishart, PCD, PPCD, and our SFAE.
ACCEPTED MANUSCRIPT
25
(b) April 2009
(c) Alpha Ratio
(d) Entropy Ratio
(g) PPCD
(h) SFAE
PT
ED
M
(a) July 2006
AN US
CR IP T
5 CONCLUSIONS
(e) Wishart
(f) PCD
AC
CE
Figure 9: Results of Region 4. (a) Image obtained in July 2006. (b) Image obtained in April 2009. (c) Alpha ratio image. (d) Entropy ratio image. From (e) to (h) are the results of Wishart, PCD, PPCD, and our SFAE.
ACCEPTED MANUSCRIPT
5 CONCLUSIONS
26
the environment with the multiplicative noise in SAR change detection. By introducing the Fisher discrimination, the features extracted by SFAE are more discriminative than the original stacked autoencoder. The results on the simulated and real datasets indicate that the proposed SFAE method has a significant advantage on detecting the changes
CR IP T
of multitemporal SAR/PolSAR systems. As a result, the proposed algorithm is a very
practical and efficient change detection method on dealing with the massive amounts of multitemporal SAR/PolSAR images. In this paper, only two temporal SAR images have
been utilized in the SFAE framework. In the future work, the features of the image sequences should be exploited to improve the performance of change detection for three or
Appendix
AN US
more temporal SAR images.
(a) The Derivation of the Equation (14)
=
∂ (l+1) ∂zi
γ log
L1 L2
h(l−1) y(l)
2
+1
!
− log
h(l−1) y(l)
(l) 2 +λ2 tr(Sw (h ) − Sb (h )) + η h 2 ! 2 sl (l−1) (l−1) X h L ∂ 1 k − log hk γ log + 1 = (l+1) (l) (l) L2 ∂zi yk yk k=1 2 sl (l−1) (l−1) X h ∂ L1 hk k + 1 − log = (l+1) γ log (l+1) L2 f z (l+1) ∂z f z (l)
(l)
PT
ED
"
M
(l+1) δi
i
k=1
k
(l−1)
k
2
(l)
CE
+ L2 yi −2γL1 (L1 + L2 ) hi (l+1) 0 , = 2 2 f zi L2 (l−1) (l) (l) L1 hi yi + L 2 yi
AC
where sl is the input vector size of the l-th layer.
(b) The Derivation of the Equation (15)
Rewrite Eq. (11) : Sw (h) =
m1 X i=1
(hi − c1 )(hi − c1 )T +
m X
(hi − c2 )(hi − c2 )T
i=m1 +1
ACCEPTED MANUSCRIPT
5 CONCLUSIONS
27 T
Sb (h) = (c1 − c2 ) (c1 − c2 )
(n)
∂zi
T r (Sw (h) − Sb (h)) = + −
∂ (n)
∂zi
i=1
∂ (n)
∂zi
Tr
m1 X
Tr
(f (zi ) − c1 )((f (zi ) − c1 )
X
T
(hi − c2 )(hi − c2 )
i=m1 +1
T T r (c − c ) (c − c ) , 1 2 1 2 (n)
∂ ∂zi
m1 X (f (zi ) − c1 )((f (zi ) − c1 )T T r (n)
∂ ∂zi
T
!
!
!
AN US
∂
CR IP T
The partial derivative of the trace of difference Fisher is
i=1
1 0 = 2 (f (zi ) − c1 ) f (zi ) − f (zi ) , m1 ! m X ∂ T Tr (hi − c2 )(hi − c2 ) (n) ∂zi i=m1 +1 1 = 2 (f (zi ) − c2 ) f (zi )0 − f (zi )0 , m2
M
0
∂
CE
PT
ED
T r (Sw (h) − Sb (h)) (n) ∂zi h i 2 (hi − c1 ) 1 − m11 − m11 (c1 − c2 ) f (zi )0 if, i ∈ {1, ..., m1 } = i h ; 2 (hi − c2 ) 1 − m12 − m12 (c1 − c2 ) f (zi )0 if, i ∈ {m1 + 1, ..., m}.
AC
Then, for the node i in the hidden layer l, the residual error is (l) δi
∂
"
L1 L2
h(l−1) y(l)
2
!
h(l−1) y(l)
γ log + 1 − log (l) ∂zi
2
+λ2 tr(Sw (h(l) ) − Sb (h(l) )) + η h(l) 2 T 0 (l) (l) (l) (l) f zi . = wi δ (l+1) + λ2 q(hi ) + 2ηhi
=
ACCEPTED MANUSCRIPT
REFERENCES
28
Acknowledgment This work was supported in part by the National Key Research and Development Program of China under Grant 2018YFB1107400, in part by the National Natural Science
CR IP T
Foundation of China under Grant 61871470 and Grant U1604153, in part by the Key Specialized Research and Development Breakthrough of Henan Province under Grant 192102210121, and in part by the Program for Science and Technology Innovation Talents in Universities of Henan Province under Grant 19HASTIT026. References
AN US
[1] J.-S. Lee, Digital image enhancement and noise filtering by use of local statistics, IEEE Trans. Pattern Anal. Mach. Intell. (2) (1980) 165–168.
[2] C.-A. Deledalle, L. Denis, F. Tupin, Iterative weighted maximum likelihood denoising with probabilistic patch-based weights, IEEE Trans. Image Process. 18 (12) (2009) 2661–2672.
M
[3] J. Chen, Y. Chen, W. An, Y. Cui, J. Yang, Nonlocal filtering for polarimetric SAR data: A pretest approach, IEEE Trans. Geosci. Remote Sens. 49 (5) (2011) 1744–
ED
1754.
[4] F. Bovolo, L. Bruzzone, A detail-preserving scale-driven approach to change detection in multitemporal SAR images, IEEE Trans. Geosci. Remote Sens. 43 (12) (2005)
PT
2963–2972.
[5] F. Bovolo, C. Marin, L. Bruzzone, A hierarchical approach to change detection in
CE
very high resolution SAR images for surveillance applications, IEEE Trans. Geosci. Remote Sens. 51 (4) (2013) 2042–2054.
AC
[6] C. Marin, F. Bovolo, L. Bruzzone, Building change detection in multitemporal very high resolution SAR images, IEEE Trans. Geosci. Remote Sens. 53 (5) (2015) 2664– 2682.
[7] Y. Ban, O. Yousif, et al., Multitemporal spaceborne SAR data for urban change detection in china, IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 5 (4) (2012) 1087–1094.
ACCEPTED MANUSCRIPT
REFERENCES
29
[8] D. C. Zanotta, V. Haertel, Gradual land cover change detection based on multitemporal fraction images, Pattern Recognition 45 (8) (2012) 2927–2937. [9] F. Chatelain, J.-Y. Tourneret, J. Inglada, A. Ferrari, Bivariate gamma distributions (2007) 1796–1806.
CR IP T
for image registration and change detection, IEEE Trans. Image Process. 16 (7)
[10] J. Inglada, G. Mercier, A new statistical similarity measure for change detection in multitemporal SAR images and its extension to multiscale change analysis, IEEE Trans. Geosci. Remote Sens. 45 (5) (2007) 1432–1445.
AN US
[11] K. Conradsen, A. A. Nielsen, J. Schou, H. Skriver, A test statistic in the complex
wishart distribution and its application to change detection in polarimetric SAR data, IEEE Trans. Geosci. Remote Sens. 41 (1) (2003) 4–19.
[12] A. Marino, S. R. Cloude, J. M. Lopez-Sanchez, A new polarimetric change detector in radar imagery, IEEE Trans. Geosci. Remote Sens. 51 (5) (2013) 2986–3000.
M
[13] D. Ciuonzo, C. Vincenzo, D. M. Antonio, On multiple covariance equality testing with application to SAR change detection, IEEE Trans. Signal Process. 65 (19) (2017)
ED
5078–5091.
[14] V. Akbari, S. N. Anfinsen, A. P. Doulgeris, T. Eltoft, G. Moser, S. B. Serpico, Polarimetric SAR change detection with the complex hotelling-lawley trace statistic,
PT
IEEE Trans. Geosci. Remote Sens. 54 (7) (2016) 3953–3966. [15] V. T. Vu, N. R. Gomes, M. Pettersson, P. Dammert, H. Hellsten, Bivariate gamma
CE
distribution for wavelength-resolution SAR change detection, IEEE Trans. Geosci. Remote Sens. 57 (1) (2019) 473–481.
AC
[16] W. Yang, X. Yang, T. Yan, H. Song, G.-S. Xia, Region-based change detection for polarimetric SAR images using wishart mixture models, IEEE Trans. Geosci. Remote Sens. 54 (11) (2016) 6746–6756.
[17] T. Celik, Unsupervised change detection in satellite images using principal component analysis and k-means clustering, IEEE Geosci. Remote Sens. Lett. 6 (4) (2009) 772–776.
ACCEPTED MANUSCRIPT
REFERENCES
30
[18] Y. Zheng, L. Jiao, H. Liu, X. Zhang, B. Hou, S. Wang, Unsupervised saliency-guided SAR image change detection, Pattern Recognition 61 (2017) 309–326. [19] M. Gong, L. Su, M. Jia, W. Chen, Fuzzy clustering with a modified MRF energy
CR IP T
function for change detection in synthetic aperture radar images, IEEE Trans. Fuzzy Syst. 22 (1) (2014) 98–109.
[20] E. M. Domínguez, E. Meier, D. Small, M. E. Schaepman, L. Bruzzone, D. Henke, A multisquint framework for change detection in high-resolution multitemporal SAR images, IEEE Trans. Geosci. Remote Sens. 56 (6) (2018) 3611–3623.
[21] X. Zhang, L. Jiao, F. Liu, L. Bo, M. Gong, Spectral clustering ensemble applied to
AN US
SAR image segmentation, IEEE Trans. Geosci. Remote Sens. 46 (7) (2008) 2126– 2136.
[22] Y. Bazi, L. Bruzzone, F. Melgani, An unsupervised approach based on the generalized gaussian model to automatic change detection in multitemporal SAR images, IEEE Trans. Geosci. Remote Sens. 43 (4) (2005) 874–887.
M
[23] G. Liu, L. Jiao, F. Liu, H. Zhong, S. Wang, A new patch based change detector for
ED
polarimetric SAR data, Pattern Recognit. 48 (3) (2015) 685–695. [24] T. T. Lê, A. M. Atto, E. Trouvé, A. Solikhin, V. Pinel, Change detection matrix for multitemporal filtering and change analysis of SAR and PolSAR image time series,
PT
ISPRS J. Photogramm. Remote Sens. 107 (2015) 64–76. [25] A. Marino, I. Hajnsek, A change detector based on an optimization with polarimetric
CE
SAR imagery, IEEE Trans. Geosci. Remote Sens. 52 (8) (2014) 4781–4798. [26] G. E. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets,
AC
Neural Computation 18 (7) (2006) 1527–1554.
[27] R. Salakhutdinov, A. Mnih, G. Hinton, Restricted boltzmann machines for collaborative filtering, in: Proc. of the 24th Int. Conf. on Machine Learning, ACM, 2007, pp. 791–798.
[28] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324.
ACCEPTED MANUSCRIPT
REFERENCES
31
[29] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion, The Journal of Machine Learning Research 11 (2010) 3371–3408.
CR IP T
[30] A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, J. Schmidhuber, A novel connectionist system for unconstrained handwriting recognition, IEEE Trans. Pattern Anal. Mach. Intell. 31 (5) (2009) 855–868.
[31] D. Ciresan, U. Meier, J. Schmidhuber, Multi-column deep neural networks for image
classification, in: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit.
AN US
(CVPR), IEEE, 2012, pp. 3642–3649.
[32] J. Xie, L. Xu, E. Chen, Image denoising and inpainting with deep neural networks, in: Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2012, pp. 341–349. [33] M. Gong, X. Niu, P. Zhang, Z. Li, Generative adversarial networks for change detection in multispectral imagery, IEEE Geosci. Remote Sens. Lett. 14 (2) (2017)
M
2310–2134.
[34] L. Mou, L. Bruzzone, X. Zhu, Learning spectral-spatial-temporal features via a recurrent convolutional neural network for change detection in multispectral imagery,
ED
IEEE Trans. Geosci. Remote Sens. 57 (2) (2019) 924–935. [35] M. Gong, J. Zhao, J. Liu, Q. Miao, L. Jiao, Change detection in synthetic aperture
PT
radar images based on deep neural networks, IEEE Trans. Neural Netw. Learn. Syst. 27 (1) (2016) 125–1382.
CE
[36] F. Liu, L. Jiao, X. Tang, S. Yang, W. Ma, B. Hou, Local restricted convolutional neural network for change detection in polarimetric SAR images, IEEE Trans. Neural
AC
Netw. Learn. Syst. 30 (3) (2019) 818–833.
[37] L. Su, M. Gong, P. Zhang, M. Zhang, J. Liu, H. Yang, Deep learning and mapping based ternary change detection for information unbalanced images, Pattern Recognition 66 (C) (2017) 213–228.
ACCEPTED MANUSCRIPT
REFERENCES
32
[38] C. P. Marc’Aurelio Ranzato, S. Chopra, Y. LeCun, Efficient learning of sparse representations with an energy-based model, in: Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2007.
CR IP T
[39] R. Socher, J. Pennington, E. H. Huang, A. Y. Ng, C. D. Manning, Semi-supervised recursive autoencoders for predicting sentiment distributions, in: Proc. of the Conf. on Empirical Methods in Natural Language Proc., Association for Computational Linguistics, 2011, pp. 151–161.
[40] J.-S. Lee, M. R. Grunes, G. De Grandi, Polarimetric SAR speckle filtering and its
implication for classification, IEEE Trans. Geosci. Remote Sens. 37 (5) (1999) 2363–
AN US
2373.
[41] J.-S. Lee, E. Pottier, Polarimetric radar imaging: from basics to applications, CRC press, 2009.
[42] A. Moreira, P. Prats-Iraola, M. Younis, G. Krieger, I. Hajnsek, K. P. Papathanassiou, 1 (1) (2013) 6–43.
M
A tutorial on synthetic aperture radar, IEEE Geosci. and Remote Sens. Magazine
[43] W. Dierking, H. Skriver, Change detection for thematic mapping by means of air-
ED
borne multitemporal polarimetric SAR imagery, IEEE Trans. Geosci. Remote Sens. 40 (3) (2002) 618–636.
PT
[44] B. Hou, Q. Wei, Y. Zheng, S. Wang, Unsupervised change detection in SAR image based on gauss-log ratio image fusion and compressed projection, IEEE J. Sel. Topics
CE
Appl. Earth Observ. Remote Sens. 7 (8) (2014) 3297–3317. [45] R. G. White, Change detection in SAR imagery, Int. J. Remote Sens. 12 (2) (1991)
AC
339–360.
[46] S. Hachicha, F. Chaabane, On the SAR change detection review and optimal decision, Int. J. Remote Sens. 35 (5) (2014) 1693–1714.
[47] Y. Li, M. Gong, L. Jiao, L. Li, R. Stolkin, Change-detection map learning using matching pursuit, IEEE Trans. Geosci. Remote Sens. 53 (8) (2015) 4712–4723.
ACCEPTED MANUSCRIPT
REFERENCES
33
[48] P. Lv, Y. Zhong, J. Zhao, L. Zhang, Unsupervised change detection based on hybrid conditional random field model for high spatial resolution remote sensing imagery, IEEE Trans. Geosci. Remote Sens. 56 (7) (2018) 4002–4015.
CR IP T
[49] D. Tao, X. Li, X. Wu, S. J. Maybank, General tensor discriminant analysis and gabor features for gait recognition, IEEE Trans. Pattern Anal. Mach. Intell. 29 (10) (2007) 1700–1715.
[50] Q. Du, Modified fisher’s linear discriminant analysis for hyperspectral imagery, IEEE Geosci. Remote Sens. Lett. 4 (4) (2007) 503.
AN US
[51] A. Ng, Sparse autoencoder, Vol. 72, CS294A Lecture notes, 2011.
[52] R. Touzi, A. Lopes, P. Bousquet, A statistical and geometrical edge detector for sar images, IEEE Trans. Geosci. Remote Sens. 26 (6) (1988) 764–773. [53] R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification, John Wiley & Sons, 2012.
M
[54] H. Wang, S. Yan, D. Xu, X. Tang, T. Huang, Trace ratio vs. ratio trace for dimensionality reduction, in: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern
ED
Recognit. (CVPR), IEEE, 2007, pp. 1–8. [55] M. Yang, L. Zhang, X. Feng, D. Zhang, Sparse representation based fisher discrimination dictionary learning for image classification, Int. J. Comput. Vis. 109 (3) (2014)
PT
209–232.
CE
Author Biography
Ganchao Liu received the Ph.D. degree from Xidian University, Xi’an, China, in
AC
2016. He is currently a postdoctoral research fellow with the Center for OPTical IMagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University, Xi’an, China. His current interests include image processing and pattern recognition. Lingling Li received the B.S. and Ph.D. degrees from Xidian University, Xi’an, China,
in 2011 and 2017 respectively. Between 2013 - 2014, she was an exchange Ph.D. student
ACCEPTED MANUSCRIPT
REFERENCES
34
with the Intelligent Systems Group, Department of Computer Science and Artificial Intelligence, University of the Basque Country UPV/EHU, Spain. She is currently a postdoctoral researcher in the School of Artificial Intelligence at Xidian University. Her current research interests include quantum evolutionary optimization, machine learning and deep
CR IP T
learning.
Licheng Jiao received the Ph.D. degrees from Xi’an Jiaotong University, Xi’an,
China, in 1990, respectively. He is a Professor in the School of Artificial Intelligence
at Xidian University. He is a Fellow of IEEE. His research interests include intelligent information processing, image processing, natural computation, machine learning and
AN US
pattern recognition.
Yongsheng Dong received his Ph. D. degree in applied mathematics from Peking University in 2012. He was a postdoctoral research fellow with the Center for Optical Imagery Analysis and Learning, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an, China from 2013 to 2016. From 2016 to 2017, he was a visiting research fellow at the School of Computer Science and Engineering, Nanyang
M
Technological University, Singapore. He is currently an associate professor with the School of Information Engineering, Henan University of Science and Technology, China.
ED
His current research interests include pattern recognition, machine learning, and computer vision. He has authored and co-authored over 30 papers at famous journals and conferences, including IEEE TIP, IEEE TNNLS, IEEE TCYB, IEEE TIE, IEEE TCSVT,
PT
and ACM TIST. He is an associate editor of Neurocomputing. Xuelong Li is a full professor with the School of Computer Science and the Center
CE
for OPTical IMagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical
AC
University, Xi’an 710072, P.R. China. He is a Fellow of IEEE.
Accepted Manuscript
Stacked Fisher Autoencoder for SAR Change Detection Ganchao Liu, Lingling Li, Licheng Jiao, Yongsheng Dong, Xuelong Li PII: DOI: Article Number: Reference:
S0031-3203(19)30274-2 https://doi.org/10.1016/j.patcog.2019.106971 106971 PR 106971
To appear in:
Pattern Recognition
Received date: Revised date: Accepted date:
11 January 2019 17 May 2019 15 July 2019
Please cite this article as: Ganchao Liu, Lingling Li, Licheng Jiao, Yongsheng Dong, Xuelong Li, Stacked Fisher Autoencoder for SAR Change Detection, Pattern Recognition (2019), doi: https://doi.org/10.1016/j.patcog.2019.106971
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
1 Highlights • The original SAE is expanded to suit with the multiplicative noise in SAR change detection.
CR IP T
• The features extracted by SFAE are more discriminative than the original stacked autoencoder due to that Fisher discriminant criterion is incorporated into SFAE.
• Experiments on the simulated and real SAR datasets reveal that the proposed SFAE algorithm is effective on multitemporal single/multi-polarization SAR change detec-
tion. Specifically, the proposed SFAE method is obviously superior to the real-time
AN US
methods on detection accuracy and the non-realtime methods on computational
AC
CE
PT
ED
M
complexity.
ACCEPTED MANUSCRIPT
Stacked Fisher Autoencoder for SAR Change Detection Ganchao Liua,b , Lingling Lib , Licheng Jiaob , Yongsheng Dongc,∗, Xuelong Lia a Center
CR IP T
for OPTical IMagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University, Xi’an, China b School of Artificial Intelligence, Xidian University, Xi’an, China c School of Information Engineering, Henan University of Science and Technology, Luoyang, China
Abstract
Stacked autoencoder is effective in image denoising and classification when it is used for
AN US
synthetic aperture radar (SAR) change detection. However, the resulting features may
not be discriminative enough for in some sense. To alleviate this problem, in this paper we propose a stacked Fisher autoencoder (SFAE) for SAR change detection. Specifically, in the framework of SFAE, unsupervised layer-wise feature learning and supervised finetuning are jointly performed when training the network. The trained network can be used to detect the changes in both of the single and multi-polarization SAR datasets
M
in real-time. The proposed SFAE has two advantages. The first one is to expand the stacked autoencoder to suit the environment with the multiplicative noise in SAR change
ED
detection. The second is that the features extracted by SFAE are more discriminative than the original stacked autoencoder due to that Fisher discriminant criterion is incorporated into SFAE. The results on the simulated and real SAR datasets indicate that the proposed
PT
SFAE algorithm has a significant advantage on multitemporal single/multi-polarization SAR (SAR/PolSAR) change detection.
CE
Keywords: Stacked Fisher autoencoder (SFAE), synthetic aperture radar (SAR),
AC
change detection, stacked autoencoder (SAE), Fisher criterion.
∗ Corresponding
author Email addresses: [email protected] (Ganchao Liu ), [email protected] (Lingling Li ), [email protected] (Licheng Jiao ), [email protected] (Yongsheng Dong), [email protected] (Xuelong Li )
Preprint submitted to Pattern Recognition
July 19, 2019
ACCEPTED MANUSCRIPT
1 INTRODUCTION
3
1. Introduction Since the number of satellites for earth observation is on the increase, there is a growing interest in the analysis of images acquired on the same geographical area at different
CR IP T
times. As a high resolution earth observation system with the capacity of all-weather and all-time, multitemporal SAR images change detection has been used in a wide variety of applications, such as agricultural surveys, urban planning and disaster management.
However, because of the coherent nature of the scattering phenomenon, SAR images are inherently affected by speckle noise, which makes the automatic change analysis difficult. Therefore, the reduction of the speckle interference becomes an important issue in SAR
AN US
change detection.
Generally, SAR change detection tends to be carried out through the following three steps: despeckling, generating difference images, and classification. In the past few decades, many creative works have been done in these respects. In the first step, the classical despeckling methods, such as Lee filter[1], the probabilistic patch-based (PPB) filter [2] and the Pretest filter[3], are widely used. In the second step, to lower the impact
M
of speckle noise, the log-ratio operator is widely applied in difference images generation [4–7]. By measuring the similarities of the multitemporal SAR images, many methods
ED
[8–16] are proposed to generate difference images based on statistical distributions. In [10] and [9], the similarity is measured by the cumulant-based Kullback-Leibler divergence (CKLD) and bivariate Gamma distributions respectively. In [11], a test statistic based on
PT
the complex Wishart distribution is used for the change detection in Polarimetric SAR (PolSAR) data. In the third step, a threshold is usually set for the previous algorithms based on statistical distributions. On the other hand, as a binary classification prob-
CE
lem, change detection can be performed by using the common clustering or classification methods, such as k-means [17, 18], fuzzy c-means (FCM) [19, 20], spectral clustering [21]
AC
and so on. It can be easily found that the detection methods with speckle reduction algorithms [22–24] are obviously more effective than those without the pre-filter procedure [10, 11, 25]. However, those despeckling-based detection methods are time-consuming, and are not suitable in the age of big data any more. Inspired by the computational models of the biological brain, deep learning [26, 27] has become a powerful method in analyzing big data. In deep learning, the output of
ACCEPTED MANUSCRIPT
1 INTRODUCTION
4
former layer is the input of the latter layer. The most commonly used deep neural network models mainly include deep belief networks (DBNs) [26], convolutional neural networks (CNNs) [28], and stacked autoencoders (SAE) [29]. Deep learning has been applied in handwriting recognition [30], image classification [31], denoising [32], etc. More recently,
CR IP T
the deep neural network has even been used in multispectral and SAR image change detection[33–37]. In [33], the supervised labels are replaced by a novel unsupervised
discriminator network. In [34], the spectral-spatial-temporal features of the multispectral
imagery are extracted by a new recurrent convolution neural network. In [35], a novel network with restricted Boltzmann machine (RBM) has been introduced to solve the
change detection problem. However, this detection method is not robust enough for
AN US
different SAR datasets, since the performance of the networks are heavily relying on the selecting samples.
As an unsupervised neural network, stacked autoencoder is widely used in image denoising [29], classification [38] and natural language recognition [39]. Inspired by the good performance on image denoising and classification, the use of SAE in SAR change
M
detection has a promising future since the difficulty of SAR change detection is to reduce the influence of the inherent speckle interference. In this paper, we propose a stacked Fisher autoencoder by incorporating Fisher discriminant criterion for SAR change detec-
ED
tion. In the framework of SFAE, unsupervised layer-wise feature learning and supervised fine-tuning are jointly performed when training the network. The trained auto-encoder is suitable for different datasets with the different equivalent number of looks. In practice,
PT
to save the time spent in computation, the network can be trained in advance. The contributions of this paper are as follows. First, the SAE network is expanded
CE
to suit the environment with the serious multiplicative noise in SAR change detection. Second, the features extracted by SFAE are more discriminative than the original stacked autoencoder due to that Fisher discriminant criterion is incorporated into SFAE. Third,
AC
experiments on the simulated and real SAR datasets reveal that the proposed SFAE algorithm is effective on multitemporal single/multi-polarization SAR change detection. Specifically, the proposed SFAE method is obviously superior to the real-time methods on detection accuracy and the non-realtime methods on computational complexity. This paper is organized in five sections. In section II, the Polarimetric SAR speckle
ACCEPTED MANUSCRIPT
2 SAR SPECKLE STATISTICS AND MOTIVATIONS
5
statistics and the motivations of our method are briefly introduced. The methodology is discussed in Section III. In Section IV, the results and discussions of the experiment on simulated images and real SAR dataset are given. Finally, the conclusions are given in
CR IP T
the last section. 2. SAR Speckle Statistics and Motivations 2.1. SAR System Introduction
For the full polarimetric SAR system, the target is measured in two polarization modes: horizontal and vertical. Combining the linear receive and transmit polarizations, the target can be characterized using the scattering matrix[40]:
AN US
S=
Shh
Shv
Svh
Svv
,
(1)
where Shv is the scattering element of horizontal transmitting and vertical receiving polarization, and the other three elements are defined similiarly. In the case that the transmitter and receiver antennas are the same, the non-diagonal elements are reciprocal, i.e.,
M
Shv = Svh . Then, the full polarimetric SAR data can be characterized with the following covariance matrix:
D E 2 |Shh | √ ∗ C= 2Shv Shh ∗ hSvv Shh i
PT
ED
√
∗ 2Shh Shv D E 2 2 |Shv |
√ ∗ 2Svv Shv
∗ i hShh Svv
√ ∗ 2Shv Svv D E 2 |Svv |
,
(2)
where the superscript “*” denotes complex conjugation, h·i stands for the average value.
CE
The measured value of the scattering is presented in the plural form, which contains
both the amplitude and phase information. The visual single polarization SAR images is shown as the amplitude A or intensity information I. Among them, I = A2 . Due
AC
to the coherent interference of waves reflected from many elementary scatters, speckle is inevitablely appearing in SAR images. The diagonal terms of C are the intensities of linear polarizations and can be representeds by the following multiplicative noise model[41].
I = RnI ,
(3)
ACCEPTED MANUSCRIPT
2 SAR SPECKLE STATISTICS AND MOTIVATIONS
Train Part
Test Part
Labeled image2
Input image1
Difference image
Input image2
Difference image
Training the labeled dataset
CR IP T
Labeled image1
6
Trained network
Trained network
Change map
AN US
Figure 1: The change detection framework with neural network.
where I represents a measured intensity value, R is the corresponding actual intensity value, nI is the speckle noise in intensity format and independent of R. In this paper, the full-polarization SAR data is seen as three independent intensity SAR images in change detection.
M
Under the assumption of fully developed speckle, an L-look intensity and amplitude
ED
SAR images obey the Gamma and Nakagami distribution, respectively [42]. 1 p(I|R) = Γ(L)
PT
2 p(A|R) = Γ(L)
L R
L R
L
L
LI exp − R
LA2 exp − R
I L−1 ,
(4)
A2L−1 ,
(5)
where Γ(·) is the gamma function. When L=1, Eqs. (4) and (5) denote the distributions
CE
of single-look intensity and amplitude SAR data, which corresponding to the exponential and Rayleigh distributions respectively. In this paper, SAR images with amplitude format
AC
will be discussed as an example. 2.2. Problems and Motivations Since the SAR/PolSAR images suffer from the inherent speckle noise, the speckle re-
duction was pointed out as a major topic for the development of a successful detection scheme [43–45]. Therefore, it is necessary to develop a robust SAR image change detection technique against the speckle noise [46]. To reduce the influence of speckle noise
ACCEPTED MANUSCRIPT
2 SAR SPECKLE STATISTICS AND MOTIVATIONS
w(d2)
...
(2)
...
h
w(n-1) ( n-1)
w(dn-1)
...
h
J
(n)
(n)
Stacked Auto-encoder
θ ~ C
Logistic Classifier
M
C
y
( n-1)
) w (n d
...
h
y
(2)
AN US
w(n)
y
CR IP T
J
w ( 2)
(1)
...
...
(1)
y
...
w(d1)
...
h
...
w
J
(1)
...
x
7
Figure 2: The flowchart of stacked Fisher autoencoder for SAR change detection.
ED
theoretically, the techniques with log-ratio operator [4, 6], speckle reduction [23, 47] and Markov random fields [22, 48] are often used to improve the efficiency of change detection.
PT
In this paper, the "seriousness" of the speckle is associated with the equivalent number of looks (ENL). The smaller the ENL is, the more serious the speckle is. The techniques without pre-processing, such as [11, 12, 25], which are not suitable for the cases when the
CE
speckle is serious. For example, the ENLs of the datasets tested in [11, 12, 25] are 13, 59 and 109, respectively.
AC
More recently, the neural network based methods are proposed to obtain the change
maps of multitemporal remote-sensing images [35–37]. As a well-known deep architecture, stacked autoencoder has been proved to be effective in denoising [32] and feature extraction [38]. Though the white Gaussian noise is very different from the SAR speckle pattern, many SAR despeckling algorithms are expanded from the filters based on white Gaussian noise models. Thus, the ideal of the stacked autoencoder is worth exploring.
ACCEPTED MANUSCRIPT
3 METHODOLOGY
8
Inspired by these characters, a novel change detection framework with autoencoder network can be designed to deal with the massive amounts of multitemporal SAR/PolSAR images. In the age of big data, SAR image change detection faces new opportunities and
CR IP T
challenges. On the one hand, the processing of the massive volumes of data is very time-consuming. On the other hand, the big data makes the features learning in deep architecture feasible. To achieve this goal, we should tackle two important issues at least.
Firstly, the autoencoder network should be expanded to suit the environment with multiplicative speckle noise in SAR system. Secondly, the neural network should be modified
to meet with the task of change detection. In other words, the features extracted from
AN US
the SAE network should be discriminative to classify the changed and unchanged pixels in SAR images. In order to make the features extracted by the network discriminative, the concept of linear discriminant analysis has been widely used in recognition [49] and
change detection [50], which give us a great inspiration. Inspired by these works, we intend to make the features more discriminative by introducing a discriminant criterion.
M
In the next section, these issues will be discussed in details. 3. Methodology
ED
In this section, we first introduce the stacked autoencoder briefly, and then propose a stacked Fisher autoencoder (SFAE) for SAR change detection. Our proposed SFAE is learnt from the simulated or labeled dataset using the backpropagation algorithm. The
PT
change detection framework with neural network is shown in Fig.1.
CE
3.1. A brief description of Stacked Autoencoder In this subsection, a brief description of the standard stacked autoencoder framework
is illustrated. As shown in Fig.2, the input vector x of the autoencoder is encoded to a
AC
hidden vector h in the first: h(1) = f (z(1) ) = f (w(1) x + b(1) ).
(6) (1)
(1)
Then, remapping the hidden vector h to a decoded vector y(1) = f (wd h(1) + bd ). Here, the superscript
(1)
represents the first layer. w(1) and b(1) are denote the encoder (1)
weight matrix and the bias vector of the first layer. wd
is the corresponding decoder
ACCEPTED MANUSCRIPT
3 METHODOLOGY
9 (1)
weight matrix of the first layer. Here, the decoder weight matrix wd
is set as the
transposition of the encoder weight matrix w(1) . In this paper, f is the sigmoid function f (t) = 1/(1 + exp(−t)) [51]. The parameters {w, b} are optimized to minimize the
CR IP T
reconstruction error [29]:
(7)
JAE (w, b) = − log p(x|y) + λ kwk ,
where λ is the weight decay parameter. As shown in eq.(7), the definition of JAE (w, b) includes two terms. The former one denotes the reconstruction error. The latter one is
a regularization term used to prevent overfitting. A stacked autoencoder network can be
AN US
seen as a multi-layer autoencoder. In this network, the output of the former layer is the input of the latter layer.
3.2. Stacked Fisher Autoencoder for SAR Change Detection
For the co-registered images with multiplicative speckle noise, the ratio difference image patch is used as the input layer, i.e., x =
A1 A2 .
A1 and A2 are the observed amplitude
SAR images patch obtained at different times. Under the unchanged hypothesis, the ratio
M
distribution with different number of looks is given in [23, 52]
ED
p(xk |R1 = R2 ) ∝
xk2L1 −1 , 1 2 L1 +L2 (L L2 xk + 1)
(8)
where xk is the k-th pixel of the ratio image patch. R1 and R2 are the corresponding
PT
reflectivity patches without noise. L1 and L2 are the looks of the corresponding observed SAR image patches.
Stacked autoencoder for SAR:
CE
Substituting equation (8) into (7), a new cost function of autoencoder suit for SAR
AC
images can be obtained as follows:
2
JAE−SAR (w, b) = − log p(x|y) + λ kwk " ! 2 # m s (9) 1 XX L1 xk,j xk,j 2 ∝ γ log + 1 − log + λ kwk , m ˜ j=1 L2 yk,j yk,j k=1
where m ˜ = m/(2L − 1), m is the number of the train samples, s is the size of each sample
patches, xk,j denotes the k-th pixel of the j-th sample patch. γ =
L1 +L2 2L1 −1
is a constant, λ
ACCEPTED MANUSCRIPT
3 METHODOLOGY
10
is the decay parameter. For the stacked autoencoder, the features of the l -th layer (l =1...,n), i.e., the hidden
JSAE−SAR (w(l) , b(l) ) = − log p(x(l) |y(l) ) + λ ∝
m X s X
j=1 k=1
γ log L1 L2
(l−1) hj,k (l) yj,k
!2
l+1
X
(i) 2
w i=l
+ 1 − log
Fisher discrimination criterion:
CR IP T
layer h(l) , is the input of the next layer x(l+1) .
(l−1)
hj,k
(l)
yj,k
!
+λ
l+1
X
(i) 2
w . i=l
(10)
AN US
In this paper, the modified SAE network is trained to detect the changes in SAR images. In [49] and [50], the applications of linear discriminant analysis have been investigated for recognition and change detection. In order to make the features extracted by SAE network more discriminative, the concept of Fisher discriminant criterion [53] is
introduced. Based on the Fisher discrimination criterion, the discrimination capability of
M
the SAE network can be improved by minimizing the within-class scatter of h, denoted by Sw (h), and maximizing the between-class scatter of h, denoted by Sb (h). X
(hi − c1 )(hi − c1 )T +
ED
Sw (h) =
i∈N1
X
i∈N2
(hi − c2 )(hi − c2 )T
,
(11)
T
Sb (h) = (c1 − c2 ) (c1 − c2 )
PT
where c1 and c2 are the mean vector of the changed and unchanged features, respectively, P P i.e., c1 = m11 i∈N1 hi , c2 = m12 i∈N2 hi . N1 and N2 are the set of changed and
unchanged sites respectively. m1 and m2 are the number of samples in their corresponding
CE
classes. In practice, the Fisher criterion is usually defined as the minimizing of the ratio trace T r (Sw (h)) /T r (Sb (h)) [54] or the minimizing of the difference trace T r (Sw (h)) −
AC
T r (Sb (h))[55], where T r(·) is the trace of matrix. The relationship between the two
types of Fisher criterions have been discussed in [55]. In this paper, the difference trace version of the Fisher criterion will be added in the objective function, while the ratio trace version of the Fisher criterion will be used as a criterion to evaluate the network in the experiments. The experiments will verify the relationship between the two types of Fisher criterions.
ACCEPTED MANUSCRIPT
3 METHODOLOGY
11
When considering the difference trace T r (Sw (h)) − T r (Sb (h)), the term −T r (Sb (h))
will make the objective function non-convex. According to the suggestions of [55], an 2
elastic term khk2 is added in our Fisher function ϕ(h): 2
(12)
where η is a weight decay parameter.
CR IP T
ϕ(h) = tr(Sw (h) − Sb (h)) + η khk2 ,
Combining the Eqs. (10) and (12), we have the following objective function of the Stacked Fisher Autoencoder model (SFAE). JSF AE (w(l) , b(l) )
1 = m ˜
j=1 k=1
γ log L1 L2
i=l
(l−1)
hj,k
(l) yj,k
l+1
X
(i) 2
w + λ2 ϕ(h(l) ), i=l
!2
+ 1 − log
(l−1)
hj,k
(l) yj,k
!
(13)
M
+ λ1
s m X X
l+1
X
(i) 2
w + λ2 ϕ(h(l) )
AN US
= − log p(x(l) |y(l) ) + λ1
where λ1 and λ2 are the weight decay parameters. In the following description, the SFAE network will be optimized by the back-propagation
ED
algorithm.
Firstly, the activations throughout the network h(l) = f (z(l) ), l = 2, 3, ..., n (the (l)
δi
PT
output layer), are computed by the feedforward algorithm. Then, the "residual error" is computed to measure the difference between the network activations [51]. (l)
CE
According to the Appendix, the "residual error" δi
AC
(l+1) δi
in the decode layer l is
"
! # 2 (l−1) L1 h(l−1) h (l) = (l+1) γ log + 1 − log + λ2 ϕ(h ) L2 y(l) y(l) ∂zi 2 (l) (l−1) (L + L ) h + L2 yi 1 2 i −2γL1 (l+1) 0 . = 2 2 f zi L2 (l−1) (l) (l) L1 hi yi + L2 yi ∂
(14)
ACCEPTED MANUSCRIPT
3 METHODOLOGY
12
For all of the nodes in the layer l, the "residual error" vector can be rewritten as:
δ
(l)
where,
=
w
(l)
T
δ
(l+1)
(l)
(l)
+ λ2 q(h ) + 2ηh
•f
0
z(l) ,
(15)
CR IP T
h i (l) (c1 −c2 ) 1 − 2 h − c 1 − 1 i m1 m1 if, i ∈ {1, ..., m1 }; (l) q(hi ) = h i (l) 2) 2 hi − c2 1 − m12 − (c1m−c 2 if, i ∈ {m1 + 1, ..., m}.
The operator "•" denotes the element-wise product, also called the Hadamard product.
AN US
Furthermore,
m ∂ 1 X (l+1) (l) T (l) (l) + λ1 w(l) , δ · hk J(w , b ) = m ∂w(l) k=1
∂ 1 J(w, b) = δ (l+1) . m ∂b(l)
(16)
(17)
M
Then, the parameters are updated to minimize the objective function.
ED
Pm (l+1) (l) T 1 (l) (l) (l) w = w − α m k=1 δ · hk + λ1 w , b(l) = b(l) − α 1 δ (l+1) . m
(18)
The main procedure of the stacked Fisher autoencoder algorithm is presented in Al-
PT
gorithm 1.
Fine tune:
CE
For the stacked Fisher autoencoder, supervised fine-tuning is commonly used to im-
prove the performance of classification. In this paper, the supervised logistic classifier is used to classify the changed and unchanged pixels. The expression of the logistic classifier
AC
is
e Θ (h(n) ) = C
1 , 1 + e−ΘT h(n)
e is the label estimated by the logistic classifier. where Θ is the model parameter, C
(19)
Then, we need to fine-tune the model parameters (w, b, Θ) by minimizing the objec-
ACCEPTED MANUSCRIPT
13
CR IP T
3 METHODOLOGY
AN US
Algorithm 1 Stacked Fisher Auto-Encoder (SFAE) algorithm Input: Train samples x Task: Train the network by minimizing the objective function J(w(l) , b(l) ). Parameters: network parameters (w, b), learning rate α, weight decay parameters λ1 , λ2 and η Initialization: Initialize the parameters w=0, b=0. 1. Compute the activations by the feedforward algorithm: h(l) = f (z(l) ) = f (w(l) h(l−1) + b(l) ),
y(l) = f (z(l+1) ) = f (w(l+1) h(l) + b(l+1) ). 2. For the decode layer l +1, set
2 0 −2γL1 (L1 + L2 ) h(l−1) + L2 y(l) (l+1) . = 2 2 • f z L2 L1 h(l−1) y(l) + L2 y(l)
M
δ
(l+1)
ED
For the hidden layer l, set T 0 δ (l) = w(l) δ (l+1) + λ2 q(h(l) ) + 2ηh(l) • f z(l) . 3. Update the parameters:
1 (l+1) (l) T (l) δ h + λw , =w −α m 1 (l+1) (l) (l) . b =b −α δ m
PT CE
w
(l)
(l)
AC
4. Repeat the above steps until the objective function JSF AE (w(l) , b(l) ) does not change. Output: The parameters w(l) , b(l) ; The activation of the output hidden layer h(l)
ACCEPTED MANUSCRIPT
4 EXPERIMENTS AND RESULTS
14
e tive function J(w, b, Θ).
n+1
T 1 λ1 X
(i) 2 λ1 2 e e J(w, b, Θ) = − C−C h(n) Θ + kΘk ,
w + m 2 i=1 2
(20)
CR IP T
C is the ground truth label.
In conclusion, the training of the SFAE network for change detection includes two parts: 1) training the stacked Fisher autoencoder parameters w and b by minimize the
objective function JSF AE (w, b); 2) Fine-tuning the model parameters w, b and Θ by
e minimize the objective function J(w, b, Θ). The flowchart of the stacked autoencoder for SAR change detection is shown in Fig.2.
AN US
4. Experiments and Results
In this section, the proposed SFAE algorithm is trained on a synthetic dataset with supervised labels. The training dataset contains more than 200,000 image patches with the size of 5×5. In order to simulate the SAR image with speckle noise, the training dataset is corrupted by the multiplicative noise obeys Rayleigh distribution. For the
M
network with 25×36×36×25 architecture, the number of the samples in the training dataset is large enough. In this paper, for the different datasets with the similar ENL,
ED
the results of the proposed method are obtained by the same trained network. In the following subsections, the parameters selection and experiments on datasets are discussed.
PT
4.1. Parameters setting and the experiments on synthetic dataset To quantitative analysis the performance of the compared methods, three indices are introduced in this paper. The first one is the percentage correct classification (PCC)
CE
calculated by the true positives (TP, changed pixels that detected) and the true negatives
AC
(TN, unchanged pixels that detected):
P CC = (T P + T N )/N,
(21)
where N is the sum of the changed pixels (Nc) and the unchanged pixels (Nu). The second one is the Kappa coefficient: Kappa = (P CC − β)/(1 − β),
(22)
ACCEPTED MANUSCRIPT
4 EXPERIMENTS AND RESULTS
15
β = ((T P + F A) · N c + (M A + T N ) · N u) /N 2 . MA and FA are the short of missed alarms and false alarms respectively [19].
The last one is the operation time to evaluate the complexity of the compared algorithms.
CR IP T
In the training stage, the SFAE network is trained by proper parameters selection. Specifically, based on a great number of tests, the decay parameters are set with λ1 =
1 × 10−7 , λ2 = 0.02 and η = 0.1. Besides that, the number of the layers is suggested to set as three in this paper. In order to explain the superiority of the selected parameters,
the parameters λ1 , λ2 , η and the the selection of the number of the hidden layers are detailedly discussed in this subsection.
AN US
Fig. 3 presents the experiment on an L-band synthetic PolSAR data which is corrupted by speckle noise with L1 =L2 =4. Figs. 3(a), (b) and (c) are the Pauli images of the noisy simulated PolSAR dataset with the size of 128×128 and their corresponding standard change image. The hidden layers are visualized in Figs. 3(d)-(f). From the figure, we can see that as the number of layers increases, the feature becomes smoother and easier
M
to be classified. It implies that the SFAE model can effectively overcome the influence of speckle noise with the increase of layers. From Fig. 3(g) to (i) are the change images of the proposed SFAE with single hidden layer, two, and three hidden layers, respectively.
ED
From the Fig. 3(g), we can see that there are many missed alarms (MA) in the upper right changed square. While the missed alarms in the upper right square are decreased gradually in Figs. 3(h) and (i). The results on the synthetic dataset may imply that
PT
the performance of the SFAE network becomes more and more effective along with the
CE
increasing of the number of hidden layers.
In table 1, the ratio version of Fisher criteria (Ratio Fisher), i.e., T r (Sw (h)) /T r (Sb (h)),
AC
is used for assessing the detection performance of the SFAE algorithm with different number of layers. The Ratio Fisher coefficient of the features extracted by the SFAE network in each polarization channels are gradually decreased when the hidden layers increased from 1 to 3. However, the Ratio Fishers become worse when the numbers of hidden layers higher than three, which implies that the network is overfitting. From table 1, we can have the following conclusions. First, by minimizing the difference version of the Fisher
ACCEPTED MANUSCRIPT
16
(b) Noisy image 2
(d) Hidden layer 1
(e) Hidden layer 2
(c) Reference
AN US
(a) Noisy image 1
CR IP T
4 EXPERIMENTS AND RESULTS
ED
M
(f) Hidden layer 3
PT
(g) Single layer
(h) Two layers
(i) Three layers
AC
CE
Figure 3: Results on the simulated PolSAR datasets. (a) and (b) are the noisy Pauli images corrupted with L1 =L2 =4. (c) Reference image. From (d) to (f) are the visualization of the hidden layers. From (g) to (i) are the results of the proposed method with single hidden layer, two hidden layers, and three hidden layers, respectively. Table 1: Ratio Fisher of the features in each hidden layers
Layers 1 2 3 4 5
HH 0.3725 0.3211 0.2117 0.5554 8.7013
Ratio Fisher HV VV 2.5142 123.7999 2.1248 97.2205 1.3999 48.8651 3.7929 187.6981 60.8815 4034.7000
ACCEPTED MANUSCRIPT
4 EXPERIMENTS AND RESULTS
17
Table 2: Quantitative comparisons on the synthetic PolSAR data
MA 860 807 151 254 232 71 8
FA 648 1591 78 11 2 62 7
PCC(%) 90.80 85.36 98.60 98.50 98.57 99.19 99.91
Kappa 0.7496 0.5818 0.9574 0.9559 0.9611 0.9783 0.9976
Time(s) 0.22 0.41 0.69 0.31 0.38 0.49 13.75
CR IP T
Methods Wishart[11] PCD[12] SAE-3 SFAE-1 SFAE-2 SFAE-3 PPCD[23]
criteria in training the network, the features extracted by SFAE network become more
discriminative with the increasing of the hidden layers. Second, with the limited size of layer is suggested to set as three.
AN US
the training dataset used in this paper, to avoid overfitting, the number of the hidden
Table 2 presents a quantitative comparison between the proposed SFAE method and the latest algorithms on PolSAR change detection. The compared algorithms including the Wishart detector [11], the polarimetric change detector (PCD) [12], and the patch based polarimetric change detector (PPCD) [23]. In the experiments, the thresholds in
M
the Wishart detector [11] and PPCD [23] are 0.1 and 0.0002, respectively. The angle difference in the PCD algorithm is set as 40◦ , Besides that, we also made a comparison
ED
between the modified autoencoder and the unmodified one. In the experiments, both of the proposed SFAE network and the ordinary autoencoder are trained with the same parameters and training samples.
PT
From table 2, we can have the following conclusions: 1) Along with the increasing of the number of layers from 1 to 3, the performance of the SFAE network are gradually
CE
better, which has been demonstrated in Fig. 3 and table 1. 2) The proposed SFAE method can detect the changes in real-time. The operation time of the SFAE method is two orders of magnitude less than that in PPCD. 3) The performance of SFAE algorithm
AC
is obviously superior to the other real-time PolSAR change detection algorithms. 4) The results of the proposed SFAE are significantly outperformed than that of the ordinary autoencoder. It further manifests the improvement of the modification done in the SFAE method.
ACCEPTED MANUSCRIPT
4 EXPERIMENTS AND RESULTS
18
1
CR IP T
2
(b) Image obtained in March 2009
AN US
(a) Image obtained in June 2006
Figure 4: The Wuhan dataset.
4.2. Experiments on single-polarization SAR images
In this subsection, a real calibrated single-polarization SAR dataset is tested by the compared methods. The compared algorithms including the principal component analysis and RBM detector [35].
M
(PCA) detector [17], MRFFCM [19], Matching Pursuit (MP) detector [47], CKLD [10]
ED
In the dataset, there are two four look L-band SAR images obtained by ALOS PALSAR with 10-m resolution. The two images are the size of 10000×10000 pixels and sensed over the city of Wuhan, China in June 2006 and March 2009. The SAR image pair of
PT
Wuhan dataset are displayed in Fig. 4. The Wuhan dataset has a rich nature scene, including the urban area, farmland, rivers and lakes. In the experiments, two representative regions have been chosen for detailly descriptions. The selected Region 1 and 2
CE
are located around the Yangtze river and Donghu lake as marked in Fig. 4(a) respectively. The size of the marked regions are 500×500. The detection results of the marked
AC
region 1 are presented in Fig. 5. The main changes in Region 1 are the Yangtze river bridge under construction and the river bank. The enlarged images of the marked Region 1 are displayed in Figs. 5(a) and (b). To compare the performance of the mentioned algorithms, the log-ratio image is used as the reference in this paper. The corresponding log-ratio image of the marked region 1 is presented in Fig. 5(c). In the log-ratio image, the brighter the region is, the higher changed probability is. From Figs. 5(d) to (i) are
ACCEPTED MANUSCRIPT
19
AN US
CR IP T
4 EXPERIMENTS AND RESULTS
(b) March 2009
(c) Log-ratio
M
(a) June 2006
(e) MRFFCM
(f) CKLD
(h) RBM
(i) SFAE
CE
PT
ED
(d) PCA
AC
(g) MP
Figure 5: Results of Region 1. (a) Image obtained in June 2006. (b) Image obtained in March 2009. (c) Log-ratio image. From (d) to (i) are the results of PCA, MRFFCM, CKLD, MP, RBM, and our SFAE.
ACCEPTED MANUSCRIPT
20
AN US
CR IP T
4 EXPERIMENTS AND RESULTS
(b) March 2009
(c) Log-ratio
M
(a) June 2006
(e) MRFFCM
(f) CKLD
(h) RBM
(i) SFAE
CE
PT
ED
(d) PCA
AC
(g) MP
Figure 6: Results of Region 2. (a) Image obtained in June 2006. (b) Image obtained in March 2009. (c) Log-ratio image. From (d) to (i) are the results of PCA, MRFFCM, CKLD, MP, RBM, and our SFAE.
ACCEPTED MANUSCRIPT
4 EXPERIMENTS AND RESULTS
21
Table 3: Comparisons of the operation times(second) on SAR change detection
Region 1 1.97 26.01 114.42 174.15 122.91 1.10
Region 2 2.16 25.46 115.08 139.71 124.61 1.12
CR IP T
Time(s) PCA[17] MRFFCM[19] CKLD[10] MP[47] RBM[35] SFAE
the results of PCA, MRFFCM, CKLD, MP, RBM, and our SFAE, respectively. Among them, both of the results of PCA and MRFFCM shown in Figs. 5(d) and (e) have many false alarms. While the result of CKLD presented in figure 5(f) has numerous missed
AN US
alarms. As displayed in the bottom of Fig. 5, the bridge under construction and the river bank marked in the Fig. 5(b) are clearly detected by the MP, RBM and SFAE algorithms and the false alarms are obviously fewer than those in the results of PCA, MRFFCM. The similar conclusion can be got from the experiments on the marked Region 2.
The results on Region 2 are presented in figure 6. From figure 6(c), we can see that the
M
changes are mainly in the upper right of Donghu Lake, i.e., the area marked with a red rectangle in Fig. 6(b),where is the new Wuhan railway station for the high-speed trains. The results shown in Figs. 6(d) and (e) seem to have a lot of false alarms. By contrast,
ED
the other four results are comparable. The operation times (unit second) of the related algorithms are compared in table 3.
PT
All of the experiments are implemented with an Intel Core i3 3.2GHz. From table 3, we can see that the computation time of our SFAE is the least, followed by PCA, MRFFCM, CKLD, RBM and MP. The computation time of the SFAE method is less than those of
CE
CKLD, RBM and MP with two orders of magnitude. 4.3. Experiments on PolSAR images
AC
In this subsection, a real L-band PALSAR PolSAR dataset is tested. The PolSAR
dataset contains two PolSAR images with four-look processing which are sensed by ALOS at the area of Tokyo, Japan. The former one is obtained in July 2006 and the latter one is obtained in April 2009. In Fig. 7, the corresponding Pauli images are displayed. The size of the two images are 2290×1050 pixels. Two representative regions with the size of
300×200 are marked in Fig. 8(a).
ACCEPTED MANUSCRIPT
22
M
3
AN US
CR IP T
4 EXPERIMENTS AND RESULTS
AC
CE
PT
ED
4
(a) July 2006
(b) April 2009
Figure 7: The Tokyo dataset.
ACCEPTED MANUSCRIPT
5 CONCLUSIONS
23
Table 4: Comparisons of the operation times(second) on PolSAR change detection
Region 3 0.41 1.30 42.83 0.80
Region 4 0.26 1.33 43.13 0.81
CR IP T
Time(s) Wishart[11] PCD[12] PPCD[23] SFAE
The enlarged images of the Region 3 are presented in Figs. 8(a) and (b). In the
pseudo-color Pauli images, the urban areas are colored in pink or green. Because of the seasonal changes, the summer plants (Fig. 8(a)) are shown in red while the spring
plants (Fig. 8(b)) are shown in purple. To compare the performance of the mentioned
AN US
algorithms, the alpha and entropy ratio images are used as the reference in this paper. The corresponding alpha and entropy ratio images of the marked region 3 are presented in figures 8(c) and (d). In the ratio images, the brighter the region is, the higher changed probability is. The results of Wishart and PCD displayed in figures 8(e) and (f) have a lot of false alarms. In contrast, the false alarms in Figs. 8(g) and (h) are much smaller
than that of the others. The similar conclusions can be reached in the experiments as
M
shown in Fig. 9. From Fig. 9, we can see that the changed areas are clearly reflected in the ratio images. The changed areas have been detected by the Wishart detector and
ED
PCD with lots of false alarms. From the results presented in Figs. 8 and 9, we can see that the change detection results of the PPCD and the proposed SFAE are significantly better than that of the Wishart detector and PCD.
PT
Table 4 compares the computation times (s) of the related algorithms. From table 4, we can find that all of the algorithms except PPCD have a fast computational speed.
CE
The computation time of the PPCD is about 50 times more than that of the proposed SFAE. Above all, when compared with the Wishart detector, PCD and the PPCD, the
AC
proposed SFAE has a significant advantage on PolSAR change detection. 5. Conclusions In this paper, a new SFAE change detection algorithm is proposed for multitempo-
ral SAR systems. In the proposed SFAE method, the network is trained on the labeled synthetic dataset with supervision and tested on the unlabeled dataset without supervision in real-time. In this paper, the original stacked autoencoder is expanded to suit
ACCEPTED MANUSCRIPT
24
(b) April 2009
(c) Alpha Ratio
(d) Entropy Ratio
(g) PPCD
(h) SFAE
PT
ED
M
(a) July 2006
AN US
CR IP T
5 CONCLUSIONS
(e) Wishart
(f) PCD
AC
CE
Figure 8: Results of Region 3. (a) Image obtained in July 2006. (b) Image obtained in April 2009. (c) Alpha ratio image. (d) Entropy ratio image. From (e) to (h) are the results of Wishart, PCD, PPCD, and our SFAE.
ACCEPTED MANUSCRIPT
25
(b) April 2009
(c) Alpha Ratio
(d) Entropy Ratio
(g) PPCD
(h) SFAE
PT
ED
M
(a) July 2006
AN US
CR IP T
5 CONCLUSIONS
(e) Wishart
(f) PCD
AC
CE
Figure 9: Results of Region 4. (a) Image obtained in July 2006. (b) Image obtained in April 2009. (c) Alpha ratio image. (d) Entropy ratio image. From (e) to (h) are the results of Wishart, PCD, PPCD, and our SFAE.
ACCEPTED MANUSCRIPT
5 CONCLUSIONS
26
the environment with the multiplicative noise in SAR change detection. By introducing the Fisher discrimination, the features extracted by SFAE are more discriminative than the original stacked autoencoder. The results on the simulated and real datasets indicate that the proposed SFAE method has a significant advantage on detecting the changes
CR IP T
of multitemporal SAR/PolSAR systems. As a result, the proposed algorithm is a very
practical and efficient change detection method on dealing with the massive amounts of multitemporal SAR/PolSAR images. In this paper, only two temporal SAR images have
been utilized in the SFAE framework. In the future work, the features of the image sequences should be exploited to improve the performance of change detection for three or
Appendix
AN US
more temporal SAR images.
(a) The Derivation of the Equation (14)
=
∂ (l+1) ∂zi
γ log
L1 L2
h(l−1) y(l)
2
+1
!
− log
h(l−1) y(l)
(l) 2 +λ2 tr(Sw (h ) − Sb (h )) + η h 2 ! 2 sl (l−1) (l−1) X h L ∂ 1 k − log hk γ log + 1 = (l+1) (l) (l) L2 ∂zi yk yk k=1 2 sl (l−1) (l−1) X h ∂ L1 hk k + 1 − log = (l+1) γ log (l+1) L2 f z (l+1) ∂z f z (l)
(l)
PT
ED
"
M
(l+1) δi
i
k=1
k
(l−1)
k
2
(l)
CE
+ L2 yi −2γL1 (L1 + L2 ) hi (l+1) 0 , = 2 2 f zi L2 (l−1) (l) (l) L1 hi yi + L 2 yi
AC
where sl is the input vector size of the l-th layer.
(b) The Derivation of the Equation (15)
Rewrite Eq. (11) : Sw (h) =
m1 X i=1
(hi − c1 )(hi − c1 )T +
m X
(hi − c2 )(hi − c2 )T
i=m1 +1
ACCEPTED MANUSCRIPT
5 CONCLUSIONS
27 T
Sb (h) = (c1 − c2 ) (c1 − c2 )
(n)
∂zi
T r (Sw (h) − Sb (h)) = + −
∂ (n)
∂zi
i=1
∂ (n)
∂zi
Tr
m1 X
Tr
(f (zi ) − c1 )((f (zi ) − c1 )
X
T
(hi − c2 )(hi − c2 )
i=m1 +1
T T r (c − c ) (c − c ) , 1 2 1 2 (n)
∂ ∂zi
m1 X (f (zi ) − c1 )((f (zi ) − c1 )T T r (n)
∂ ∂zi
T
!
!
!
AN US
∂
CR IP T
The partial derivative of the trace of difference Fisher is
i=1
1 0 = 2 (f (zi ) − c1 ) f (zi ) − f (zi ) , m1 ! m X ∂ T Tr (hi − c2 )(hi − c2 ) (n) ∂zi i=m1 +1 1 = 2 (f (zi ) − c2 ) f (zi )0 − f (zi )0 , m2
M
0
∂
CE
PT
ED
T r (Sw (h) − Sb (h)) (n) ∂zi h i 2 (hi − c1 ) 1 − m11 − m11 (c1 − c2 ) f (zi )0 if, i ∈ {1, ..., m1 } = i h ; 2 (hi − c2 ) 1 − m12 − m12 (c1 − c2 ) f (zi )0 if, i ∈ {m1 + 1, ..., m}.
AC
Then, for the node i in the hidden layer l, the residual error is (l) δi
∂
"
L1 L2
h(l−1) y(l)
2
!
h(l−1) y(l)
γ log + 1 − log (l) ∂zi
2
+λ2 tr(Sw (h(l) ) − Sb (h(l) )) + η h(l) 2 T 0 (l) (l) (l) (l) f zi . = wi δ (l+1) + λ2 q(hi ) + 2ηhi
=
ACCEPTED MANUSCRIPT
REFERENCES
28
Acknowledgment This work was supported in part by the National Key Research and Development Program of China under Grant 2018YFB1107400, in part by the National Natural Science
CR IP T
Foundation of China under Grant 61871470 and Grant U1604153, in part by the Key Specialized Research and Development Breakthrough of Henan Province under Grant 192102210121, and in part by the Program for Science and Technology Innovation Talents in Universities of Henan Province under Grant 19HASTIT026. References
AN US
[1] J.-S. Lee, Digital image enhancement and noise filtering by use of local statistics, IEEE Trans. Pattern Anal. Mach. Intell. (2) (1980) 165–168.
[2] C.-A. Deledalle, L. Denis, F. Tupin, Iterative weighted maximum likelihood denoising with probabilistic patch-based weights, IEEE Trans. Image Process. 18 (12) (2009) 2661–2672.
M
[3] J. Chen, Y. Chen, W. An, Y. Cui, J. Yang, Nonlocal filtering for polarimetric SAR data: A pretest approach, IEEE Trans. Geosci. Remote Sens. 49 (5) (2011) 1744–
ED
1754.
[4] F. Bovolo, L. Bruzzone, A detail-preserving scale-driven approach to change detection in multitemporal SAR images, IEEE Trans. Geosci. Remote Sens. 43 (12) (2005)
PT
2963–2972.
[5] F. Bovolo, C. Marin, L. Bruzzone, A hierarchical approach to change detection in
CE
very high resolution SAR images for surveillance applications, IEEE Trans. Geosci. Remote Sens. 51 (4) (2013) 2042–2054.
AC
[6] C. Marin, F. Bovolo, L. Bruzzone, Building change detection in multitemporal very high resolution SAR images, IEEE Trans. Geosci. Remote Sens. 53 (5) (2015) 2664– 2682.
[7] Y. Ban, O. Yousif, et al., Multitemporal spaceborne SAR data for urban change detection in china, IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 5 (4) (2012) 1087–1094.
ACCEPTED MANUSCRIPT
REFERENCES
29
[8] D. C. Zanotta, V. Haertel, Gradual land cover change detection based on multitemporal fraction images, Pattern Recognition 45 (8) (2012) 2927–2937. [9] F. Chatelain, J.-Y. Tourneret, J. Inglada, A. Ferrari, Bivariate gamma distributions (2007) 1796–1806.
CR IP T
for image registration and change detection, IEEE Trans. Image Process. 16 (7)
[10] J. Inglada, G. Mercier, A new statistical similarity measure for change detection in multitemporal SAR images and its extension to multiscale change analysis, IEEE Trans. Geosci. Remote Sens. 45 (5) (2007) 1432–1445.
AN US
[11] K. Conradsen, A. A. Nielsen, J. Schou, H. Skriver, A test statistic in the complex
wishart distribution and its application to change detection in polarimetric SAR data, IEEE Trans. Geosci. Remote Sens. 41 (1) (2003) 4–19.
[12] A. Marino, S. R. Cloude, J. M. Lopez-Sanchez, A new polarimetric change detector in radar imagery, IEEE Trans. Geosci. Remote Sens. 51 (5) (2013) 2986–3000.
M
[13] D. Ciuonzo, C. Vincenzo, D. M. Antonio, On multiple covariance equality testing with application to SAR change detection, IEEE Trans. Signal Process. 65 (19) (2017)
ED
5078–5091.
[14] V. Akbari, S. N. Anfinsen, A. P. Doulgeris, T. Eltoft, G. Moser, S. B. Serpico, Polarimetric SAR change detection with the complex hotelling-lawley trace statistic,
PT
IEEE Trans. Geosci. Remote Sens. 54 (7) (2016) 3953–3966. [15] V. T. Vu, N. R. Gomes, M. Pettersson, P. Dammert, H. Hellsten, Bivariate gamma
CE
distribution for wavelength-resolution SAR change detection, IEEE Trans. Geosci. Remote Sens. 57 (1) (2019) 473–481.
AC
[16] W. Yang, X. Yang, T. Yan, H. Song, G.-S. Xia, Region-based change detection for polarimetric SAR images using wishart mixture models, IEEE Trans. Geosci. Remote Sens. 54 (11) (2016) 6746–6756.
[17] T. Celik, Unsupervised change detection in satellite images using principal component analysis and k-means clustering, IEEE Geosci. Remote Sens. Lett. 6 (4) (2009) 772–776.
ACCEPTED MANUSCRIPT
REFERENCES
30
[18] Y. Zheng, L. Jiao, H. Liu, X. Zhang, B. Hou, S. Wang, Unsupervised saliency-guided SAR image change detection, Pattern Recognition 61 (2017) 309–326. [19] M. Gong, L. Su, M. Jia, W. Chen, Fuzzy clustering with a modified MRF energy
CR IP T
function for change detection in synthetic aperture radar images, IEEE Trans. Fuzzy Syst. 22 (1) (2014) 98–109.
[20] E. M. Domínguez, E. Meier, D. Small, M. E. Schaepman, L. Bruzzone, D. Henke, A multisquint framework for change detection in high-resolution multitemporal SAR images, IEEE Trans. Geosci. Remote Sens. 56 (6) (2018) 3611–3623.
[21] X. Zhang, L. Jiao, F. Liu, L. Bo, M. Gong, Spectral clustering ensemble applied to
AN US
SAR image segmentation, IEEE Trans. Geosci. Remote Sens. 46 (7) (2008) 2126– 2136.
[22] Y. Bazi, L. Bruzzone, F. Melgani, An unsupervised approach based on the generalized gaussian model to automatic change detection in multitemporal SAR images, IEEE Trans. Geosci. Remote Sens. 43 (4) (2005) 874–887.
M
[23] G. Liu, L. Jiao, F. Liu, H. Zhong, S. Wang, A new patch based change detector for
ED
polarimetric SAR data, Pattern Recognit. 48 (3) (2015) 685–695. [24] T. T. Lê, A. M. Atto, E. Trouvé, A. Solikhin, V. Pinel, Change detection matrix for multitemporal filtering and change analysis of SAR and PolSAR image time series,
PT
ISPRS J. Photogramm. Remote Sens. 107 (2015) 64–76. [25] A. Marino, I. Hajnsek, A change detector based on an optimization with polarimetric
CE
SAR imagery, IEEE Trans. Geosci. Remote Sens. 52 (8) (2014) 4781–4798. [26] G. E. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets,
AC
Neural Computation 18 (7) (2006) 1527–1554.
[27] R. Salakhutdinov, A. Mnih, G. Hinton, Restricted boltzmann machines for collaborative filtering, in: Proc. of the 24th Int. Conf. on Machine Learning, ACM, 2007, pp. 791–798.
[28] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324.
ACCEPTED MANUSCRIPT
REFERENCES
31
[29] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion, The Journal of Machine Learning Research 11 (2010) 3371–3408.
CR IP T
[30] A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, J. Schmidhuber, A novel connectionist system for unconstrained handwriting recognition, IEEE Trans. Pattern Anal. Mach. Intell. 31 (5) (2009) 855–868.
[31] D. Ciresan, U. Meier, J. Schmidhuber, Multi-column deep neural networks for image
classification, in: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit.
AN US
(CVPR), IEEE, 2012, pp. 3642–3649.
[32] J. Xie, L. Xu, E. Chen, Image denoising and inpainting with deep neural networks, in: Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2012, pp. 341–349. [33] M. Gong, X. Niu, P. Zhang, Z. Li, Generative adversarial networks for change detection in multispectral imagery, IEEE Geosci. Remote Sens. Lett. 14 (2) (2017)
M
2310–2134.
[34] L. Mou, L. Bruzzone, X. Zhu, Learning spectral-spatial-temporal features via a recurrent convolutional neural network for change detection in multispectral imagery,
ED
IEEE Trans. Geosci. Remote Sens. 57 (2) (2019) 924–935. [35] M. Gong, J. Zhao, J. Liu, Q. Miao, L. Jiao, Change detection in synthetic aperture
PT
radar images based on deep neural networks, IEEE Trans. Neural Netw. Learn. Syst. 27 (1) (2016) 125–1382.
CE
[36] F. Liu, L. Jiao, X. Tang, S. Yang, W. Ma, B. Hou, Local restricted convolutional neural network for change detection in polarimetric SAR images, IEEE Trans. Neural
AC
Netw. Learn. Syst. 30 (3) (2019) 818–833.
[37] L. Su, M. Gong, P. Zhang, M. Zhang, J. Liu, H. Yang, Deep learning and mapping based ternary change detection for information unbalanced images, Pattern Recognition 66 (C) (2017) 213–228.
ACCEPTED MANUSCRIPT
REFERENCES
32
[38] C. P. Marc’Aurelio Ranzato, S. Chopra, Y. LeCun, Efficient learning of sparse representations with an energy-based model, in: Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2007.
CR IP T
[39] R. Socher, J. Pennington, E. H. Huang, A. Y. Ng, C. D. Manning, Semi-supervised recursive autoencoders for predicting sentiment distributions, in: Proc. of the Conf. on Empirical Methods in Natural Language Proc., Association for Computational Linguistics, 2011, pp. 151–161.
[40] J.-S. Lee, M. R. Grunes, G. De Grandi, Polarimetric SAR speckle filtering and its
implication for classification, IEEE Trans. Geosci. Remote Sens. 37 (5) (1999) 2363–
AN US
2373.
[41] J.-S. Lee, E. Pottier, Polarimetric radar imaging: from basics to applications, CRC press, 2009.
[42] A. Moreira, P. Prats-Iraola, M. Younis, G. Krieger, I. Hajnsek, K. P. Papathanassiou, 1 (1) (2013) 6–43.
M
A tutorial on synthetic aperture radar, IEEE Geosci. and Remote Sens. Magazine
[43] W. Dierking, H. Skriver, Change detection for thematic mapping by means of air-
ED
borne multitemporal polarimetric SAR imagery, IEEE Trans. Geosci. Remote Sens. 40 (3) (2002) 618–636.
PT
[44] B. Hou, Q. Wei, Y. Zheng, S. Wang, Unsupervised change detection in SAR image based on gauss-log ratio image fusion and compressed projection, IEEE J. Sel. Topics
CE
Appl. Earth Observ. Remote Sens. 7 (8) (2014) 3297–3317. [45] R. G. White, Change detection in SAR imagery, Int. J. Remote Sens. 12 (2) (1991)
AC
339–360.
[46] S. Hachicha, F. Chaabane, On the SAR change detection review and optimal decision, Int. J. Remote Sens. 35 (5) (2014) 1693–1714.
[47] Y. Li, M. Gong, L. Jiao, L. Li, R. Stolkin, Change-detection map learning using matching pursuit, IEEE Trans. Geosci. Remote Sens. 53 (8) (2015) 4712–4723.
ACCEPTED MANUSCRIPT
REFERENCES
33
[48] P. Lv, Y. Zhong, J. Zhao, L. Zhang, Unsupervised change detection based on hybrid conditional random field model for high spatial resolution remote sensing imagery, IEEE Trans. Geosci. Remote Sens. 56 (7) (2018) 4002–4015.
CR IP T
[49] D. Tao, X. Li, X. Wu, S. J. Maybank, General tensor discriminant analysis and gabor features for gait recognition, IEEE Trans. Pattern Anal. Mach. Intell. 29 (10) (2007) 1700–1715.
[50] Q. Du, Modified fisher’s linear discriminant analysis for hyperspectral imagery, IEEE Geosci. Remote Sens. Lett. 4 (4) (2007) 503.
AN US
[51] A. Ng, Sparse autoencoder, Vol. 72, CS294A Lecture notes, 2011.
[52] R. Touzi, A. Lopes, P. Bousquet, A statistical and geometrical edge detector for sar images, IEEE Trans. Geosci. Remote Sens. 26 (6) (1988) 764–773. [53] R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification, John Wiley & Sons, 2012.
M
[54] H. Wang, S. Yan, D. Xu, X. Tang, T. Huang, Trace ratio vs. ratio trace for dimensionality reduction, in: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern
ED
Recognit. (CVPR), IEEE, 2007, pp. 1–8. [55] M. Yang, L. Zhang, X. Feng, D. Zhang, Sparse representation based fisher discrimination dictionary learning for image classification, Int. J. Comput. Vis. 109 (3) (2014)
PT
209–232.
CE
Author Biography
Ganchao Liu received the Ph.D. degree from Xidian University, Xi’an, China, in
AC
2016. He is currently a postdoctoral research fellow with the Center for OPTical IMagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University, Xi’an, China. His current interests include image processing and pattern recognition. Lingling Li received the B.S. and Ph.D. degrees from Xidian University, Xi’an, China,
in 2011 and 2017 respectively. Between 2013 - 2014, she was an exchange Ph.D. student
ACCEPTED MANUSCRIPT
REFERENCES
34
with the Intelligent Systems Group, Department of Computer Science and Artificial Intelligence, University of the Basque Country UPV/EHU, Spain. She is currently a postdoctoral researcher in the School of Artificial Intelligence at Xidian University. Her current research interests include quantum evolutionary optimization, machine learning and deep
CR IP T
learning.
Licheng Jiao received the Ph.D. degrees from Xi’an Jiaotong University, Xi’an,
China, in 1990, respectively. He is a Professor in the School of Artificial Intelligence
at Xidian University. He is a Fellow of IEEE. His research interests include intelligent information processing, image processing, natural computation, machine learning and
AN US
pattern recognition.
Yongsheng Dong received his Ph. D. degree in applied mathematics from Peking University in 2012. He was a postdoctoral research fellow with the Center for Optical Imagery Analysis and Learning, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an, China from 2013 to 2016. From 2016 to 2017, he was a visiting research fellow at the School of Computer Science and Engineering, Nanyang
M
Technological University, Singapore. He is currently an associate professor with the School of Information Engineering, Henan University of Science and Technology, China.
ED
His current research interests include pattern recognition, machine learning, and computer vision. He has authored and co-authored over 30 papers at famous journals and conferences, including IEEE TIP, IEEE TNNLS, IEEE TCYB, IEEE TIE, IEEE TCSVT,
PT
and ACM TIST. He is an associate editor of Neurocomputing. Xuelong Li is a full professor with the School of Computer Science and the Center
CE
for OPTical IMagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical
AC
University, Xi’an 710072, P.R. China. He is a Fellow of IEEE.