Adaptive detection of a subspace signal in Gaussian noise and rank-one interference

Digital Signal Processing 96 (2020) 102610

Contents lists available at ScienceDirect

Digital Signal Processing www.elsevier.com/locate/dsp

Adaptive detection of a subspace signal in Gaussian noise and rank-one interference Zuozhen Wang School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan 611731, China

a r t i c l e

i n f o

Article history: Available online 31 October 2019 Keywords: Adaptive subspace detection Nonhomogeneous environment Rank-one interference Generalized likelihood ratio test

a b s t r a c t Adaptive detection of a subspace signal is studied in this paper, in nonhomogeneous environment which is due to a rank-one interference that disturbs the primary data but not the secondary data. As no signal component lies in the signal-free subspace, all the data are projected onto the signal subspace. This projection maintains the signal, removes the contributions of noise and interference outside the signal subspace, reduces the dimension of the detection problem and the demand for secondary data volume. Based on these projected data, the generalized likelihood ratio test is derived, which is a generalization of the existing detectors for homogeneous environment. Numerical examples show that the derived detector, although deeply affected by the interference, outperforms its counterparts. © 2019 Elsevier Inc. All rights reserved.

1. Introduction Adaptive detection of a subspace signal in Gaussian disturbance (thermal noise, noise-like interference, clutter, etc.) with unknown covariance matrix (CM) is a fundamental issue in the field of radar signal processing. The detection problem is a binary hypothesis test. In this test, we discriminate between the H 0 /null hypothesis (target is absent) and the H 1 /alternative hypothesis (target is present), based upon the primary and secondary data. The primary data that recorded from cells under test (CUTs) contain disturbance and/or signal of interest (SOI). The secondary data that recorded from cells adjacent to the CUTs contain only independent and identically distributed (i.i.d.) disturbance. It is mainly used to estimate the unknown disturbance CM (DCM) of the primary data. This estimation is valid if and only if the relation between the DCMs of the primary and secondary data is specified. The widely studied situation is the homogeneous environment (HE) where the primary and secondary data have the same DCM. Many famous detectors have been designed in HE. In [1], the Kelly’s generalized likelihood ratio test (KGLRT) is derived by resorting to an one-step design procedure, i.e., all unknown parameters (the amplitude of target echo and DCM, etc.) are replaced by their maximum likelihood estimations (MLEs) within one step. F.C. Robey et al. [2] reconsider the problem in [1] and design the adaptive matched filter (AMF) by using an ad hoc two-step GLRT. Namely, the DCM is assumed to be known a priori and then replaced by its MLE after the GLRT statistic is obtained. The AMF is not as effective

E-mail address: [email protected]. https://doi.org/10.1016/j.dsp.2019.102610 1051-2004/© 2019 Elsevier Inc. All rights reserved.

as the KGLRT, while it is more concise and has better calculation efficiency than the KGLRT. The detection problem in [1,2] is also studied by using the Wald test in [3], and the Rao test in [4]. The Wald-based detector in [3] is the AMF. The homogeneous Raobased detector (H-Rao) in [4] is not as effective as the KGLRT and AMF in general, but it has better rejection capability of side-lobe targets than the KGLRT and AMF. If slightly mismatched signatures of main-lobe targets occur [5–8] or multiple polarimetric channels are available for target detection simultaneously [9–12], the SOI is more reasonable to be modeled as a multichannel signal which lies in a subspace spanned by a known full-column-rank matrix. When extended to the multichannel versions, the KGLRT, AMF and H-Rao become the polarization-space-time GLR (PST-GLR) [9], the polarimetric AMF (PAMF) [10] and multichannel H-Rao (MH-Rao) [11,12], respectively. The HE is an ideal situation which might not be met in practices. For instant, the CUTs and their adjacent cells divide various terrains, the disturbance returns with respect to (w.r.t.) different cells are possible to share the same spectral property but different power levels. Thus a more general case that has been widely studied is the partially HE (PHE) where the DCMs of the primary and secondary data are the same up to an unknown scaling factor. This scaling factor accounts for the disturbance power level mismatch between the primary and secondary data. The GLRT-based detector in PHE is the famous adaptive coherence estimator (ACE) [13,14]. Remarkably, A. De Maio et al. [15] and W.J. Liu et al. [16] show that the ACE is the unique detector in PHE whichever design technique is used. The multichannel version of the ACE is the adaptive subspace detector (ASD) [17].

2

Z. Wang / Digital Signal Processing 96 (2020) 102610

In realistic situations, however, neither HE nor PHE is possible to be encountered due to the heterogeneity, i.e., DCM mismatch, between the primary and secondary data. This mismatch leads to nonhomogeneous environment (NHE). A possible reason for this NHE is that a rank-one interference (ROI), for instance, noise-like interference, disturbs the primary data but not the secondary data [18–20]. In this case, the secondary data are useless unless the relation between the DCMs of the primary and secondary data is specified. In [18–20], the ROI is assumed to be independent of the noise, hence the DCM of the primary data equals the DCM of the secondary data plus a rank-one matrix spanned by the ROI. O. Besson [18] proves that the ACE is the suboptimal detector for the aforementioned scenario in the sense of GLRT. Aiming to design a detector with enhanced selectivity properties in HE, D. Orlando [19] introduces a fictitious ROI in the primary data but not in the secondary data. Hence the data model in [19] becomes the one in [18]. Based on the derivations in [18], the double-normalized AMF (DN-AMF) is derived in [19] via the Rao test. When compared with the whitened adaptive beamformer orthogonal rejection test (W-ABORT) [21] in HE, the DN-AMF achieves larger probabilities of detection (PDs) if sufficient secondary data are used, and has enhanced rejection capabilities of mismatched signals if, for instance, the signal-to-noise ratio (SNR) is sufficiently large. W.J. Liu et al. [20] reconsider the problem in [19] by resorting to the Wald test and prove that the Wald-based detector is the DN-AMF. The multichannel version of the detection problem in [18–20] is studied in [22]. Therein a structured deterministic interference appears in the primary data but not in the secondary data. If the structured deterministic interference is absent, the GLRT-, Rao- and Wald-based detectors derived in [22] degrade to the ASD, double-normalized PAMF (DN-PAMF) and PAMF, which are the subspace generalizations of the ACE, DN-AMF and AMF, respectively. For the detection problem studied in [18–20], the ROI has the minimum influence to SOI when the generalized eigenrelation (GER) holds. The GER is originally introduced to model undernulled interference [23,24]. When the interference has one-rank (such as the ROI in this paper), it is orthogonal to the SOI in the whitened space. The detection problem involving GER has been fully investigated in [25–27], where the derived detectors are the ones derived in HE. Precisely, the GLRT is the KGLRT [25], the Rao and Wald tests are the H-Rao and AMF [26], respectively. The multichannel versions of the GLRT, Rao and Wald tests are the PST-GLR, MH-Rao and PAMF [27], respectively. Numerical results in [27] show that the PST-GLR outperforms the PAMF and MH-Rao in NHE characterized by the GER. The GER is also introduced in [28], where the ROI is deterministic rather than random. Although the detection problem involving a ROI have been fully studied in [18–20,25–27], we have the concern that the ROI results in considerable performance degradation and the detectors mentioned above have constraints on the secondary data volume K . Precisely, K ≥ M is needed to form a nonsingular MLE of the DCM, where M is the dimension of the observation subspace. And K ≥ 2M is required when reliable detection performance is expected [29,30]. Considering that sufficient homogeneous secondary data are usually difficult to be accessed in practice, target detection in NHE is reconsidered in this paper based upon limited secondary data. In order to deal with a general case of surprise ROI, the GER is not introduced in this paper. The whole observation subspace is composed of two complementary subspaces, i.e., the signal and signal-free subspaces, target detection can be separately conducted within those two subspaces. Namely, a target is present if it is detected within either of the two subspaces, and no target is present if none detection is triggered within both of the two subspaces. Whereas the signal-free subspace is free of SOI, target detection should be conducted within the signal subspace alone. Hence the primary and secondary data

are projected onto the signal subspace. After this projection, the SOI is preserved, the contributions of noise and ROI outside the signal subspace are removed. With the aid of singular value decomposition (SVD), the detection problem is compressed to a new/lower-dimensional one within the signal subspace. This new detection problem needs less secondary data to form a nonsingular MLE of the noise CM. Finally, a GLRT-based detector is designed in this paper. Remarkably, this detector shares the same test statistic as those of the subspace transformation-based detector in HE (STBD-HE) [31] and the ASD for system-dependent clutter (SDCASD) [32,33]. This GLRT-based detector in NHE has a similar design procedure as that of the STBD-HE, it is labeled as the STBD-NHE. Although sharing the same test statistic, the STBD-HE, SDC-ASD and STBD-NHE are designed for different scenarios. The STBD-HE and SDC-ASD are derived in HE, the STBD-NHE is derived in NHE. Therefore the research in this paper is a supplement/generalization of those in [31–33]. Besides, the computational complexity of the STBD-NHE is discussed. The detection performance of the STBD-NHE in NHE, which is not given in [31–33], is assessed from several aspects. For example, the PDs under different SNRs and interference-to-noise ratios (INRs), the sensitiveness to secondary data volume, etc. The ASD, PST-GLR, DN-PAMF, PAMF and MH-Rao are also considered for comparison purpose. The remainder of this paper is organized as follows. Sec. 2 formulates the detection problem at hand. Detector is designed in Sec. 3. Sec. 4 presents numerical results. Conclusions are drawn in the final section. Notation: Matrices (vectors) are denoted as boldface upper-case (low-case) letters. Cm×n is a set of m × n complex matrices. 0m is an m × 1 zero vector, 0m×n is an m × n zero matrix. I m is an m × m unit matrix. tr (·), (·)−1 , (·) H , | · |, || · ||, λmax (·) and ∈ denote trace, inverse, Hermitian transpose, determinant, Euclidean norm, maximum eigenvalue and “belongs to”, respectively. max f (·) deu

notes the maximum of f (·) w.r.t. u. g ∼ CN (μ, C ) means that g is distribute as Gaussian with mean μ and CM C . Let S ∈ C M × N be a full-column-rank matrix, then  S  is the subspace spanned by the columns of S ,  S ⊥ is the complementary subspace of  S ,



P S = S SH S

−1

S H (P⊥ S = I M − P S ) is the orthogonal projection

matrix w.r.t.  S  ( S ⊥ ).

2. Problem formulation Assume that M s snapshots are taken by a detection system with M a antennas for a single detection, then the primary data are arranged as an M × 1 vector y, where M = M s M a . y contains disturbance (noise plus ROI) under H 0 and contains disturbance plus SOI under H 1 . The disturbance is n and n ∼ CN (0 M , R T ), where R T is the DCM of the primary data. The SOI is modeled as a subspace vector Ax, where A ∈ C M × P is the known full-columnrank signal subspace matrix, x ∈ C P ×1 is an unknown coordinate. A target is present if x = 0 P , otherwise no target is present. As customary, several secondary data y k ’s that contain only i.i.d. noise nk ’s are available, where nk ∼ CN (0 M , R ), k = 1, 2, · · · , K . K is the secondary data volume. R is the noise CM of the secondary data. Based on the above descriptions, the detection problem at hand can be formulated as a binary hypothesis test,

 ⎧ y=n ⎪ ⎪ ⎨ H 0 : y = n , k = 1, 2, · · · , K k  k . ⎪ y = Ax + n ⎪ ⎩ H1 : yk = nk , k = 1, 2, · · · , K

(1)

In NHE, we have R T = R and

R T = R + qq H ,

(2)

Z. Wang / Digital Signal Processing 96 (2020) 102610

where q ∈ C M ×1 accounts for a ROI, such as noise-like interference, and leads to covariance mismatch between the primary and secondary data. The NHE satisfying (2) is different from the one caused by heterogeneous clutter [34–37] where R is assumed to obey an inverse complex Wishart distribution with mean R T . The detection problem at hand is the one in [22] if no deterministic interference is involved. It reduces to the ones in [18–20] if P = 1, and contains the ones in [25–27] involving the GER, i.e., A H R −1 q = 0 P , as special cases. While the procedure to design a detector in this paper is different from those in [18–20,22,25–27]. 3. Detector design In this section, firstly the data model in (1) is reformulated as a reduced-dimensional one with the aid of SVD. Then the GLRT is applied to design a detector. Finally the requirement for K and computational complexity of the derived detector are discussed, in comparison with other detectors of similar kind. 3.1. Problem reformulation The whole observation subspace is composed of two complementary subspaces, i.e., the signal subspace  A  and the signal-free subspace  A ⊥ . Thus target detection can be separately conducted within  A  and  A ⊥ . A target is present if H 1 is positive within  A  or  A ⊥ , and no target is present if H 0 is positive within  A  and  A ⊥ . Since P A + P ⊥ A = I M , the detection problem in (1), can be rewritten as

⎧     ⊥ ⎪ P A + P⊥ ⎪ A y = PA + PA n ⎪ ⎪ ⎨ H 0 :  P + P ⊥  y =  P + P ⊥  n , k = 1, 2, · · · , K A k A k  A  A   . (3) ⊥ Ax + P + P ⊥ n ⎪ P A + P⊥ y = P + P A A ⎪ A A A ⎪ ⎪ ⎩ H 1 :  P + P ⊥  y =  P + P ⊥  n , k = 1, 2, · · · , K A A k k A A From (3), we know that the primary and secondary data can be separately projected onto  A  and  A ⊥ . As P ⊥ A Ax = 0 M and P A Ax = Ax ∈  A , the detection problem in (3) is only expressed within  A , which is

⎧ ⎪ ⎪ ⎪ ⎪ ⎨ H0 : ⎪ ⎪ ⎪ ⎪ ⎩ H1 :

P A y k = P A nk , k = 1, 2, · · · , K

Next the data model in (4) will be compressed to a lowerdimensional one within  A . The SVD of A is given by

A=U



D 0( M − P )× P

V

H

=U



DV H

PA = U

= U

IP

0( M − P )× P

0 P ×( M − P )

0 0

( M −HP )× P  ( M − P )×( M − P ) U1 0( M − P )× P

3.2. GLRT According to [1,9], the GLRT test statistic is expressed as

max f 1 ( Z )

,θ, w

max f 0 ( Z )

(9)

,

,θ

where Z = [ z , z 1 , z 2 , · · · , z K ], f i ( Z ) is the probability density function (PDF) of Z under H i , i = 0, 1. Assume that {n, n1 , n2 , · · · , n K } are independent, then { v , v 1 , v 2 , · · · , v K } are independent, and the PDF of Z under H 1 is expressed as

  ( z − w ) − tr −1 S Z   , π P ( K +1)  + θθ H  || K





exp − ( z − w ) H  + θθ H

K



U 2H



exp −tr −1 S Z



(11)

Maximizing (11) w.r.t. θ is equivalent to minimizing θ H −1 θ w.r.t. θ . Since −1 is positive definite, we have θ H −1 θ ≥ 0 for any θ (with equality holds when θ = 0 P ). Hence the MLE of θ under H 1 is θˆ 1 = 0 P . With this in mind, we rewrite (11) as



θ ,w

(6)

(10)

H k=1 y k y k

  K H π P ( K +1)   + θθ  −1||  . exp −tr  S Z =   π P ( K +1) || K +1 1 + θ H −1 θ

max f 1 ( Z ) =

.

K

U 1H



w

U 1H

− 1

where S Z = = S U 1, S = is the sample CM w.r.t. the secondary data. It can be easily obtained from (10) that the MLE of w under ˆ 1 = z, thus we have H 1 is w

(5)

,

(8)

Note that the dimension of the data model in (7) is P , which is lower than M (the dimension of the original data model in (1)). Hence the data model in (7) is a reduced-dimensional version of the one in (1). Since D and V are invertible, D V H is invertible and w is equivalent to x. Hence a target is present if w = 0 P , otherwise no target is present. In Sec. 3.2, the GLRT will be adopted for the detection problem in (7).

max f 1 ( Z ) =

where U ∈ C M × M and V ∈ C P × P are unitary matrices, D ∈ C P × P is a diagonal matrix whose diagonal elements are the singular values of A. Let U = [U 1 , U 2 ], where U 1 ∈ C M × P and U 2 ∈ C M ×( M − P ) , then A = U 1 D V H and P A is written as



⎧ z = U 1H y ⎪ ⎪ ⎪ ⎪ ⎪ zk = U 1H yk , k = 1, 2, · · · , K ⎪ ⎪ ⎪ H ⎪ ⎪ ⎨ w = DV x H  = U 1 RU 1 . ⎪ ⎪ H ⎪ θ = U1 q ⎪ ⎪   ⎪ ⎪ ⎪ v = U 1H n ∼ CN 0 P ,  + θθ H ⎪ ⎪ ⎩ v k = U 1H nk ∼ CN (0 P , ) , k = 1, 2, · · · , K

H k=1 zk zk

P A yk = P A nk , k = 1, 2, · · · , K

(7)

where

(4)

.

P A y = Ax + P A n



Substituting (5) and (6) into (4), after some algebras, leads to

 ⎧ z=v ⎪ ⎪ H : ⎨ 0 z k = v k , k = 1, 2, · · · , K  , ⎪ z=w+v ⎪ ⎩ H1 : z k = v k , k = 1, 2, · · · , K

f1 (Z ) =

P A y = P An

3



exp −tr −1 S Z



π P ( K +1) || K +1

.

(12)

ˆ 1 , is derived by nulling the The MLE of  under H 1 , noted as  partial derivative of (12) w.r.t. , i.e.,

4

Z. Wang / Digital Signal Processing 96 (2020) 102610

ˆ1= 

1 K +1

(13)

SZ.

Substituting (13) into (12) leads to

 max f 1 ( Z ) =

K +1

 P ( K +1 )

| S Z |−( K +1) .

πe

,θ , w

(14)

Replacing w in (10) with 0 P , we have the PDF of Z under H 0 , which is





exp − z H  + θθ H

− 1



f0 (Z ) =



z − tr −1 S Z





π P ( K +1)  + θθ H  || K



,

  exp g (θ ) − tr −1 S Z + zz H

=



(15)

π P ( K + 1 ) | | K + 1

where

Fig. 1. Block diagram of the STBD-NHE.

g (θ ) =

θ

H

−1 zz H −1 θ

1 + θ H −1 θ





− ln 1 + θ H −1 θ .

(16)

It is obvious that maximizing (15) w.r.t. θ is equivalent to maximizing (16) w.r.t. θ . According to the generalized Rayleigh quotient, we know that

θ

H

−1 zz H −1 θ θ H −1 θ



−1

H

≤ λmax zz 



H

−1

=z 

z,

(17)

H

H

−1

×z 

 −1 θ



−1

H

z − ln 1 + θ 

θ ,

(18)

Substituting θˆ 0 for θ , (16) is rewritten as





max g (θ ) = z H −1 z − ln z H −1 z − 1.

(19)

θ

Equivalently, (15) is rewritten as





exp −1 − tr −1 S Z



π P ( K +1) || K +1 × z H −1 z

(20)

.

Nulling the partial derivative of (20) w.r.t. , then the MLE of

K +1



SZ +

1 ˆ−  0 = ( K + 1)

⎧ ⎨ ⎩

max f 0 ( Z )

=

( K + 1) K +1 KK

(25)

.

−1

ˆ0 z zH 

1 S− Z −

(21)

.

1 H −1 S− Z zz S Z

−1

⎫ ⎬

ˆ 0 z + z H S −1 z ⎭ zH  Z

,

(22)

Premultiplying and postmultiplying (22) by z H and z, respectively, after some algebras, leads to

ˆ 0 z = K × z H S −1 z . zH  Z

(23)

Combining (21) and (23), we obtain the closed-form expression ˆ 0 , which is for 

1 × zH S− Z z,

(26)

equivalently, the GLRT statistic can be formulated as



1 H H zH S− Z z = y U1 U1 SU1

− 1

U 1H y .

(27)

Based on the above derivations, we can easily obtain the test statistics w.r.t. the Rao test, Wald test, two-step procedures of the GLRT, Rao and Wald tests. Remarkably, those test statistics are all equivalent to the one in (27). Since A = U 1 D V H and D V H is invertible, the test statistic in (27) is identical to those of the STBD-HE [31] and SDC-ASD [32,33], i.e.,



− 1



U 1H y = y H A A H S A

− 1

A H y.

(28)

Since the design procedure in this paper is similar to that of the STBD-HE [31], the detector derived in NHE is noted as the STBD-NHE. Let t STBD−NHE and T STBD−NHE be the test statistic and detection threshold of the STBD-NHE, respectively, we write STBDNHE as





zz H

ˆ 0 in (21), we have Taking inverse of 

−1

1 (π e ) P ( K +1) | S Z | K +1 z H S − Z z

y H U 1 U 1H S U 1

ˆ 0 , satisfies  under H 0 , noted as 

ˆ0= 

( K + 1)( P −1)( K +1) K K

,θ

H

1

,θ

,θ , w

portional to z, ii) θˆ 0 −1 θˆ 0 = z H −1 z − 1.

θ

(24)

.

1 K × zH S− Z z

Substituting (24) into (20) leads to

max f 1 ( Z )



with equality holds when θ is the principle eigenvector of zz H −1 , or, when θ is parallel/proportional to z. Nulling the partial derivative of the term on the right hand side of (18) w.r.t. θ H −1 θ , the maximum of the aforementioned term is obtained when θ H −1 θ = z H −1 z − 1. Based on the above analyses, we know that the MLE of θ under H 0 , noted as θˆ 0 , satisfies i) θˆ 0 is the principle eigenvector of zz H −1 , or θˆ 0 is parallel/pro-

max f 0 ( Z ) =



zz H

Substituting (14) and (25) into (9), we have

θ H −1 θ 1+θ

SZ +

K +1

max f 0 ( Z ) =

or

g (θ ) ≤

ˆ0= 

1



t STBD−NHE = y H A A H S A

− 1

H

A H y ≷ H 10 T STBD−NHE ,

(29)

where T STBD−NHE is determined by an expected probability of false alarm (PFA). The block diagram of the STBD-NHE is plotted in Fig. 1. Firstly, the primary and secondary data are premultiplied by A H . Secondly, A H y is quasiwhitened by A H S A to obtain ( A H S A )−1/2 A H y. We say quasiwhitened since A H S A rather than A H R A is used. Thirdly, computing the energy of the quasiwhitened signal ( A H S A )−1/2 A H y, i.e., ( A H S A )−1/2 A H y 2 , results in t STBD−NHE . Finally we obtain the decision result by comparing t STDB−NHE with T STBD−NHE . Although the STBD-HE, SDC-ASD and STBD-NHE share the same test statistic, they are designed for different situations. Precisely, the STBD-HE is a GLRT-based detector in HE where no ROI is involved. And it is designed based on the data filtered by A H . The

Z. Wang / Digital Signal Processing 96 (2020) 102610

SDC-ASD is derived in HE where the disturbance is composed of thermal noise plus system-dependent clutter. And this clutter, which is assumed to be recorded from a detection system whose beam is narrow enough, shares the same subspace with the SOI. The STBD-NHE is derived in NHE where a ROI is present in the primary data but not in the secondary data. Hence the STBD-HE, SDC-ASD and STBD-NHE represent various application scenarios of the same detector. In other words, the three detectors give a full description of the application of the detector whose test statistic is given in (28), and the research in this paper is a supplement of those in [31–33].

Table 1 Computational complexities of different detectors. Detectors





t 1 = y H S −1 A A H S −1 A

− 1

A H S −1 y

t 2 = y H S −1 y

,

(30)

then the ASD, DN-PAMF, PAMF, PST-GLR and MH-Rao are formulated in (31)–(35), respectively,

t ASD =

t1 t2

≷ HH 10 T ASD ,

t DN−PAMF =

t1 t 2 (t 2 − t 1 )

≷ HH 10

T DN−PAMF ,

H

t MH−Rao =

t1 1 + t2

Real additions

ASD

4M 3+ M 2 (2K + 4N  + 3) + M 4N 2 + 2N + 2 +4N 3 + 2N 2 − 4

DN-PAMF

4M 3+ M 2 (2K + 4N  + 4) + M 4N 2 + 4N + 4 +4N 3 + 4N 2 + 4N + 2

4M 3+ M 2 (2K + 4N  + 3) + M 4N 2 + 2N + 2 +4N 3 + 2N 2 − 3

PAMF

4M 3+ M 2 (2K + 4N ) + M 4N 2 + 4N +4N 3 + 4N 2 + 4N

4M 3+ M 2 (2K + 4N − 1) + M 4N 2 + 2N +4N 3 + 2N 2 − 2

PST-GLR

4M 3+ M 2 (2K + 4N  + 4) + M 4N 2 + 4N + 4 +4N 3 + 4N 2 + 4N + 1

4M 3+ M 2 (2K + 4N  + 3) + M 4N 2 + 2N + 2 +4N 3 + 2N 2 − 3

MH-Rao

4M 3+ M 2 (2K + 4N  + 4) + M 4N 2 + 4N + 4 +4N 3 + 4N 2 + 4N + 2

4M 3+ M 2 (2K + 4N  + 3) + M 4N 2 + 2N + 2 +4N 3 + 2N 2 − 2

STBD-NHE

M 2 (2K + 4P )  + M 4P 2 + 4P +4P 3 + 4P 2 + 4P

M 2 (2K + 4P −1) + M 4P 2 + 2P +4P 3 + 2P 2 − 2

test statistics. The PAMF, which is free of t 2 , needs less calculation than the ASD, DN-PAMF, PST-GLR and MH-Rao. The STNB-NHE, which is derived based upon the compressed data in (7), has a lower computational complexity than other detectors. 4. Simulation results

(31)

t PAMF = t 1 ≷ H 10 T PAMF , t PST−GLR =

Real multiplications 4M 3+ M 2 (2K + 4N  + 4) + M 4N 2 + 4N + 4 +4N 3 + 4N 2 + 4N + 1

3.3. Requirement for K and computational complexity If there is no deterministic interference, the detection problem in [22] becomes the one in this paper, the GLRT, Rao and Wald tests in [22] become the ASD, DN-PAMF and PAMF, respectively. The PST-GLR and MH-Rao in [16] are derived for the special case characterized by the GER. Hence the STBD-NHE is compared with the ASD, DN-PAMF, PAMF, PST-GLR and MH-Rao in terms of the demand for K and the computational complexity. Let’s define t 1 and t 2 as

5

(33)

≷ HH 10 T PST−GLR , t1

(1 + t 2 ) (1 + t 2 − t 1 )

(32)

(34)

≷ HH 10 T MH−Rao ,

(35)

where t ASD (T ASD ), t DN−PAMF (T DN−PAMF ), t PAMF (T PAMF ), t PST−GLR (T PST−GLR ) and t MH−Rao (T MH−Rao ) are the test statistics (detection thresholds) of the ASD, DN-PAMF, PAMF, PST-GLR and MH-Rao, respectively. It is obvious that the ASD, DN-PAMF, PAMF, PST-GLR and MHRao are valid only when the sample CM S ∈ C M × M is invertible, hence K ≥ M is needed. The STBD-NHE is valid when A H S A ∈ C P × P is invertible, hence the minimum requirement of the STBDNHE for K is P , which is smaller than M. In other words, the STBD-NHE has a lower demand for K than other detectors mentioned above. And the STBD-NHE is still valid when limited secondary data are available, i.e., K < M. This will be confirmed in Sec. 4.2. The computational complexity is an important evaluation index of a detector, it can be quantified by the numbers of real multiplication and real addition that needed to calculate a test statistic. Assume that the primary and secondary data are organized as complex-valued vectors, the computational complexities of different detectors, as enumerated in Table 1, are derived with the aid of the Table 5.7 on page 240 of [38]. M and P are the dimensions of the observation subspace and signal subspace, respectively. K is the secondary data volume. Note that the explicit expressions of the computational complexity of the STBD-NHE is not presented in [31–33]. Table 1 shows that the ASD, DN-PAMF, PST-GLR and MH-Rao have similar computational complexities since they have similar

Although the STBD-NHE shares the same test statistic as those of the STBD-HE [31] and SDC-ASD [32,33] which are derived in HE, its performance in NHE is not assessed in [31–33]. Hence some numerical experiments are conducted in this section to demonstrate the detection performance of the STBD-NHE in NHE. Precisely, we will mainly investigate the influences of the ROI and the secondary data volume K . For comparison purposes, the ASD, PAMF, PST-GLR and MH-Rao are also considered. Note that the DN-PAMF is not shown below since it is effective when SNR is large and/or sufficient secondary data are available [19,22]. Without loss of generality, we set the dimension of the whole observation subspace M = 8, the dimension of the signal subspace P = 4. PFA = 10−3 . The detection thresholds and PDs are evaluated by resorting to 105 and 103 independent Monte Carlo trials, respectively. In the 105 trials, the signal subspace matrix A, the scattering coefficient vector x, the ROI q and the noise CM R of the secondary data are fixed, while the noise is random. In each of the 103 independent trials, A is randomly generated ensuring that it is a full-column-rank matrix. x is randomly generated ensuing that x = 0 P under H 1 . The elements of R are randomly selected ensuring that R is a Hermitian matrix with positive diagonal elements, each diagonal element is larger than the absolute values of the off-diagonal elements within the same column. q is selected to meet a specified cos2 (φ) defined in (36). Therein φ is the angle between A and q in the whitened space. The parameters { A , x, R , q} are properly scaled to meet specified SNR (defined in (37)) and INR (defined in (38)).



cos2 (φ) =

q H R −1 A A H R −1 A

− 1

q H R −1 q

A H R −1 q

.

(36)

SNR = x H A H R −1 Ax.

(37)

INR = q H R −1 q.

(38)

6

Z. Wang / Digital Signal Processing 96 (2020) 102610

Fig. 2. Contours of constant PDs versus SNR and INR with K = 2M = 16. (a) cos2 (φ) = 0; (b) cos2 (φ) = 0.5; (c) cos2 (φ) = 1.

4.1. Influence of the ROI The NHE considered in this paper is caused by a ROI which is assumed to interfere the primary data but not the secondary data. The influence of this ROI can be quantified by cos2 (φ) defined in (36) and INR defined in (38). Precisely, INR accounts for the power of the ROI in the whitened space, while cos2 (φ) stands for the degree of coincidence between the ROI and SOI in the whitened space. In order to describe the influence of the ROI, the contours of constant PDs versus INR and SNR are plotted in Fig. 2 with different cos2 (φ)’s, i.e., cos2 (φ) = 0, 0.5 and 1 in Figs. 2 (a) – (c), respectively. In Fig. 2, we set PD = [0.7, 0.9, 0.99], K = 2M = 16. SNR is horizontally plotted and it ranges from 10 dB to 30 dB, INR is vertically plotted and it ranges from −10 dB to 20 dB. Aiming to avoid confusion of different curves, the STBD-NHE is compared with the ASD, PAMF, PST-GLR and MH-Rao separately. From Fig. 2, we know that all the detectors suffer from detection performance degradation as either INR or cos2 (φ) increases. While the STBD-NHE outperforms other detectors across wide ranges of SNR and INR. In most of the scenarios, the STBD-NHE is more robust to INR and cos2 (φ) than other detectors, especially when the desired PD is large, such as PD = 0.99. Therefore the STBD-NHE is less sensitive to the ROI than other detectors. Whereas in Fig. 2 (b), the ASD and PST-GLR are more robust to INR than the STBD-NHE when PD = 0.7 and PD = [0.7, 0.9], respectively. Fig. 2 also shows that the STBD-NHE is almost insensitive to INR when INR is relative small, for instance, INR < 0 dB in Fig. 2 (a). Therefore when INR is relative small, the ROI is negligible and the NHE in this paper can be treated as the HE. Correspondingly, the STBD-NHE becomes the STBD-HE whose detection performance has been intensively studied in [31]. Aiming to investigate the detection performance of the STBD-NHE in NHE, we consider the

scenario of heavy interference and set INR = 20 dB in the following simulations, i.e., the ROI is dominant in the overall disturbance. In order to give a full description about the influence of cos2 (φ), the contours of constant PDs across wide ranges of SNR and cos2 (φ) are shown in Fig. 3. Therein PD = [0.7, 0.9, 0.99], INR = 20 dB, K = 2M = 16. The SNR, which is plotted horizontally, ranges from 20 dB to 40 dB. The cos2 (φ), which is plotted vertically, ranges from 0 to 1. The STBD-NHE is compared with other detectors separately. Since the PDs of the MH-Rao are less than 0.7, the contours of the MH-Rao are not presented. The ASD, which is proved to be the suboptimal detector in NHE [22] when no structured interference is involved, is almost insensitive to cos2 (φ) across a wide range of cos2 (φ). Whereas it suffers from severe detection performance degradation when cos2 (φ) approaches to 1. The STBD-NHE, which is not as robust as the ASD against cos2 (φ) when cos2 (φ) is small, guarantees better detection performance, especially when the desired PD is large, such as PD = 0.99. The PAMF has similar behavior as the STBD-NHE when the ROI is encountered. However it is not as effective as the STBD-NHE across wide ranges of SNR and cos2 (φ). The PST-GLR is the suboptimal detector in the sense of GLRT when the GER holds. It ensures better detection performance when cos2 (φ) is large for PD = [0.7, 0.9]. However it is not as effective as the STBD-NHE when the desired PD is large, such as PD = 0.99. Since large PD is always expected in practice, the STBD-NHE is a better choice than other detectors in NHE. 4.2. Influence of the secondary data volume K As shown in Table 1, the previous detectors are valid when K ≥ M, the STBD-NHE can work when K ≥ P . Thus the STBD-NHE has a lower demand for secondary data volume K than other detectors. This is confirmed in Fig. 4 where the case of P ≤ K < M, i.e., K =

Z. Wang / Digital Signal Processing 96 (2020) 102610

7

are shown in Figs. 5 (a) – (c), respectively. Besides, the STBD-NHE is compared with other detectors separately. As shown in Fig. 5, all detectors suffer from detection performance degradation when cos2 (φ) increases. The STBD-NHE, ASD, PAMF and PST-GLR become less sensitive to K when K is relative large. From Fig. 5, we know that the STBD-NHE is superior to other detectors across a wide range of K , especially when the desired PD is large, such as PD = 0.99. Yet in Fig. 5 (c), the PAMF and PSTGLR have their curves approach to those of the STBD-NHE when K increases. And they are better choices than the STBD-NHE when sufficient secondary data are available. For example, the PAMF and PST-GLR outperform the STBD-NHE when K / M > 5 in Fig. 5 (a) for PD = 0.99. The PAMF and PST-GLR outperform the STBDNHE when K / M > 6 and K / M > 3, respectively, in Fig. 5 (b) for PD = 0.99. Notice that the homogeneous secondary data volume is usually not large in practice, due to, for instance, the heterogeneity of surveillance area. Hence the STBD-NHE is a more reliable choice than other detectors. Note that if SNR is sufficiently large, any desired PD can be achieved by a detector when the minimum demand for K is met. Fig. 5 shows that when SNR increases, the curves of the ASD, PAMF and PST-GLR w.r.t. various PDs converge to the same point near K / M = 1. This indicates that the minimum demand of those three detectors for the secondary data volume is K = M. The curves of the STBD-NHE converge to a point which is smaller than K / M = 1. This means that the STBD-NHE has a lower demand for the secondary data volume than other detectors. Although the MH-Rao is valid when K / M ≥ 1, its curves do not converge as SNR increases. This is because that the MH-Rao achieves reliable detection performance when K and/or SNR is relative large [19,22]. 5. Conclusion Fig. 3. Contours of constant PDs versus SNR and cos2 (φ) with INR = 20 dB and K = 2M = 16.

[4, 5, 6, 7], is investigated. cos2 (φ) = 0, 0.5 and 1 are shown in Figs. 4 (a) – (c), respectively. The STBD-NHE, as shown in Fig. 4, requires large SNRs to achieve reliable PDs when K = P = 4. It achieves significant improvement on detection performance when K increases from 4 to 5. Whereas this improvement is slight when K increases from 5 to 6, and from 6 to 7. This indicates that the STBD-NHE is less insensitive to K when K is relative large. In order to give a global view of the influences of SNR and K , the contours of constant PDs versus SNR and K are shown in Fig. 5. Therein PD = [0.7, 0.9, 0.99]. K is vertically plotted and it ranges from 1 to 10M. SNR is horizontally plotted and it ranges from 10 dB to 50 dB. Three scenarios, i.e., cos2 (φ) = 0, 0.5 and 1,

This paper studied adaptive radar detection of a subspace signal in NHE. This NHE is caused by a ROI which disturbs the primary data but not the secondary data. Considering that target detection can be conducted within the signal and signal-free subspaces, whereas the signal-free subspace contains no SOI component, we focused on the signal subspace alone in this paper. With the aid of SVD, the original detection problem was compressed into a lowerdimensional one within the signal subspace. After this compression, the target signature is preserved, the ROI and noise outside the signal subspace is zeroed out, and the demand for homogeneous secondary data volume is alleviated. For this compressed detection problem, the GLRT was adopted and the STBD-NHE was derived in this paper. Remarkably, this STBD-NHE is a generalization of the existing ones derived in HE, i.e., the STBD-HE and SDC-ASD. Therefore the researches in this paper verify that the STBD-HE and SDC-ASD are effective in both HE and NHE, and the

Fig. 4. PDs of the STBD-NHE versus SNR with INR = 20 dB and K = [4, 5, 6, 7]. (a) cos2 (φ) = 0; (b) cos2 (φ) = 0.5; (c) cos2 (φ) = 1.

8

Z. Wang / Digital Signal Processing 96 (2020) 102610

Fig. 5. Contours of constant PDs versus SNR and K with INR = 20 dB. (a) cos2 (φ) = 0; (b) cos2 (φ) = 0.5; (c) cos2 (φ) = 1.

STBD-NHE can be regarded as a supplement of the STBD-HE and SDC-ASD. Although the STBD-NHE has the same test statistic as those of the STBD-HE and SDC-ASD, its detection performance in NHE has not been investigated before. Fortunately, this gap was filled in this paper. Several Monte Carlo simulations were conducted, investigating the influences of the ROI and the secondary data volume. The simulation results show that the ROI does affect the detection performance of all the detectors. While the STBD-NHE is less sensitive to the ROI than other detectors, as confirmed in Figs. 2 and 3. The STBD-NHE has a lower requirement for the secondary data volume than other detectors, and it can work when limited secondary data are available, as verified in Fig. 4 where P ≤ K < M. When K changes in a wide range, such as the case of 1 ≤ K ≤ 10M in Fig. 5, the STBD-NHE is the best choice for large PDs in most of the scenarios. However when sufficient homogeneous secondary data are available, we should choose the PAMF and PST-GLR. The ROI was studied. Yet several interferences may enter the main-lobe from various directions. Hence it is necessary to consider the case of multi-rank interference. Besides, the signal subspace matrix may deviate from the actual one due to array/pointing errors, etc. Therefore it is also worth studying the mismatched signals [39]. Declaration of competing interest There is no conflict of interest. References [1] E.J. Kelly, An adaptive detection algorithm, IEEE Trans. Aerosp. Electron. Syst. 22 (1) (1986) 115–127. [2] F.C. Robey, D.R. Fuhrmann, E.J. Kelly, R. Nitzberg, A CFAR adaptive matched filter detector, IEEE Trans. Aerosp. Electron. Syst. 28 (1) (1992) 208–216. [3] A. De Maio, A new derivation of the adaptive matched filter, IEEE Signal Process. Lett. 11 (10) (2004) 792–793.

[4] A. De Maio, Rao test for adaptive detection in Gaussian interference with unknown covariance matrix, IEEE Trans. Signal Process. 55 (7) (2007) 3577–3584. [5] G.A. Fabrizio, A. Farina, M.D. Turley, Spatial adaptive subspace detection in OTH radar, IEEE Trans. Signal Process. 39 (4) (2003) 1407–1428. [6] O. Besson, L.L. Scharf, F. Vincent, Matched direction detectors and estimators for array processing with subspace steering vector uncertainties, IEEE Trans. Signal Process. 53 (12) (2005) 4453–4463. [7] F. Bandiera, O. Besson, D. Orlando, G. Ricci, L.L. Scharf, GLRT-based direction detectors in homogeneous noise and subspace interference, IEEE Trans. Signal Process. 55 (6) (2007) 2386–2394. [8] J. Liu, Z.J. Zhang, Y.H. Cao, M. Wang, Distributed target detection in subspace interference plus Gaussian noise, Signal Process. 95 (2014) 88–100. [9] H.R. Park, J. Li, H. Wang, Polarization-space-time domain generalized likelihood ratio detection of radar targets, Signal Process. 41 (2) (1995) 153–164. [10] A. De Maio, G. Ricci, A polarimetric adaptive matched filter, Signal Process. 81 (12) (2001) 2583–2589. [11] W. Liu, W. Xie, J. Liu, Y. Wang, Adaptive double subspace signal detection in Gaussian background—Part I: homogeneous environments, IEEE Trans. Signal Process. 62 (9) (2014) 2345–2357. [12] J. Liu, W. Liu, B. Chen, H. Liu, H. Li, C. Hao, Modified Rao test for multichannel adaptive signal detection, IEEE Trans. Signal Process. 64 (3) (2016) 714–725. [13] S. Kraut, L.L. Scharf, The CFAR adaptive subspace detector is a scale-invariant GLRT, IEEE Trans. Signal Process. 47 (9) (1999) 2538–2541. [14] S. Kraut, L.L. Scharf, Adaptive subspace detectors, IEEE Trans. Signal Process. 49 (1) (2001) 1–16. [15] A. De Maio, S. Iommelli, Coincidence of the Rao test, Wald test and GLRT in partially homogeneous environment, IEEE Signal Process. Lett. 15 (2008) 385–388. [16] W. Liu, W. Xie, J. Liu, Y. Wang, Adaptive double subspace signal detection in Gaussian background—Part II: partially homogeneous environments, IEEE Trans. Signal Process. 62 (9) (2014) 2358–2369. [17] J. Liu, Z. Zhang, Y. Yang, H. Liu, A CFAR adaptive subspace detector for firstorder or second-order Gaussian signals based on single observation, IEEE Trans. Signal Process. 59 (11) (2011) 5126–5140. [18] O. Besson, Detection in the presence of surprise or undernulled interference, IEEE Signal Process. Lett. 14 (6) (2007) 352–354. [19] D. Orlando, G. Ricci, A Rao test with enhanced selectivity properties in homogeneous scenarios, IEEE Trans. Signal Process. 58 (10) (2010) 5385–5390. [20] W.J. Liu, W.C. Xie, Y.L. Wang, A Wald test with enhanced selectivity properties in homogeneous environments, EURASIP J. Adv. Signal Process. 2013 (14) (2013).

Z. Wang / Digital Signal Processing 96 (2020) 102610

[21] F. Bandiera, O. Besson, G. Ricci, An abort-like detector with improved mismatched signals rejection capabilities, IEEE Trans. Signal Process. 56 (1) (2008) 14–25. [22] W. Liu, H. Han, J. Liu, H. Li, et al., Multichannel radar adaptive signal detection in interference and structure nonhomogeneity, Sci. China Inf. Sci. 60 (11) (2017) 112302. [23] C.D. Richmond, Performance of a class of adaptive detection algorithms in nonhomogeneous environments, IEEE Trans. Signal Process. 48 (5) (2000) 1248–1262. [24] C.D. Richmond, Statistics of adaptive nulling and use of the generalized eigenrelation (GER) for modeling inhomogeneities in adaptive processing, IEEE Trans. Signal Process. 48 (5) (2000) 1263–1273. [25] O. Besson, D. Orlando, Adaptive detection in non-homogeneous environments using the generalized eigenrelation, IEEE Signal Process. Lett. 14 (10) (2007) 731–734. [26] C. Hao, D. Orlando, C. Hou, Rao and Wald tests for nonhomogeneous scenarios, Sensors 12 (4) (2012) 4730–4736. [27] Z. Shang, Q. Du, Z. Tang, T. Zhang, W. Liu, Multichannel adaptive signal detection in structural nonhomogeneous environment characterized by the generalized eigenrelation, Signal Process. 148 (2018) 214–222. [28] W. Liu, Y.L. Wang, J. Liu, W. Xie, H. Chen, R.F. Li, Design and performance analysis of adaptive subspace detectors in orthogonal interference and Gaussian noise, IEEE Trans. Aerosp. Electron. Syst. 52 (5) (2016) 2068–2079. [29] A. De Maio, D. Orlando, C. Hao, G. Foglia, Adaptive detection of point-like targets in spectrally symmetric interference, IEEE Trans. Signal Process. 64 (12) (2016) 3207–3220. [30] A. De Maio, D. Orlando, I. Soloveychik, A. Wiesel, Invariance theory for adaptive detection in interference with group symmetric covariance matrix, IEEE Trans. Signal Process. 64 (23) (2016) 6299–6312. [31] W.J. Liu, J. Liu, L. Huang, Q.L. Du, Y.L. Wang, Performance analysis of reduceddimension subspace signal filtering and detection in sample-starved environment, J. Franklin Inst. (2018), https://doi.org/10.1016/j.jfranklin.2018.10.017.

9

[32] S. Lei, Z. Zhao, Z. Nie, Q.-H. Liu, A CFAR adaptive subspace detector based on a single observation in system-dependent clutter background, IEEE Trans. Signal Process. 62 (20) (2014) 5260–5269. [33] Z. Wang, M. Li, H. Chen, L. Zuo, P. Zhang, Y. Wu, Adaptive detection of a subspace signal in signal-dependent interference, IEEE Trans. Signal Process. 65 (18) (2017) 4812–4820. [34] C.Y. Chong, F. Pascal, J.P. Ovarlez, M. Lesturgie, MIMO radar detection in nonGaussian and heterogeneous clutter, IEEE J. Sel. Top. Signal Process. 4 (1) (2010) 115–126. [35] S. Bidon, O. Besson, J.Y. Tourneret, Knowledge-aided STAP in heterogeneous clutter using a hierarchical Bayesian algorithm, IEEE Trans. Aerosp. Electron. Syst. 47 (3) (2011) 1863–1879. [36] T.X. Zhang, G.L. Cui, L.J. Kong, X.B. Yang, Adaptive Bayesian detection using MIMO radar in spatially heterogeneous clutter, IEEE Signal Process. Lett. 20 (6) (2013) 547–550. [37] Y.K. Wang, W. Xia, Z.S. He, CFAR knowledge-aided radar detection with heterogeneous samples, IEEE Signal Process. Lett. 24 (5) (2017) 693–697. [38] A.H. Sayed, Fundamentals of Adaptive Filtering, Wiley, New York, 2003. [39] Weijian Liu, Jun Liu, Yongchan Gao, Guoshi Wang, Yong-Liang Wang, Multichannel signal detection in interference and noise when signal mismatch happens, Signal Process. (2019), https://doi.org/10.1016/j.sigpro.2019.107268.

Zuozhen Wang received the B.S. degree and M.S. degree in electronic engineering at University of Electronic Science and technology of China (UESTC). Now he is pursuing a Ph.D. degree in the school of information & communication engineering, UESTC. His current research is radar target detection.