On the perturbation of an L2 -orthogonal projection

Journal of Computational and Applied Mathematics xxx (xxxx) xxx

Contents lists available at ScienceDirect

Journal of Computational and Applied Mathematics journal homepage: www.elsevier.com/locate/cam

On the perturbation of an L2 -orthogonal projection Xuefeng Xu Department of Mathematics, Purdue University, West Lafayette, IN 47907, USA

article

info

Article history: Received 16 October 2018 Received in revised form 24 February 2019 MSC: 15A09 15A18 47A55 65F35 Keywords: Orthogonal projection Perturbation Singular value decomposition

a b s t r a c t The L2 -orthogonal projection onto a subspace is an important mathematical tool, which has been widely applied in many fields such as linear least squares problems, eigenvalue problems, ill-posed problems, and randomized algorithms. In some numerical applications, the entries of a matrix will seldom be known exactly, so it is necessary to develop some bounds to characterize the effects of the uncertainties caused by matrix perturbation. In this paper, we establish new perturbation bounds for the L2 -orthogonal projection onto the column space of a matrix, which involve upper (lower) bounds and combined upper (lower) bounds. The new bounds contain some sharper counterparts of the existing ones. Numerical examples are also given to illustrate our theoretical results. © 2019 Elsevier B.V. All rights reserved.

1. Introduction The L2 -orthogonal projection onto a subspace is an important geometric construction in finite-dimensional spaces, which has been applied in many fields such as linear least squares problems, eigenvalue (singular value) problems, illposed problems, and randomized algorithms (see, e.g., [1–12]). However, in some numerical applications, the entries of a matrix will seldom be known exactly. Thus, it is necessary to establish some bounds to characterize the effects arising from matrix perturbation. Over the past decades, many researchers have investigated the stability of an L2 -orthogonal projection and developed various upper bounds to characterize the deviation of an L2 -orthogonal projection after perturbation, which can be found, e.g., in [13–19]. ×n Let Cm×n , Cm , and Un be the set of all m × n complex matrices, the set of all m × n complex matrices of rank r, r and the set of all n × n unitary matrices, respectively. For any M ∈ Cm×n , the symbols M ∗ , M † , rank(M), ∥M ∥U , ∥M ∥F , ∥M ∥2 , and PM denote the conjugate transpose, the Moore–Penrose inverse, the rank, the unitarily invariant norm (see, e.g., [20, Page 357]), the Frobenius norm, the spectral norm, and the L2 -orthogonal projection onto the column space of M (i.e., PM = MM † ), respectively. ×n ×n Let A ∈ Cm , B ∈ Cm , and E = B − A. Sun [14] established the following estimates: r s

( ) ∥PB − PA ∥U ≤ ∥A† ∥2 + ∥B† ∥2 ∥E ∥U , ( ) ∥PB − PA ∥2F ≤ ∥A† ∥22 + ∥B† ∥22 ∥E ∥2F , { } ∥PB − PA ∥2 ≤ max ∥A† ∥2 , ∥B† ∥2 ∥E ∥2 .

(1.1a) (1.1b) (1.1c)

In particular, if s = r, then

{ } ∥PB − PA ∥U ≤ 2 min ∥A† ∥2 , ∥B† ∥2 ∥E ∥U ,

(1.2a)

E-mail addresses: [email protected], [email protected]. https://doi.org/10.1016/j.cam.2019.06.043 0377-0427/© 2019 Elsevier B.V. All rights reserved.

Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

2

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

{ } ∥PB − PA ∥2F ≤ 2 min ∥A† ∥22 , ∥B† ∥22 ∥E ∥2F , { } ∥PB − PA ∥2 ≤ min ∥A† ∥2 , ∥B† ∥2 ∥E ∥2 .

(1.2b) (1.2c)

Recently, Chen et al. [18, Theorems 2.4 and 2.5] improved the above estimates and proved that

∥PB − PA ∥U ≤ ∥EA† ∥U + ∥EB† ∥U , PA 2F

∥PB −

∥ ≤

∥PB − PA ∥2 ≤

(1.3a)

+ ∥EB† ∥2F , { † } max ∥EA ∥2 , ∥EB† ∥2 . ∥EA† ∥2F

(1.3b) (1.3c)

In particular, if s = r, then

{ } ∥PB − PA ∥U ≤ 2 min ∥EA† ∥U , ∥EB† ∥U , { } ∥PB − PA ∥2F ≤ 2 min ∥EA† ∥2F , ∥EB† ∥2F , { } ∥PB − PA ∥2 ≤ min ∥EA† ∥2 , ∥EB† ∥2 .

(1.4a) (1.4b) (1.4c)

Moreover, Chen et al. [18, Theorem 2.8] derived the following combined estimate:

} ( ) ∥A† ∥22 ∥B† ∥22 ∥ + min ∥PB∗ − PA∗ ∥2F ≤ ∥A† ∥22 + ∥B† ∥22 ∥E ∥2F . , ∥B† ∥22 ∥A† ∥22 {

PA 2F

∥PB −

(1.5)

In particular, if s = r, then

} { } ∥A† ∥22 ∥B† ∥22 ∥PB∗ − PA∗ ∥2F ≤ 2 min ∥A† ∥22 , ∥B† ∥22 ∥E ∥2F , , 2 2 † † ∥B ∥2 ∥A ∥2 4∥A† ∥2 ∥B† ∥2 ∥PB − PA ∥2F + ∥PB∗ − PA∗ ∥2F ≤ † 2 2 † 22 ∥E ∥2F . ∥A ∥2 + ∥B ∥2

∥PB − PA ∥2F + min

{

(1.6) (1.7)

More recently, Li et al. [19, Corollary 2.4] showed that

( ) ∥A† ∥2 ∥B† ∥2 ∥PB − PA ∥2F ≤ ∥A† ∥22 + ∥B† ∥22 ∥E ∥2F − † 22 ∥A† E ∥2F − † 22 ∥B† E ∥2F . ∥A ∥2 ∥B ∥2

(1.8)

In particular, if s = r, then

{ } ∥B† ∥2 ∥A† ∥2 ∥PB − PA ∥2F ≤ 2 min ∥B† ∥22 ∥E ∥2F − † 22 ∥A† E ∥2F , ∥A† ∥22 ∥E ∥2F − † 22 ∥B† E ∥2F . ∥A ∥2 ∥B ∥2

(1.9)

In addition, Li et al. [19, Theorem 2.5] obtained the following combined estimate:

{ } ∥A† EB† ∥2F + ∥B† EA† ∥2F { } . ∥PB − PA ∥2F + ∥PB∗ − PA∗ ∥2F ≤ 2 max ∥A† ∥22 , ∥B† ∥22 ∥E ∥2F − min ∥A† ∥22 , ∥B† ∥22

(1.10)

In particular, if s = r, then PA 2F

∥PB −

∥ + ∥PB∗ −

PA∗ 2F

∥ ≤

4∥A† ∥22 ∥B† ∥22

∥A† ∥22 + ∥B† ∥22

2 ∥A† EB† ∥2F + ∥B† EA† ∥2F

(

2 F

∥E ∥ −

∥A† ∥22 + ∥B† ∥22

) .

(1.11)

Although the estimate (1.3b) has improved (1.1b), the upper bound in (1.3b) is still too large in certain cases. We now give a simple example:

( A=

)

1 0

(

0 , 0

B=

ε 1+ε

0

0

10

ε

)

,

(1.12)

where 0 < ε < 1. In this example, it holds that ∥PB − PA ∥2F ≡ 1. Direct computation yields that the upper bound in (1.3b) is 1+

1

ε2

+

1 (1 + ε )2

,

which is very large if 0 < ε ≪ 1. Alternatively, applying (1.8) to the above example, we have that the upper bound for ∥PB − PA ∥2F is 99 100

+

1 (1 + ε )2

.

Obviously, under the setting of (1.12), the upper bound in (1.8) is smaller than that in (1.3b). In [19], Li et al. also demonstrated the superiority of (1.8) (compared with (1.3b)) via some examples. However, it is difficult to compare (1.8) Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

3

with (1.3b) theoretically. Actually, the estimate (1.8) is not always sharper than (1.3b), which can be illustrated by the following example:

(

)

1 0

A=

(1

0 , 0

B=

2

0

)

1 . 1 √

and 18+34 65 , respectively. Therefore, there is no Direct calculations yield that the upper bounds in (1.3b) and (1.8) are 25 4 determined relation between the estimates (1.3b) and (1.8). Motivated by these observations, we revisit the perturbation of an L2 -orthogonal projection under the Frobenius norm. In this paper, we establish new upper bounds for ∥PB − PA ∥2F , which include the counterparts of (1.3b), (1.4b), (1.8), and (1.9). Some new combined upper bounds for ∥PB − PA ∥2F and ∥PB∗ − PA∗ ∥2F are also derived, which contain the counterparts of (1.5), (1.6), (1.7), (1.10), and (1.11). Theoretical analysis shows that the new upper bounds are sharper than the existing ones. On the other hand, we also develop novel lower bounds for ∥PB − PA ∥2F and combined lower bounds for ∥PB − PA ∥2F and ∥PB∗ − PA∗ ∥2F . Furthermore, we give two examples to illustrate the performances of our theoretical results. The rest of this paper is organized as follows. In Section 2, we introduce a trace inequality and several identities on ∥PB − PA ∥2F and ∥PB∗ − PA∗ ∥2F . In Section 3, we present some new perturbation bounds for ∥PB − PA ∥2F and ∥PB∗ − PA∗ ∥2F , which involve upper bounds, lower bounds, combined upper bounds, and combined lower bounds. In Section 4, we exhibit some numerical comparisons between the new bounds and the existing ones. 2. Preliminaries In this section, we introduce a useful trace inequality and several important identities on the deviations ∥PB − PA ∥2F and ∥PB∗ − PA∗ ∥2F . Let M ∈ Cn×n and N ∈ Cn×n be Hermitian matrices. The following lemma provides an estimate for the trace of MN (see, e.g., [20, Theorem 4.3.53]). Lemma 2.1. Let {λi }ni=1 and {µi }ni=1 be the spectra of the Hermitian matrices M ∈ Cn×n and N ∈ Cn×n , respectively, where λ1 ≥ · · · ≥ λn and µ1 ≥ · · · ≥ µn . Then n ∑

λi µn−i+1 ≤ tr(MN) ≤

i=1

n ∑

λi µi .

(2.1)

i=1

Using the singular value decomposition (SVD) of a matrix, we can derive some identities on ∥PB − PA ∥2F and ∥PB∗ − PA∗ ∥2F . ×n ×n Let A ∈ Cm and B ∈ Cm (throughout this paper, we only consider the nontrivial case that r ≥ 1 and s ≥ 1) have the r s following SVDs:

( A=U

Σ1 0

( B=˜ U

˜1 Σ 0

)

0 V ∗ = U1 Σ1 V1∗ , 0

(2.2a)

)

0 ˜∗ ˜1˜ V1∗ , V =˜ U1 Σ 0

(2.2b)

where U = (U1 , U2 ) ∈ Um , V = (V1 , V2 ) ∈ Un , ˜ U = (˜ U1 , ˜ U2 ) ∈ Um , ˜ V = (˜ V1 , ˜ V2 ) ∈ Un , U1 ∈ Cm×r , V1 ∈ Cn×r , ˜ U1 ∈ Cm×s , n × s ˜ ˜1 = diag(˜ V1 ∈ C , Σ1 = diag(σ1 , . . . , σr ), Σ σ1 , . . . , ˜ σs ), σ1 ≥ · · · ≥ σr > 0, and ˜ σ1 ≥ · · · ≥ ˜ σs > 0. In view of (2.2a) and (2.2b), the Moore–Penrose inverses A† and B† can be explicitly expressed as follows: A† = V

(

Σ1−1 0

( −1 ˜1 ˜ Σ

B† = V

0

)

0 U ∗ = V1 Σ1−1 U1∗ , 0

(2.3a)

)

0 ˜∗ ˜1−1˜ U =˜ V1 Σ U1∗ . 0

(2.3b)

By (2.2a), (2.2b), (2.3a), and (2.3b), we have PA = AA† = U1 U1∗ ,

PA∗ = A† A = V1 V1∗ ,

PB = BB† = ˜ U1 ˜ U1∗ ,

PB∗ = B† B = ˜ V1 ˜ V1∗ .

The following lemma (see [18, Lemma 2.3]) is the foundation of our analysis, which gives the expressions for ∥PB − PA ∥2F and ∥PB∗ − PA∗ ∥2F . Lemma 2.2.

×n ×n Let A ∈ Cm and B ∈ Cm have the SVDs (2.2a) and (2.2b), respectively. Then r s

∥PB − PA ∥2F = ∥˜ U1∗ U2 ∥2F + ∥˜ U2∗ U1 ∥2F , ∥PB∗ − PA∗ ∥2F = ∥˜ V1∗ V2 ∥2F + ∥˜ V2∗ V1 ∥2F .

(2.4a) (2.4b)

Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

4

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

In particular, if s = r, then U2∗ U1 ∥2F , U1∗ U2 ∥2F = 2∥˜ ∥PB − PA ∥2F = 2∥˜ V2∗ V1 ∥2F . V1∗ V2 ∥2F = 2∥˜ ∥PB∗ − PA∗ ∥2F = 2∥˜

(2.5a) (2.5b)

Based on Lemma 2.2, we can get the following identities on ∥PB − auxiliary matrices Ui , ˜ Ui , Vi , and ˜ Vi (i = 1, 2). Lemma 2.3.

PA 2F

∥ and ∥PB∗ −

PA∗ 2F ,

∥ which do not involve the

×n ×n Let A ∈ Cm , B ∈ Cm , and E = B − A. Then r s

∥PB − PA ∥2F = ∥EA† ∥2F + ∥EB† ∥2F − ∥BB† EA† ∥2F − ∥AA† EB† ∥2F ,

(2.6a)

∥PB∗ − PA∗ ∥2F = ∥A† E ∥2F + ∥B† E ∥2F − ∥A† EB† B∥2F − ∥B† EA† A∥2F .

(2.6b)

In particular, if s = r, then

) ) ( ( ∥PB − PA ∥2F = 2 ∥EA† ∥2F − ∥BB† EA† ∥2F = 2 ∥EB† ∥2F − ∥AA† EB† ∥2F , ( ) ( ) ∥PB∗ − PA∗ ∥2F = 2 ∥A† E ∥2F − ∥A† EB† B∥2F = 2 ∥B† E ∥2F − ∥B† EA† A∥2F .

(2.7a) (2.7b)

Proof. By (2.2a), (2.2b), (2.3a), and (2.3b), we have

) ˜1˜ Σ V1∗ V1 Σ1−1 − ˜ U1∗ U1 0 , U EA U = −˜ U2∗ U1 0 ) ( ˜1˜ V1∗ V1 Σ1−1 − ˜ U1∗ U1 0 Σ ˜ . U ∗ BB† EA† U = (

˜∗

†

0

0

Hence,

˜1˜ U1∗ U1 ∥2F + ∥˜ U2∗ U1 ∥2F , V1∗ V1 Σ1−1 − ˜ ∥EA† ∥2F = ∥Σ − 1 ∗ ∗ 2 † † 2 ˜1˜ V1 V1 Σ − ˜ U1 U1 ∥F . ∥BB EA ∥F = ∥Σ 1

(2.8) (2.9)

Using (2.8) and (2.9), we obtain

∥˜ U2∗ U1 ∥2F = ∥EA† ∥2F − ∥BB† EA† ∥2F .

(2.10)

Similarly, we have ∗

†˜

(

U EB U =

˜1−1 V1 Σ U1 − Σ1 V1∗˜ U1∗˜ ∗˜ U2 U1

U ∗ AA† EB†˜ U =

(

)

0 , 0

˜1−1 U1∗˜ U1 − Σ1 V1∗˜ V1 Σ 0

)

0 . 0

Thus,

˜1−1 ∥2F + ∥˜ V1 Σ U1∗ U2 ∥2F , U1 − Σ1 V1∗˜ ∥EB† ∥2F = ∥U1∗˜ ˜ −1 ∥2F . ∥AA† EB† ∥2F = ∥U1∗˜ U1 − Σ1 V1∗˜ V1 Σ 1

(2.11) (2.12)

From (2.11) and (2.12), we have

∥˜ U1∗ U2 ∥2F = ∥EB† ∥2F − ∥AA† EB† ∥2F .

(2.13)

The identity (2.6a) then follows by combining (2.4a), (2.10), and (2.13). In particular, if s = r, using (2.5a), (2.10), and (2.13), we can obtain the identity (2.7a). Replacing A and B in (2.6a) by A∗ and B∗ , respectively, we can get the identity (2.6b). Analogously, the identity (2.7b) can be deduced from (2.7a). This completes the proof. □ On the basis of Lemma 2.3, we can easily get the following corollary. Corollary 2.1.

×n ×n Let A ∈ Cm , B ∈ Cm , and ˜ E = B† − A† . Then r s

∥PB − PA ∥2F = ∥A˜ E ∥2F + ∥B˜ E ∥2F − ∥A˜ EBB† ∥2F − ∥B˜ EAA† ∥2F , 2 2 2 † 2 ∥PB∗ − PA∗ ∥F = ∥˜ EA∥F + ∥˜ EB∥F − ∥B B˜ EA∥F − ∥A† A˜ EB∥2F . In particular, if s = r, then

( ) ( ) ∥PB − PA ∥2F = 2 ∥A˜ E ∥2F − ∥A˜ EBB† ∥2F = 2 ∥B˜ E ∥2F − ∥B˜ EAA† ∥2F , ( ) ( ) ∥PB∗ − PA∗ ∥2F = 2 ∥˜ EA∥2F − ∥B† B˜ EA∥2F = 2 ∥˜ EB∥2F − ∥A† A˜ EB∥2F . Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

5

In the next section, we will apply Lemmas 2.2 and 2.3 to establish the perturbation bounds for an L2 -orthogonal projection. The corresponding results based on Corollary 2.1 can be derived in a similar manner. 3. Main results In this section, we present new upper and lower bounds for ∥PB − PA ∥2F . Some novel combined upper and lower bounds for ∥PB − PA ∥2F and ∥PB∗ − PA∗ ∥2F are also developed. We mention that the upper and lower bounds for ∥PB∗ − PA∗ ∥2F will be omitted, because they can be directly deduced from that for ∥PB − PA ∥2F . We first give an estimate for ∥PB − PA ∥2F , which depends only on the ranks of A and B. ×n ×n Theorem 3.1. Let A ∈ Cm and B ∈ Cm . r s (i) If s + r ≤ m, then

|s − r | ≤ ∥PB − PA ∥2F ≤ s + r .

(3.1)

(ii) If s + r > m, then

|s − r | ≤ ∥PB − PA ∥2F ≤ 2m − s − r .

(3.2)

Proof. Since both PA and PB are Hermitian and idempotent, we have

∥PB − PA ∥2F = tr(PB + PA − PB PA − PA PB ) = s + r − 2 tr(PB PA ), where we have used the fact that the trace of an idempotent matrix equals its rank. If s + r ≤ m, by (2.1), we have 0 ≤ tr(PB PA ) ≤ min{s, r }, which yields

|s − r | ≤ ∥PB − PA ∥2F ≤ s + r . On the other hand, if s + r > m, then s + r − m ≤ tr(PB PA ) ≤ min{s, r }, which leads to

|s − r | ≤ ∥PB − PA ∥2F ≤ 2m − s − r . This completes the proof. □ Remark 3.1. According to the lower bounds in (3.1) and (3.2), we deduce that a necessary condition for lim PB = PA (B B→A

is viewed as a variable) is that rank(B) = rank(A) always holds when B tends to A. Indeed, it is also a sufficient condition for lim PB = PA (see [14,16]). B→A

In what follows, we develop some perturbation bounds involving the matrices E = B − A and ˜ E = B† − A† . 3.1. Upper bounds In this subsection, we present several new upper bounds for ∥PB − PA ∥2F , which improve the existing results. On the basis of (2.6a) and (2.7a), we can derive the following estimates for ∥PB − PA ∥2F , which are sharper than (1.3b) and (1.4b). Theorem 3.2.

×n ×n Let A ∈ Cm , B ∈ Cm , E = B − A, and ˜ E = B† − A† . Define r s

} ∥B† EA† ∥2F ∥B˜ EA∥2F , , ∥B† ∥22 ∥A∥22 { † † 2 } ∥A EB ∥F ∥A˜ EB∥2F α2 := max , . ∥A† ∥22 ∥B∥22

α1 := max

{

Then

∥PB − PA ∥2F ≤ ∥EA† ∥2F + ∥EB† ∥2F − α1 − α2 .

(3.3)

In particular, if s = r, then

{ } ∥PB − PA ∥2F ≤ 2 min ∥EA† ∥2F − α1 , ∥EB† ∥2F − α2 .

(3.4)

Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

6

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

Proof. Using (2.2a), (2.2b), (2.3a), and (2.3b), we obtain

( ˜ ˜1−1˜ V1∗ V1 Σ1−1 − Σ U1∗ U1 ∗ † † ˜ V B EA U = 0 ( −1 ˜1−1 Σ1 U1∗˜ U1 − V1∗˜ V1 Σ V ∗ A† EB†˜ U = 0

)

0 , 0

)

0 . 0

Thus,

˜1−1˜ U1∗ U1 ∥2F , V1∗ V1 Σ1−1 − Σ ∥B† EA† ∥2F = ∥˜ ˜ −1 ∥2F . U1 − V1∗˜ V1 Σ ∥A† EB† ∥2F = ∥Σ −1 U1∗˜

(3.5) (3.6)

1

1

According to (2.9), (2.12), (3.5), and (3.6), we deduce that

∥B† EA† ∥2F , ∥B† ∥22 ∥A† EB† ∥2F ˜1−1 )∥2F ≥ ∥AA† EB† ∥2F = ∥Σ1 (Σ1−1 U1∗˜ . U1 − V1∗˜ V1 Σ ∥A† ∥22

˜1 (˜ ˜1−1˜ ∥BB† EA† ∥2F = ∥Σ V1∗ V1 Σ1−1 − Σ U1∗ U1 )∥2F ≥

(3.7) (3.8)

Similarly, we have

)

( ˜ ˜1˜ V1∗ V1 U1∗ U1 Σ1 − Σ ∗ ˜ ˜ U BEAV = 0 ( ˜1 Σ1 V1∗˜ V1 − U1∗˜ U1 Σ U ∗ A˜ EB˜ V =

0 , 0

)

0 . 0

0

Hence,

˜1˜ ∥B˜ EA∥2F = ∥Σ V1∗ V1 − ˜ U1∗ U1 Σ1 ∥2F , ˜1 − Σ1 V1∗˜ V1 ∥2F . ∥A˜ EB∥2F = ∥U1∗˜ U1 Σ

(3.9) (3.10)

From (2.9), (2.12), (3.9), and (3.10), we deduce that

∥B˜ EA∥2F , ∥A∥22 ∥A˜ EB∥2F ˜1 − Σ1 V1∗˜ ˜1−1 ∥2F ≥ ∥AA† EB† ∥2F = ∥(U1∗˜ U1 Σ V1 )Σ . ∥B∥22

˜1˜ V1∗ V1 − ˜ U1∗ U1 Σ1 )Σ1−1 ∥2F ≥ ∥BB† EA† ∥2F = ∥(Σ

(3.11) (3.12)

Based on (3.7), (3.8), (3.11), and (3.12), we arrive at

} { † † 2 ∥B EA ∥F ∥B˜ EA∥2F , ∥BB† EA† ∥2F ≥ max , ∥B† ∥22 ∥A∥22 { † † 2 } ∥A EB ∥F ∥A˜ EB∥2F ∥AA† EB† ∥2F ≥ max , . ∥A† ∥22 ∥B∥22

(3.13) (3.14)

The inequality (3.3) then follows by combining (2.6a), (3.13), and (3.14). In particular, if s = r, using (2.7a), (3.13), and (3.14), we can obtain the inequality (3.4). □ Based on (2.4a) and (2.5a), we can derive the following theorem. Theorem 3.3.

×n ×n Let A ∈ Cm , B ∈ Cm , E = B − A, and ˜ E = B† − A† . Define r s

{ ( ) ( )} β1 := min ∥A† ∥22 ∥E ∥2F − ∥BB† E ∥2F , ∥A∥22 ∥˜ E ∥2F − ∥˜ EBB† ∥2F , { ( ) ( )} β2 := min ∥B† ∥22 ∥E ∥2F − ∥AA† E ∥2F , ∥B∥22 ∥˜ E ∥2F − ∥˜ EAA† ∥2F . Then

∥PB − PA ∥2F ≤ β1 + β2 .

(3.15)

In particular, if s = r, then

{ } ∥PB − PA ∥2F ≤ 2 min β1 , β2 .

(3.16)

Proof. By (2.2a), (2.2b), (2.3a), and (2.3b), we have U ∗ E˜ V =

(

˜1 − Σ1 V1∗˜ U1∗˜ U1 Σ V1 ˜1 U2∗˜ U1 Σ

−Σ1 V1∗˜ V2 0

)

,

(3.17)

Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

U ∗ AA† E˜ V =

(

˜1 − Σ1 V1∗˜ U1∗˜ U1 Σ V1 0

−Σ1 V1∗˜ V2

)

0

.

7

(3.18)

From (3.17) and (3.18), we deduce that

˜1˜ ∥Σ U1∗ U2 ∥2F = ∥E ∥2F − ∥AA† E ∥2F . Due to

˜1˜ U1∗ U2 ∥2F , ∥˜ U1∗ U2 ∥2F ≤ ∥B† ∥22 ∥Σ it follows that

) ( ∥˜ U1∗ U2 ∥2F ≤ ∥B† ∥22 ∥E ∥2F − ∥AA† E ∥2F . In addition, we have

( −1 ∗ ) ˜1 ˜ ˜1−1˜ Σ U1 U1 − ˜ V1∗ V1 Σ1−1 Σ U1∗ U2 ∗˜ ˜ V EU = , −˜ V ∗ V1 Σ −1 0 ( −1 2 ∗ 1 ∗ ) ˜1 ˜ Σ U1 U1 − ˜ V1 V1 Σ1−1 0 ˜ V ∗˜ EAA† U = . ∗ ˜ −V2 V1 Σ1−1 0

(3.19) (3.20)

By (3.19) and (3.20), we have

˜1−1˜ ∥Σ U1∗ U2 ∥2F = ∥˜ E ∥2F − ∥˜ EAA† ∥2F . Since

˜1−1˜ ∥˜ U1∗ U2 ∥2F ≤ ∥B∥22 ∥Σ U1∗ U2 ∥2F , it follows that

( ) ∥˜ U1∗ U2 ∥2F ≤ ∥B∥22 ∥˜ E ∥2F − ∥˜ EAA† ∥2F . Thus,

{ ( ) ( )} ∥˜ U1∗ U2 ∥2F ≤ min ∥B† ∥22 ∥E ∥2F − ∥AA† E ∥2F , ∥B∥22 ∥˜ E ∥2F − ∥˜ EAA† ∥2F .

(3.21)

Similarly, we have

) ˜1˜ ˜1˜ Σ V1∗ V1 − ˜ U1∗ U1 Σ1 Σ V1∗ V2 , U EV = −˜ U2∗ U1 Σ1 0 ) ( ˜1˜ ˜1˜ V1∗ V2 V1∗ V1 − ˜ U1∗ U1 Σ1 Σ Σ ∗ † ˜ , U BB EV = 0 0 ) ( ∗ ˜1−1 − Σ1−1 U1∗˜ U2 U1 −Σ1−1 U1∗˜ V1 Σ V1 ˜ , E˜ U = V ∗˜ ˜ −1 0 V1 Σ V ∗˜ ) ( ∗ 2 −1 1 ˜1 − Σ1−1 U1∗˜ U1 0 V1 Σ V1 ˜ . V ∗˜ EBB†˜ U = ˜1−1 V2∗˜ V1 Σ 0 (

˜∗

(3.22)

(3.23) (3.24) (3.25)

Using (3.22) and (3.23), we obtain

( ) ∥˜ U2∗ U1 ∥2F ≤ ∥A† ∥22 ∥˜ U2∗ U1 Σ1 ∥2F = ∥A† ∥22 ∥E ∥2F − ∥BB† E ∥2F . In light of (3.24) and (3.25), we have

( ) ∥˜ U2∗ U1 ∥2F ≤ ∥A∥22 ∥˜ U2∗ U1 Σ1−1 ∥2F = ∥A∥22 ∥˜ E ∥2F − ∥˜ EBB† ∥2F . Hence,

{ ( ) ( )} ∥˜ U2∗ U1 ∥2F ≤ min ∥A† ∥22 ∥E ∥2F − ∥BB† E ∥2F , ∥A∥22 ∥˜ E ∥2F − ∥˜ EBB† ∥2F .

(3.26)

In view of (2.4a), (3.21), and (3.26), we conclude that the inequality (3.15) holds. In particular, if s = r, using (2.5a), (3.21), and (3.26), we can get the inequality (3.16). □ Remark 3.2. By (3.18), we have

˜1 − V1∗˜ ∥AA† E ∥2F = ∥Σ1 (Σ1−1 U1∗˜ U1 Σ V1 )∥2F + ∥Σ1 V1∗˜ V2 ∥2F ≥

∥A† E ∥2F , ∥A† ∥22

where we have used the fact that

˜1 − V1∗˜ ∥A† E ∥2F = ∥Σ1−1 U1∗˜ U1 Σ V1 ∥2F + ∥V1∗˜ V2 ∥2F . Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

8

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

Analogously, it holds that

∥BB† E ∥2F ≥

∥B† E ∥2F . ∥B† ∥22

Then

( ) ∥B† E ∥2F β1 ≤ ∥A† ∥22 ∥E ∥2F − , ∥B† ∥22 ( ) ∥A† E ∥2F β2 ≤ ∥B† ∥22 ∥E ∥2F − . ∥A† ∥22 Therefore, the estimates (3.15) and (3.16) are sharper than (1.8) and (1.9), respectively. The following corollary provides an alternative version of Theorem 3.3. Corollary 3.1.

×n ×n Let A ∈ Cm , B ∈ Cm , E = B − A, and ˜ E = B† − A† . Define r s

{ ( ) ( )} γ1 := min ∥A† ∥22 ∥EA† A∥2F − ∥B˜ EA∥2F , ∥A∥22 ∥A† A˜ E ∥2F − ∥A† EB† ∥2F , { ( ) ( )} γ2 := min ∥B† ∥22 ∥EB† B∥2F − ∥A˜ EB∥2F , ∥B∥22 ∥B† B˜ E ∥2F − ∥B† EA† ∥2F . Then

∥PB − PA ∥2F ≤ γ1 + γ2 .

(3.27)

In particular, if s = r, then

{ } ∥PB − PA ∥2F ≤ 2 min γ1 , γ2 .

(3.28)

Proof. By (2.2a), (2.2b), (2.3a), and (2.3b), we have U ∗ EB† B˜ V =

(

˜1 − Σ1 V1∗˜ V1 U1 Σ U1∗˜ ˜1 U2∗˜ U1 Σ

(

)

0 , 0

˜1−1˜ V1∗ V1 Σ1−1 U1∗ U1 − ˜ Σ ˜ V ∗ B† B˜ EU =

(3.29)

˜1−1˜ Σ U1∗ U2

0

0

)

.

(3.30)

According to (3.10) and (3.29), we deduce that

( ) ˜1˜ U1∗ U2 ∥2F = ∥B† ∥22 ∥EB† B∥2F − ∥A˜ EB∥2F . ∥˜ U1∗ U2 ∥2F ≤ ∥B† ∥22 ∥Σ On the other hand, we get from (3.5) and (3.30) that

( ) ˜1−1˜ U1∗ U2 ∥2F = ∥B∥22 ∥B† B˜ E ∥2F − ∥B† EA† ∥2F . ∥˜ U1∗ U2 ∥2F ≤ ∥B∥22 ∥Σ Hence,

{ ( ) ( )} ∥˜ U1∗ U2 ∥2F ≤ min ∥B† ∥22 ∥EB† B∥2F − ∥A˜ EB∥2F , ∥B∥22 ∥B† B˜ E ∥2F − ∥B† EA† ∥2F .

(3.31)

Similarly, we have

( ) ˜1˜ Σ V1∗ V1 − ˜ U1∗ U1 Σ1 0 ∗ † ˜ U EA AV = , −˜ U2∗ U1 Σ1 0 ( ) ˜1−1 − Σ1−1 U1∗˜ V1∗˜ V1 Σ U1 −Σ1−1 U1∗˜ U2 V ∗ A† A˜ E˜ U = . 0

0

(3.32)

(3.33)

Using (3.9) and (3.32), we obtain

( ) ∥˜ U2∗ U1 ∥2F ≤ ∥A† ∥22 ∥˜ U2∗ U1 Σ1 ∥2F = ∥A† ∥22 ∥EA† A∥2F − ∥B˜ EA∥2F . In view of (3.6) and (3.33), we have

( ) ∥˜ U2∗ U1 ∥2F ≤ ∥A∥22 ∥˜ U2∗ U1 Σ1−1 ∥2F = ∥A∥22 ∥A† A˜ E ∥2F − ∥A† EB† ∥2F . Thus,

{ ( ) ( )} ∥˜ U2∗ U1 ∥2F ≤ min ∥A† ∥22 ∥EA† A∥2F − ∥B˜ EA∥2F , ∥A∥22 ∥A† A˜ E ∥2F − ∥A† EB† ∥2F .

(3.34)

The rest of the proof is similar to Theorem 3.3. □ Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

9

3.2. Lower bounds As is well known, the L2 -orthogonal projection onto the column space of a matrix is not necessarily a continuous function of the entries of the matrix (see, e.g., [14,16]). In this subsection, we attempt to establish some lower bounds for ∥PB − PA ∥2F . The first theorem is based on the identities (2.6a) and (2.7a). Theorem 3.4.

×n ×n Let A ∈ Cm , B ∈ Cm , E = B − A, and ˜ E = B† − A† . Define r s

} { EA∥2F , α1′ := min ∥B∥22 ∥B† EA† ∥2F , ∥A† ∥22 ∥B˜ { } α2′ := min ∥A∥22 ∥A† EB† ∥2F , ∥B† ∥22 ∥A˜ EB∥2F . Then

∥PB − PA ∥2F ≥ ∥EA† ∥2F + ∥EB† ∥2F − α1′ − α2′ .

(3.35)

In particular, if s = r, then

{ } ∥PB − PA ∥2F ≥ 2 max ∥EA† ∥2F − α1′ , ∥EB† ∥2F − α2′ .

(3.36)

Proof. According to the proof of Theorem 3.2, we have

˜1 (˜ ˜1−1˜ U1∗ U1 )∥2F ≤ ∥B∥22 ∥B† EA† ∥2F , ∥BB† EA† ∥2F = ∥Σ V1∗ V1 Σ1−1 − Σ ∗ ∗ † † 2 ˜1˜ V1 V1 − ˜ U1 U1 Σ1 )Σ −1 ∥2F ≤ ∥A† ∥22 ∥B˜ EA∥2F . ∥BB EA ∥F = ∥(Σ 1

Hence,

{ } ∥BB† EA† ∥2F ≤ min ∥B∥22 ∥B† EA† ∥2F , ∥A† ∥22 ∥B˜ EA∥2F . Similarly, it is easy to check that

{ } ∥AA† EB† ∥2F ≤ min ∥A∥22 ∥A† EB† ∥2F , ∥B† ∥22 ∥A˜ EB∥2F . The desired result then follows from the identities (2.6a) and (2.7a). □ U2∗ U1 ∥2F directly. The following theorem is derived by bounding ∥˜ U1∗ U2 ∥2F and ∥˜ Theorem 3.5.

×n ×n Let A ∈ Cm , B ∈ Cm , E = B − A, and ˜ E = B† − A† . Define r s

} EAA† ∥2F E ∥2F − ∥˜ ∥E ∥2F − ∥AA† E ∥2F ∥˜ , , ∥B∥22 ∥B† ∥22 { } ∥E ∥2F − ∥BB† E ∥2F ∥˜ E ∥2F − ∥˜ EBB† ∥2F β2′ := max . , ∥A∥22 ∥A† ∥22 β1′ := max

{

Then

∥PB − PA ∥2F ≥ β1′ + β2′ .

(3.37)

In particular, if s = r, then

{ } ∥PB − PA ∥2F ≥ 2 max β1′ , β2′ .

(3.38)

Proof. Based on the proof of Theorem 3.3, we have

˜1˜ ∥Σ U1∗ U2 ∥2F ∥E ∥2F − ∥AA† E ∥2F = , 2 ∥B∥2 ∥B∥22 ˜1−1˜ ∥Σ U1∗ U2 ∥2F ∥˜ E ∥2F − ∥˜ EAA† ∥2F ∥˜ U1∗ U2 ∥2F ≥ = . 2 † † ∥B ∥2 ∥B ∥22 ∥˜ U1∗ U2 ∥2F ≥

Thus,

∥˜ U1∗ U2 ∥2F ≥ max

{

} ∥E ∥2F − ∥AA† E ∥2F ∥˜ E ∥2F − ∥˜ EAA† ∥2F , . ∥B∥22 ∥B† ∥22

Analogously, we have

∥˜ U2∗ U1 ∥2F ≥

∥˜ U2∗ U1 Σ1 ∥2F ∥E ∥2F − ∥BB† E ∥2F = , 2 ∥A∥2 ∥A∥22

Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

10

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

∥˜ U2∗ U1 Σ1−1 ∥2F ∥˜ E ∥2F − ∥˜ EBB† ∥2F = . ∥A† ∥22 ∥A† ∥22

∥˜ U2∗ U1 ∥2F ≥ Hence,

{ } ∥E ∥2F − ∥BB† E ∥2F ∥˜ E ∥2F − ∥˜ EBB† ∥2F 2 ∗ ˜ ∥U2 U1 ∥F ≥ max , . ∥A∥22 ∥A† ∥22 Using (2.4a) and (2.5a), we can obtain the estimates (3.37) and (3.38).

□

Using the similar argument as in Corollary 3.1, we can get the following corollary, which is an alternative version of Theorem 3.5. ×n ×n Corollary 3.2. Let A ∈ Cm , B ∈ Cm , E = B − A, and ˜ E = B† − A† . Define r s

} ∥EB† B∥2F − ∥A˜ EB∥2F ∥B† B˜ E ∥2F − ∥B† EA† ∥2F , , ∥B∥22 ∥B† ∥22 } { ∥EA† A∥2F − ∥B˜ EA∥2F ∥A† A˜ E ∥2F − ∥A† EB† ∥2F . , γ2′ := max ∥A∥22 ∥A† ∥22 γ1′ := max

{

Then

∥PB − PA ∥2F ≥ γ1′ + γ2′ .

(3.39)

In particular, if s = r, then

{ } ∥PB − PA ∥2F ≥ 2 max γ1′ , γ2′ .

(3.40)

3.3. Combined upper bounds In this subsection, we present new combined upper bounds for ∥PB − PA ∥2F and ∥PB∗ − PA∗ ∥2F , which are established in a parameterized manner. In order to show the combined upper bounds concisely, we first define IM (t) := Theorem 3.6.

t

∥M † ∥22

+

1−t

∀ M ∈ Cm×n \{0}, t ∈ [0, 1].

∥M ∥22

×n ×n Let A ∈ Cm , B ∈ Cm , E = B − A, and ˜ E = B† − A† . Define r s

( ) ( ) Φ (λ) := λ ∥E ∥2F − ∥A˜ EB∥2F + (1 − λ) ∥˜ E ∥2F − ∥B† EA† ∥2F , ( ) ( ) Ψ (µ) := µ ∥E ∥2F − ∥B˜ EA∥2F + (1 − µ) ∥˜ E ∥2F − ∥A† EB† ∥2F , where λ ∈ [0, 1] and µ ∈ [0, 1] are parameters. Then

} Φ (λ) Ψ (µ) ∥PB∗ − PA∗ ∥2F ≤ + , IB (λ) IA (µ) IB (λ) IA (µ) Φ (λ) + Ψ (µ) { }. ∥PB − PA ∥2F + ∥PB∗ − PA∗ ∥2F ≤ min IA (λ), IB (λ), IA (µ), IB (µ)

∥PB − PA ∥2F + min

{

IA (λ) IB (µ)

,

(3.41) (3.42)

In particular, if s = r, then IB (λ)∥PB − PA ∥2F + IA (λ)∥PB∗ − PA∗ ∥2F ≤ 2Φ (λ),

(3.43)

IA (µ)∥PB −

(3.44)

∥ + IB (µ)∥PB∗ −

PA 2F

∥ ≤ 2Ψ (µ).

PA∗ 2F

Proof. Using (3.17) and (3.10), we obtain

˜1 − Σ1 V1∗˜ ˜1 ∥2F ∥E ∥2F = ∥U1∗˜ U1 Σ V1 ∥2F + ∥Σ1 V1∗˜ V2 ∥2F + ∥U2∗˜ U1 Σ ∗˜ 2 ∗˜ 2 ∥V1 V2 ∥F ∥U U1 ∥ ≥ ∥A˜ EB∥2F + + 2† 2 F , 2 † ∥A ∥2 ∥B ∥2 which gives

∥˜ U1∗ U2 ∥2F ∥˜ V ∗ V1 ∥ 2 + 2 † 2 F ≤ ∥E ∥2F − ∥A˜ EB∥2F . 2 † ∥B ∥2 ∥A ∥2

(3.45)

By (3.22) and (3.9), we have

˜1˜ ˜1˜ ∥E ∥2F = ∥Σ V1∗ V1 − ˜ U1∗ U1 Σ1 ∥2F + ∥Σ V1∗ V2 ∥2F + ∥˜ U2∗ U1 Σ1 ∥2F Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

≥ ∥B˜ EA∥2F +

11

∥˜ U2∗ U1 ∥2F ∥˜ V1∗ V2 ∥2F + , ∥B† ∥22 ∥A† ∥22

which yields

∥˜ U2∗ U1 ∥2F ∥˜ V1∗ V2 ∥2F EA∥2F . + ≤ ∥E ∥2F − ∥B˜ ∥A† ∥22 ∥B† ∥22

(3.46)

Similarly, we can derive from (3.19), (3.24), (3.5), and (3.6) that

∥˜ U1∗ U2 ∥2F ∥˜ V2∗ V1 ∥2F + ≤ ∥˜ E ∥2F − ∥B† EA† ∥2F , ∥B∥22 ∥A∥22 ∥˜ U2∗ U1 ∥2F ∥˜ V1∗ V2 ∥2F + ≤ ∥˜ E ∥2F − ∥A† EB† ∥2F . ∥A∥22 ∥B∥22

(3.47) (3.48)

From (3.45) and (3.47), we deduce that IB (λ)∥˜ U1∗ U2 ∥2F + IA (λ)∥˜ V2∗ V1 ∥2F ≤ Φ (λ).

(3.49)

In light of (3.46) and (3.48), we have IA (µ)∥˜ U2∗ U1 ∥2F + IB (µ)∥˜ V1∗ V2 ∥2F ≤ Ψ (µ).

(3.50)

Combining (2.4a), (2.4b), (3.49), and (3.50), we can arrive at the estimates (3.41) and (3.42). In particular, if s = r, using (2.5a), (2.5b), (3.49), and (3.50), we can obtain the estimates (3.43) and (3.44). □ Under the assumptions of Theorem 3.6, taking λ = µ = 1, we can get the following corollary. ×n ×n Corollary 3.3. Let A ∈ Cm , B ∈ Cm , E = B − A, and ˜ E = B† − A† . Then r s

} ( ) ∥A† ∥22 ∥B† ∥22 ∥PB − ∥ + min EB∥2F − ∥A† ∥22 ∥B˜ EA∥2F , ∥PB∗ − PA∗ ∥2F ≤ ∥A† ∥22 + ∥B† ∥22 ∥E ∥2F − ∥B† ∥22 ∥A˜ , ∥B† ∥22 ∥A† ∥22 { }( ) ∥PB − PA ∥2F + ∥PB∗ − PA∗ ∥2F ≤ max ∥A† ∥22 , ∥B† ∥22 2∥E ∥2F − ∥A˜ EB∥2F − ∥B˜ EA∥2F . PA 2F

{

(3.51) (3.52)

In particular, if s = r, then

} { ( ) ( )} ∥A† ∥22 ∥B† ∥22 ∥PB − ∥ + min EA∥2F , ∥B† ∥22 ∥E ∥2F − ∥A˜ EB∥2F , , † 2 ∥PB∗ − PA∗ ∥2F ≤ 2 min ∥A† ∥22 ∥E ∥2F − ∥B˜ 2 † ∥B ∥2 ∥A ∥2 ) 2∥A† ∥2 ∥B† ∥2 ( ∥PB − PA ∥2F + ∥PB∗ − PA∗ ∥2F ≤ † 2 2 † 22 2∥E ∥2F − ∥A˜ EB∥2F − ∥B˜ EA∥2F . ∥A ∥2 + ∥B ∥2 PA 2F

{

(3.53) (3.54)

Remark 3.3. Evidently, the estimates (3.51), (3.53), and (3.54) are sharper than (1.5), (1.6), and (1.7), respectively. In addition, since

∥AA† EB† ∥2F ∥A† EB† ∥2F ≥ , ∥B† ∥22 ∥A† ∥22 ∥B† ∥22 ∥BB† EA† ∥2F ∥B† EA† ∥2F ˜1˜ ≥ , ∥B˜ EA∥2F = ∥(Σ V1∗ V1 Σ1−1 − ˜ U1∗ U1 )Σ1 ∥2F ≥ ∥A† ∥22 ∥A† ∥22 ∥B† ∥22

˜1−1 )Σ ˜1 ∥2F ≥ ∥A˜ EB∥2F = ∥(U1∗˜ U1 − Σ1 V1∗˜ V1 Σ

we conclude that (3.52) and (3.54) are sharper than (1.10) and (1.11), respectively. 3.4. Combined lower bounds In this subsection, we develop some combined lower bounds for ∥PB − PA ∥2F and ∥PB∗ − PA∗ ∥2F . For simplicity, we define JM (t) := t ∥M ∥22 + (1 − t)∥M † ∥22

∀ M ∈ Cm×n \{0}, t ∈ [0, 1].

×n ×n Theorem 3.7. Let A ∈ Cm , B ∈ Cm , E = B − A, and ˜ E = B† − A† . Let Φ (ξ ) and Ψ (η) be defined as in Theorem 3.6, where r s ξ ∈ [0, 1] and η ∈ [0, 1] are parameters. Then

} Φ (ξ ) Ψ (η ) ∥PB − ∥ + max , ∥PB∗ − PA∗ ∥2F ≥ + , J B (ξ ) J A (η ) J B (ξ ) JA (η) Φ (ξ ) + Ψ (η) { }. ∥PB − PA ∥2F + ∥PB∗ − PA∗ ∥2F ≥ max JA (ξ ), JB (ξ ), JA (η), JB (η) PA 2F

{

JA (ξ ) JB (η)

(3.55) (3.56)

Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

12

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

In particular, if s = r, then JB (ξ )∥PB − PA ∥2F + JA (ξ )∥PB∗ − PA∗ ∥2F ≥ 2Φ (ξ ),

(3.57)

JA (η)∥PB − PA ∥2F + JB (η)∥PB∗ − PA∗ ∥2F ≥ 2Ψ (η).

(3.58)

Proof. According to the proof of Theorem 3.6, we deduce that EB∥2F , V2∗ V1 ∥2F ≥ ∥E ∥2F − ∥A˜ U1∗ U2 ∥2F + ∥A∥22 ∥˜ ∥B∥22 ∥˜ ∥A∥22 ∥˜ U2∗ U1 ∥2F + ∥B∥22 ∥˜ V1∗ V2 ∥2F ≥ ∥E ∥2F − ∥B˜ EA∥2F , E ∥2F − ∥B† EA† ∥2F , V2∗ V1 ∥2F ≥ ∥˜ U1∗ U2 ∥2F + ∥A† ∥22 ∥˜ ∥B† ∥22 ∥˜

(3.59)

≥ ∥˜ E ∥2F − ∥A† EB† ∥2F .

(3.62)

U2∗ U1 ∥2F ∥A† ∥22 ∥˜

+

V1∗ V2 ∥2F ∥B† ∥22 ∥˜

(3.60) (3.61)

Using (3.59), (3.60), (3.61), and (3.62), we can obtain V2∗ V1 ∥2F ≥ Φ (ξ ), JB (ξ )∥˜ U1∗ U2 ∥2F + JA (ξ )∥˜ JA (η)∥˜ U2∗ U1 ∥2F + JB (η)∥˜ V1∗ V2 ∥2F ≥ Ψ (η). The rest of the proof is similar to Theorem 3.6. □ Taking ξ = η = 0, we can obtain the following corollary. ×n ×n Corollary 3.4. Let A ∈ Cm , B ∈ Cm , E = B − A, and ˜ E = B† − A† . Then r s

} ( ) ∥A† ∥22 ∥B† ∥22 ∥A† EB† ∥2F ∥B† EA† ∥2F 1 1 2 ˜ ∗ ∗ ∥ P − P ∥ ≥ ∥ E ∥2F − , + − , B A F 2 2 2 2 2 ∥B† ∥2 ∥A† ∥2 ∥A† ∥2 ∥B† ∥2 ∥A† ∥2 ∥B† ∥22 2∥˜ E ∥2F − ∥A† EB† ∥2F − ∥B† EA† ∥2F { } . ∥PB − PA ∥2F + ∥PB∗ − PA∗ ∥2F ≥ max ∥A† ∥22 , ∥B† ∥22 ∥PB − PA ∥2F + max

{

(3.63) (3.64)

In particular, if s = r, then

} { } ∥A† ∥22 ∥B† ∥22 ∥˜ E ∥2F − ∥B† EA† ∥2F ∥˜ E ∥2F − ∥A† EB† ∥2F 2 ∗ − PA∗ ∥ ≥ 2 max , ∥ P , , B F ∥B† ∥22 ∥A† ∥22 ∥B† ∥22 ∥A† ∥22 ( ) 2 2∥˜ E ∥2F − ∥A† EB† ∥2F − ∥B† EA† ∥2F . ∥PB − PA ∥2F + ∥PB∗ − PA∗ ∥2F ≥ † 2 ∥A ∥2 + ∥B† ∥22 ∥PB − PA ∥2F + max

{

(3.65) (3.66)

Remark 3.4. The parameters λ, µ, ξ , and η in Theorems 3.6 and 3.7 can be chosen flexibly. Different parameters will yield different types of combined estimates. Thus, one can optimize the combined bounds in Theorems 3.6 and 3.7 by selecting some sophisticated parameters. 4. Numerical experiments In Section 3, we have developed new perturbation bounds for the L2 -orthogonal projection onto the column space of a matrix, and compared the new results with the existing ones theoretically. In this section, we give two examples to illustrate the differences between the new bounds and the existing ones. The first one is in fact the example in (1.12). Example 4.1. Let

( A=

1 0

0 0

)

( and B =

ε 1+ε

0

0

10

)

ε

,

where 0 < ε < 1. In this example, we have

( E=

1 − 1+ε

0

0 ε

) ,

A† =

10

(

1 0

)

0 , 0

B† =

(

1+ 0

1

ε

0 10

ε

)

, ˜ E=

(1

0

0

10

ε

ε

)

.

It is easy to see that

∥PB − PA ∥2F ≡ 1 ∀ 0 < ε < 1. (I) Upper and lower bounds Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

13

Table 1 The upper bounds in (1.3b), (1.8), (3.2), (3.3), and (3.15). Upper bound for ∥PB − PA ∥2F

Estimate (1.3b)

1

1+

(1.8)

1 + (1+ε )2 1 + (1+ε)2

ε2

99 100

(3.2) (3.3) (3.15)

1 1 1

Fig. 1. Numerical comparison of the upper bounds listed in Table 1. Table 2 The combined upper bounds in (1.5) and (3.51). Combined upper bound for C1

Estimate (1.5)

1+

(3.51)

ε2 100

+

1 (1+ε )2

1+

ε2

+

100

ε2 (1+ε)2

100

Under the setting of Example 4.1, the upper bounds in (1.3b), (1.8), (3.2), (3.3), and (3.15) are listed in Table 1. And the numerical behaviors (ε is confined in (0.1, 1)) of these bounds are shown in Fig. 1. From Table 1, we see that the upper bounds in (3.2), (3.3), and (3.15) have attained the exact value 1. Fig. 1 shows that the upper bound in (1.3b) will deviate from the exact value seriously when ε is small. In addition, direct computations yield that the lower bounds in (3.2), (3.35), and (3.37) are all the exact value 1. (II) Combined upper and lower bounds For simplicity, we define C1 := ∥PB − PA ∥2F + min

{

} ∥A† ∥22 ∥B† ∥22 , ∥PB∗ − PA∗ ∥2F , ∥B† ∥22 ∥A† ∥22

C2 := ∥PB − PA ∥2F + ∥PB∗ − PA∗ ∥2F , C3 := ∥PB − PA ∥2F + max

{

} ∥A† ∥22 ∥B† ∥22 , ∥PB∗ − PA∗ ∥2F . ∥B† ∥22 ∥A† ∥22

Under the setting of Example 4.1, we have C1 = 1 +

ε2 100

,

C2 = 2,

and C3 = 1 +

100

ε2

.

The combined upper bounds for C1 in (1.5) and (3.51) are given in Table 2, and the combined upper bounds for C2 in (1.10) and (3.52) are listed in Table 3. The numerical behaviors (ε is confined in (0.1, 1)) of these bounds are shown in Fig. 2. ε2 From Tables 2 and 3, we see that the combined upper bounds in (3.51) and (3.52) have attained the exact values 1 + 100 and 2, respectively. Fig. 2 displays that the existing bounds in (1.5) and (1.10) will deviate from the corresponding exact values seriously when ε is small. Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

14

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx Table 3 The combined upper bounds in (1.10) and (3.52). Combined upper bound for C2

Estimate (1.10)

2−

2

ε2

(3.52)

+

200

ε 2 (1+ε )2

2

Fig. 2. Numerical comparison of the combined upper bounds in Table 2 (left); numerical comparison of the combined upper bounds in Table 3 (right). Table 4 The upper bounds in (1.3b), (1.8), (3.2), (3.3), and (3.15). Upper bound for ∥PB − PA ∥2F

Estimate

2(1 + ε + ε 2 ) +

(1.3b)

2ε 2 (1 + ε ) +

(1.8)

ε 1+ε

(3.2) (3.3) 1 2

(3.15)

ε2

(1+ε )2

√ + (1 + ε + ε2 ) 4ε4 + 1 1√

+ ε2 (1 + ε)2 +

1 4

1 (1+ε )4

+ ε4 (1 + ε)4

Furthermore, straightforward calculations yield that the lower bound for C3 in (3.63) is 1 + 100 and the lower bound ε2 for C2 in (3.64) is 2. Thus, the combined lower bounds in (3.63) and (3.64) have attained the corresponding exact values. The next example provides a complex matrix case. Example 4.2. Let

( A=

)

i 0

0 0

( and B =

i 1+ε

0

) ε , ε

√ where i = −1 and 0 < ε < 12 . In this example, we have

( E=

εi − 1+ε

0

) ε , ε

A† =

( −i 0

)

0 , 0

B† =

( −(1 + ε )i 0

(1 + ε )i 1

ε

)

( −ε i , ˜ E= 0

(1 + ε )i 1

ε

)

.

Obviously, it holds that

∥PB − PA ∥2F ≡ 1 ∀ 0 < ε <

1 2

.

(I) Upper and lower bounds Under the setting of Example 4.2, the upper bounds in (1.3b), (1.8), (3.2), (3.3), and (3.15) are given in Table 4, and the lower bounds in (3.2), (3.35), and (3.37) are listed in Table 5. Numerical behaviors of these bounds are shown in Fig. 3. From Table 4, we see that the upper bounds in (3.2) and (3.3) have attained the exact value 1. And Fig. 3 (left) shows that the estimate (3.15) is sharper than both (1.3b) and (1.8). Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx

15

Table 5 The lower bounds in (3.2), (3.35), and (3.37). Lower bound for ∥PB − PA ∥2F

Estimate (3.2) (3.35)

1 1 2+2ε 2 (1+ε )2

(3.37)

1+2ε 2 (1+ε )2 +

√

1+4ε 4 (1+ε )4

Fig. 3. Numerical comparison of the upper bounds in Table 4 (left); numerical comparison of the lower bounds in Table 5 (right). Table 6 The combined upper bounds in (1.5) and (3.51). Combined upper bound for C1

Estimate 1+2(1+ε )2 2(1+ε )2

(1.5)

1 2

(3.51)

(

1 + 2ε 2 + 2ε 2 (1 + ε )2 +

+ 2ε2 + ε2 (1 + ε)2 +

1 2

√

√

1 + 4ε 4 (1 + ε )4

)

1 + 4ε 4 (1 + ε )4

From Table 5, we see that the lower bounds in (3.2) and (3.35) have attained the exact value 1. Moreover, Fig. 3 (right) displays that the lower bound in (3.37) is very close to the exact value (especially when ε is small). (II) Combined upper and lower bounds Under the setting of Example 4.2, we have C1 = 1 +

2ε 2 1 + 2ε 2 (1 + ε )2 +

√

1 + 4ε 4 (1 + ε )4

,

C2 = 2, C3 = 1 + (1 + ε ) + 2

1 2ε 2

√ +

(1 + ε )4 +

1 4ε 4

.

The combined upper bounds for C1 in (1.5) and (3.51) are listed in Table 6, and the combined upper bounds for C2 in (1.10) and (3.52) are given in Table 7. In addition, the lower bound for C3 in (3.63) is 1

ε2

+

2 + 2ε 2 (1 + ε )2 1 + 2ε 2 (1 + ε )2 +

√

1 + 4ε 4 (1 + ε )4

,

and the lower bound for C2 in (3.64) is 4 + 2ε 2 (1 + ε )2 1 + 2ε 2 (1 + ε )2 +

√

1 + 4ε 4 (1 + ε )4

.

Numerical behaviors of these bounds are shown in Figs. 4 and 5. Fig. 4 displays that the new combined upper bounds for C1 and C2 are smaller than the existing ones. Moreover, Fig. 5 shows that the combined lower bounds in (3.63) and (3.64) are very close to the corresponding exact values (especially when ε is small). Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

16

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx Table 7 The combined upper bounds in (1.10) and (3.52). Combined upper bound for C2

Estimate (1.10) (3.52)

1+2(1+ε )2 (1+ε )2

1 + 2ε 2 (1 + ε )2 +

(

3 2

√

1 + 4ε 4 (1 + ε )4 − 2ε 2 − (1 + ε )2

+ 3ε2 (1 + ε)2 +

)

√ 3 2

1 + 4ε 4 (1 + ε )4

Fig. 4. Numerical comparison of the combined upper bounds for C1 (left); numerical comparison of the combined upper bounds for C2 (right).

Fig. 5. Numerical behavior of the combined lower bound for C2 (left); numerical behavior (ε is confined in (0.05, 0.5)) of the combined lower bound for C3 (right).

Acknowledgments The author would like to thank the anonymous referees for their valuable comments and suggestions, which greatly improved the original version of this paper. This research was carried out by the author during his Ph.D. study at the Academy of Mathematics and Systems Science, Chinese Academy of Sciences. The author is grateful to Professor Chen-Song Zhang for his kind support. References [1] S. Nelson, M. Neumann, Generalizations of the projection method with applications to SOR theory for Hermitian positive semidefinite linear systems, Numer. Math. 51 (1987) 123–141. [2] R.D. Fierro, J.R. Bunch, Orthogonal projection and total least squares, Numer. Linear Algebra Appl. 2 (1995) 135–153. [3] R.D. Fierro, J.R. Bunch, Perturbation theory for orthogonal projection methods with applications to least squares and total least squares, Linear Algebra Appl. 234 (1996) 71–96.

Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.

X. Xu / Journal of Computational and Applied Mathematics xxx (xxxx) xxx [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]

17

Z. Jia, Composite orthogonal projection methods for large matrix eigenproblems, Sci. China Ser. A 42 (1999) 577–585. S. Morigi, L. Reichel, F. Sgallari, A truncated projected SVD method for linear discrete ill-posed problems, Numer. Algorithms 43 (2006) 197–213. S. Morigi, L. Reichel, F. Sgallari, Orthogonal projection regularization operators, Numer. Algorithms 44 (2007) 99–114. J.F. Grcar, Spectral condition numbers of orthogonal projections and full rank linear least squares residuals, SIAM J. Matrix Anal. Appl. 31 (2010) 2934–2949. M.E. Hochstenbach, L. Reichel, Subspace-restricted singular value decompositions for linear discrete ill-posed problems, J. Comput. Appl. Math. 235 (2010) 1053–1064. E.S. Coakley, V. Rokhlin, M. Tygert, A fast randomized algorithm for orthogonal projection, SIAM J. Sci. Comput. 33 (2011) 849–868. P. Drineas, M.W. Mahoney, S. Muthukrishnan, T. Sarlós, Faster least squares approximation, Numer. Math. 117 (2011) 219–249. G.H. Golub, C.F. Van Loan, Matrix Computations, fourth ed., Johns Hopkins University Press, Baltimore, 2013. M.M. Betcke, H. Voss, Restarting iterative projection methods for Hermitian nonlinear eigenvalue problems with minmax property, Numer. Math. 135 (2017) 397–430. G.W. Stewart, On the perturbation of pseudo-inverses, projections and linear least squares problems, SIAM Rev. 19 (1977) 634–662. J.-G. Sun, The stability of orthogonal projections, J. Grad. Sch. 1 (1984) 123–133, (in Chinese). G.W. Stewart, J.-G. Sun, Matrix Perturbation Theory, Academic Press, Boston, 1990. J.-G. Sun, Matrix Perturbation Analysis, second ed., Science Press, Beijing, 2001, (in Chinese). B. Li, W. Li, L. Cui, New bounds for perturbation of the orthogonal projection, Calcolo 50 (2013) 69–78. Y. Chen, X. Chen, W. Li, On perturbation bounds for orthogonal projections, Numer. Algorithms 73 (2016) 433–444. W. Li, Y. Chen, S. Vong, Q. Luo, Some refined bounds for the perturbation of the orthogonal projection and the generalized inverse, Numer. Algorithms 79 (2018) 657–677. R.A. Horn, C.R. Johnson, Matrix Analysis, second ed., Cambridge University Press, Cambridge, 2013.

Please cite this article as: X. Xu, On the perturbation of an L2 -orthogonal projection, Journal of Computational and Applied Mathematics (2019), https://doi.org/10.1016/j.cam.2019.06.043.