- Email: [email protected]
The best approximation of matrices under inequality constraints
Applied Mathematics and Computation 137 (2003) 487–497 www.elsevier.com/locate/amc
The best approximation of matrices under inequality constraints Xi-Yan Hu a
a,*
, Lei Zhang b, Qin Zhang
a
Department of Applied Mathematics, Hunan University, Changsha 410082, China b Hunan Computing Center, Changsha 410012, China
Abstract In this paper, a class of best approximation problems of matrices under inequality constraints are firstly presented and studied. For real matrices, symmetric matrices and antisymmetric matrices respectively, we have given the conditions for the existence of solutions and expressions of general solutions. Furthermore, the best approximation solutions and algorithms of them are provided. Ó 2002 Elsevier Science Inc. All rights reserved. Keywords: Best approximation; Inequality constraints; Inverse problem; Linear complementarity problem
1. Introduction Although a series of perfect results have been given for the best approximation problems of matrices under spectral constraints [1], the problems under inequality constraints have not been considered. In this paper, we will discuss these problems. Let Rn , Rn0 denote the sets of n-dimensional real vectors, nonnegative vectors, and Rnm , SRnn , ASRnn , ORnn denote the sets of real n m matrices, real n n symmetric matrices, real n n antisymmetric matrices and orthogonal n n matrices respectively. k k stands for the Frobenius norm of a matrix. Aþ represents the Moore-Penrose generalized inverse matrix of A and I is an
*
Corresponding author. E-mail address: [email protected] (X.-Y. Hu).
0096-3003/02/$ - see front matter Ó 2002 Elsevier Science Inc. All rights reserved. PII: S 0 0 9 6 - 3 0 0 3 ( 0 2 ) 0 0 1 5 3 - 4
488
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
identity matrix. Define inner product ðA; BÞ ¼ trðBT AÞ in Rnm and the folT lowing symbols are introduced for vector f ¼ ðf1 ; f2 ; . . . ; fn Þ : T ½f þ ¼ ðf1 ; f2 ; . . . ; fn Þ ; T
½f ¼ ðf~1 ; f~2 ; . . . ; f~n Þ ;
fi ¼ maxð0; fi Þ;
i ¼ 1; 2; . . . ; n;
f~i ¼ minð0; fi Þ;
i ¼ 1; 2; . . . ; n;
then f ¼ ½f þ þ ½f . Now we can present the best approximation problems of matrices under inequality constraints as follows: Problem I. Given 0 6¼ x, b 2 Rn and S Rnn , find A 2 S such that Ax P b. Problem II. Given A~ 2 Rnn , find A^ 2 SA such that k A^ A~k ¼ min kA A~k; A2SA
where SA is the set of solutions of Problem I. In this paper, we study three cases when S ¼ Rnn , SRnn , ASRnn respectively. The conditions for existence and expressions of general solutions of Problem I are obtained, and the expression of solution of Problem II and the algorithm to get it are also given. For convenience of discussion in the later sections, here we give the singularvalue decomposition (SVD) of a nonzero vector x 2 Rn as follows: r x¼U ¼ ru1 ; ð1:1Þ 0 pffiffiffiffiffiffiffi where U ¼ ðu1 ; U2 Þ 2 ORnn , r ¼ xT x > 0, u1 2 Rn1 . It is obvious to see that: xþ ¼ r 1 uT1 ;
I xxþ ¼ I u1 uT1 ¼ U2 U2T :
ð1:2Þ
2. Case S ¼ Rnn Lemma 2.1. Given 0 6¼ x; b 2 Rn , the general solution of equation Ax ¼ b is A ¼ bx þ Y ðI xxþ Þ, 8 Y 2 Rnn [2]. With the help of nonnegative parameter vector c, problem Ax P b is equivalent to equation Ax ¼ b þ c, c 2 Rn0 . Furthermore, by Lemma 2.1 we can draw the following conclusion. Theorem 2.1. Given S ¼ Rnn , there must exist solutions to Problem I and solution set SA can be expressed as SA ¼ fA ¼ ðb þ cÞxþ þ Y ðI xxþ Þj 8 c 2 Rn0 ; 8 Y 2 Rnn g:
ð2:1Þ
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
489
Theorem 2.2. If S ¼ Rnn , then there exists a unique best approximation solution to Problem II and it can be expressed as A^ ¼ ðb þ c~Þxþ þ A~ðI xxþ Þ;
c~ ¼ ½A~x b þ :
ð2:2Þ
Proof. From (2.1) we know that solution set SA of Problem I is a closed convex set, so that there exists a unique solution to Problem II. By (1.1), (1.2) and (2.1), we get A~ A ¼ A~ ðb þ cÞxþ Y ðI xxþ Þ ¼ ½A~ r 1 ðb þ cÞuT YU2 U T UU T 1
2
¼ ½ðA~u1 ; A~U2 Þ r 1 ðb þ c; 0Þ ð0; YU2 Þ U T : By invariance property of orthogonal matrix norm, it follows that kA~ Ak2 ¼ kA~u1 r 1 ðb þ cÞk2 þ kA~U2 YU2 k2 ; therefore kA~ Ak ¼ minA2SA is equivalent to kA~u1 r 1 ðb þ cÞk ¼ minn ;
ð2:3Þ
kA~U2 YU2 k ¼ min : nn
ð2:4Þ
c2R0
Y 2R
Noticing (1.1), when c~ ¼ ½A~x b þ , (2.3) reaches its minimum. It is also obvious that when Y ¼ A~ þ guT1 , 8 g 2 Rn , (2.4) reaches its minimum. Substitute (2.1) with above results to gain (2.2) and prove the theorem. 3. Case S ¼ SRnn Lemma 3.1. Given 0 6¼ x; b 2 Rn , there must exist solutions to equation Ax ¼ b in SRnn and solution set S0 can be expressed as [1] S0 ¼ fA ¼ bxþ þ ðbxþ ÞT ðxþ ÞT bT xxþ þ ðI xxþ ÞY ðI xxþ Þj 8 Y 2 SRnn g:
ð3:1Þ
Similar to Section 2, we can get the following result by introducing nonnegative parameter vector c and applying Lemma 3.1. Theorem 3.1. If S ¼ SRnn , then there must exist solutions to Problem I and solution set SA can be expressed as T
T
T
SA ¼ fA ¼ ðb þ cÞxþ þ ½ðb þ cÞxþ ðxþ Þ ðb þ cÞ xxþ þ ðI xxþ ÞY ðI xxþ Þj 8 c 2 Rn0 ; 8 Y 2 SRnn g:
ð3:2Þ
490
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
It is easy to find SA is a closed convex set, so that Problem II has a unique best approximation solution. Let A~1 ¼ ðA~ þ A~T Þ=2 and notice (1.1), (1.2), for A in (3.2) we have kA~ Ak2 ¼ kðA~ A~T Þ=2k2 þ kA~1 Ak2
A~ A~T 2 ¼ kðA~ A~T Þ=2k þ kU T ðA~1 AÞU k ¼ 2 ! uT A~ u r 1 uT ðb þ cÞ uT A~ U r 1 ðb þ cÞT U 2 2 1 1 1 1 1 1 2 þ U2T A~1 u1 r 1 U2T ðb þ cÞ U2T ðA~1 Y ÞU2 2 ! uT ½c ðA~ x bÞ ½c ðA~ x bÞ T U 2 A~ A~T 1 1 2
2 1 ¼ þr U2T ½c ðA~1 x bÞ 2 0 2
2
þ kU2T ðA~1 Y ÞU2 k2 ;
therefore minA2SA kA~ Ak is equivalent to uT ½c ðA~1 x bÞ ½c ðA~1 x bÞ T U2 ¼ min; 1 U T ½c ðA~ x bÞ c2Rn 0 1 0 2 kU2T ðA~1 Y ÞU2 k ¼ minnn : Y 2SR
ð3:3Þ ð3:4Þ
It is obvious that Y ¼ A~1 þ au1 uT1 ; 8 a 2 R
ð3:5Þ
is the minimum solution of (3.4). For studying the minimum problem of (3.3), we define T u1 h hT U2 n USRnn ¼ ; 8 h 2 R U2T h 0 T u1 h hT U2 nn n 8 h 2 R USR0 ¼ 0 U2T h 0 to orthogonal matrix U ¼ ðu1 ; U2 Þ in (1.1). Obviously, when orthogonal matrix U is definite, the elements in USRnn and USR0nn are one-to-one corresponding with elements in Rn and Rn0 respectively. Furthermore, USRnn is a subspace and USRnn is a closed convex cone. If ele0 ments in USRnn ðUSRnn Þ are denoted by capital letters in English, we denote 0 elements in Rn ðRn0 Þ with lowercase letters. Now, we consider the following problem: Problem M. Given H 2 USRnn ðh 2 Rn Þ, find G 2 USRnn ðg 2 Rn0 Þ such that 0 nn kG H k ¼ minD2USR0 kD H k
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
491
Note 1. If h ¼ A~1 x b in Problem M, g is the minimum solution to (3.3). Lemma 3.2. If V is a Hilbert space, k kv is inner product norm after h; i is defined as inner product in V. K V is a nonempty closed convex cone, we can know that there exists a unique t0 2 K ? , t1 2 K, t2 2 K for arbitrary t 2 V such that [3] t ¼ t0 þ t1 t2 ;
ht1 ; t2 i ¼ 0
where K is the dual cone of K in K
and ??
kt t1 kv 6 kt skv ; 8 s 2 K
.
By (1.2), for arbitrary F, H 2 USRnn , T T T u1 h hT U2 u1 f ðF ; H Þ ¼ trH T F ¼ tr T U2 h 0 U2T f T
¼ h ð2I
f T U2 0
ð3:6Þ
u1 uT1 Þf :
Lemma 3.3. In subspace USRnn , we have
? ðiÞ USR0nn ¼ f0g;
?? ¼ USRnn ; ðiiÞ USR0nn T
u1 f f T U2 nn T n ðiiiÞ USR0 ¼ f ¼ ðI þ u1 u1 Þy; 8 y 2 R0 ; U2T f 0
nn in USRnn . where ðUSRnn 0 Þ is the dual cone of USR0
Proof. (i) and (ii) are easy to prove. Now we prove (iii). By the
definition of dual . Hence, by (3.6), cone, ðUSR0nn Þ ¼ F 2 USRnn jðF ; GÞ P 0; 8 G 2 USRnn 0 we find for arbitrary F 2 ðUSR0nn Þ , ðF ; GÞ ¼ gT ð2I u1 uT1 Þf P 0, 8 g 2 Rn0 . By the arbitrary property of g 2 Rn0 , ð2I u1 uT1 Þf P 0 is obtained. Suppose 2y ð2I u1 uT1 Þf P 0, then f ¼ ðI þ u1 uT1 Þy, 8 y 2 Rn0 . This ends the lemma. We describe the following result immediately by applying Lemmas 3.2 and 3.3. Theorem 3.2. Given H 2 USRnn ðh 2 Rn Þ, there must exist a unique G 2 USR0nn T n ðg 2 Rn0 Þ, F 2 ðUSRnn 0 Þ ðf ¼ ðI þ u1 u1 Þy; 8 y 2 R0 Þ, such that H ¼ G F ðh ¼ g f ¼ g ðI þ u1 uT1 ÞyÞ;
ð3:7Þ
0 ¼ ðG; F Þ ¼ gT ð2I u1 uT1 Þf ¼ 2gT y;
ð3:8Þ
and GðgÞ is the solution of Problem M.
492
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
By (3.7) and (3.8), the problem of solving g; y 2 Rn0 can be transformed into linear complementarity problem: Problem L. Given u1 ; h, find y; g 2 Rn0 such that g ¼ h þ ðI þ u1 uT1 Þy P 0: gT y ¼ 0:
ð3:9Þ
Based on Theorems 3.1, 3.2 and Note 1, by (3.3)–(3.5) we can get such result as follows: Theorem 3.3. If S ¼ SRnn , then Problem II has a unique best approximation solution and it can be expressed as A^ ¼ ðb þ gÞxþ þ ½ðb þ gÞxþ T ðxþ ÞT ðb þ gÞT xxþ þ ðI xxþ ÞA~1 ðI xxþ Þ; where A~1 ¼ ðA~ þ A~T Þ=2, g is the solution of Problem M when h ¼ A~1 x b. Note 2. In Problem L, because ðI þ u1 uT1 Þ is a symmetric positive definite matrix, we can solve it by projective point successive over relaxation (PPSOR) method [4]. Algorithm 1. 2. 3. 4. 5.
Compute matrix A~1 ¼ ðA~ þ A~Þ=2. Compute h ¼ A~1 x b. Make a SVD for x, calculate xþ and u1 further. Solve Problem L through PPSOR method to get g. Calculate the best approximation solution A^ by (3.10). T
Example 1. Given vectors x ¼ ð 2; 1; 0; 5; 4; 2; 3Þ , b ¼ ð0; 2; 1; 1; 2; 5; 0Þ and matrix 1 0 4 2 1 4 0 0 6 B0 0 5 12 10 2 2 C C B C B B3 1 2 6 3 0 2 C C B A~ ¼ B 0 1 0 C C; B 0 2 4 1 C B 0 6 9 0
2
3
5 C B C B @ 2 4 0 3 1 0 2 A 4 0 6 2
1 0 2 find matrix A^ in matrix set SA ¼ f AjAx P b; A 2 SR77 g to satisfy k A^ A~k ¼ min kA A~k: 8 A2SA
T
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
493
The result obtained by computing in computer is A^ ¼ 0
4:000000
B B 1:000000 B B B 0:667966 B B B 2:000000 B B B 0:346667 B B B 1:000000 @ 5:000000
1:000000
0:667966
2:000000
0:346667
1:000000
0:000000
3:661017
7:000000
7:826667
1:000000
3:661017
2:000000
1:694915
3:355932
1:322034
7:000000
1:694915
1:000000
0:866667
2:000000
7:826667
3:355932
0:866667
3:386667
0:653333
1:000000
1:322034
2:000000
0:653333
0:000000
1:000000
5:983051
1:000000
2:480000
1:000000
5:000000
1
C 1:000000 C C C 5:983051 C C C 1:000000 C C: C
2:480000 C C C 1:000000 C A 2:000000
4. Case S ¼ ASRnn Lemma 4.1. If x; b 2 Rn , then there exists A 2 ASRnn to satisfy equation Ax ¼ b if and only if xT b ¼ 0. Furthermore, if solutions exist, general solutions can be expressed as [5] A ¼ bxþ ðxþ ÞT bT þ ðI xxþ ÞY ðI xxþ Þ; 8 Y 2 ASRnn :
ð4:1Þ
Sb ¼ fcjxT ðb þ cÞ ¼ 0;
ð4:2Þ
Let c 2 Rn0 g;
similar to Section 2, we can draw the following conclusion by introducing nonnegative parameter vector c and applying Lemma 4.1. Theorem 4.1. If S ¼ ASRnn , the necessary and sufficient condition for the existence of solutions of Problem I is that set Sb is nonempty. If solutions exist, general solution set is T T SA ¼ fA ¼ ðb þ cÞxþ ðxþ Þ ðb þ cÞ þ ðI xxþ ÞY ðI xxþ Þ 8 c 2 Sb ; 8 Y 2 ASRnn g:
ð4:3Þ
Theorem 4.2. If x P 0 ðx 6 0Þ, Sb is nonempty if and only if xT b 6 0 ðxT b P 0Þ. If x is common, Sb must be nonempty. Proof. When x P 0, if Sb is nonempty, i.e. exists c 2 Rn0 to satisfy xT ðb þ cÞ ¼ 0. Because xT c P 0, xT b ¼ xT c 6 0 is deduced. On the other hand, if xT b 6 0, xT c ¼ xT b P 0, it is apparent to know that this equation has nonnegative vector solution c. It is similar to prove the case when x 6 0. When x has not only positive entries but also negative entries, it is obvious for the equation xT c ¼ xT b to have nonnegative vector solution c. The theorem is proved.
494
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
Theorem 4.3. Given x; b 2 Rn , A~ 2 Rnn , S ¼ ASRnn , if Sb is nonempty, there exists a unique best approximation solution to Problem II and it can be expressed as T T A^ ¼ ðb þ c~Þxþ ðxþ Þ ðb þ c~Þ þ ðI xxþ ÞA~1 ðI xxþ Þ;
ð4:4Þ
where A~1 ¼ ðA~ A Þ=2 and c~ is the minimum solution to the following problem ~T
kA~1 x b c~k ¼ min kA~1 x b ck: c2Sb
ð4:5Þ
Proof. SA in (4.3) is obviously a closed convex set, therefore Problem II has a unique solution. Let A~1 ¼ ðA~ A~T Þ=2, noticing (1.1) and (1.2), for A in SA we have 2 2 2 kA~ Ak ¼ kðA~ þ A~T Þ=2k þ kA~1 Ak 2 2 ¼ kðA~ þ A~T Þ=2k þ kU T ðA~1 AÞU k A~ þ A~T 2 ¼ 2 !2 T uT1 A~1 U2 þ r 1 ðb þ cÞ U2 uT1 A~1 u1 þ U2T A~1 u1 r 1 U2T ðb þ cÞ U2T ðA~1 Y ÞU2
¼ kðA~ þ A~T Þ=2k2 þ 2kU2T A~1 u1 r 1 U2T ðb þ cÞk2 þ kU T ðA~1 Y ÞU2 k2 ; 2
ð4:6Þ the reason for the appearance of last expression is that because A1 2 ASRnn , T uT1 A~1 u1 ¼ 0 and ðuT1 A~1 U2 Þ ¼ U2T A~1 u1 . T Since x ðb þ cÞ ¼ 0, i.e. r 1 uT1 ðb þ cÞ ¼ 0, ! 2 uT A~ u r 1 uT ðb þ cÞ 2 T~
1 T 1 1 1 1 U2 A1 u1 r U2 ðb þ cÞ ¼ U2T A~1 u1 r 1 U2T ðb þ cÞ 2 2 ¼ kA~1 u1 r 1 ðb þ cÞk ¼ r 2 kA~1 x b ck :
ð4:7Þ From (4.6) and (4.7), we know that kA~ Ak ¼ minA2SA is equivalent to kA~1 x b ck ¼ min;
ð4:8Þ
kU2T ðA~1 Y ÞU2 k ¼ minnn :
ð4:9Þ
c2Sb
Y 2ASR
Substituting (4.3) with the minimum solutions of (4.8) and (4.9), we get (4.4) and end the proof.
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
495
Now if Sb is nonempty, we begin to analyse and consider the algorithm of solving the minimum solution c~ in (4.5). For arbitrary x 2 Rn , there exists a permutation matrix P such that x1 Px ¼ , where x1 2 Rk and each entry of x1 is nonzero. Let h,A~1 x b, 0 c1 b1 h1 k k Ph ¼ , h1 2 R , Pc ¼ , c1 2 R , Pb ¼ , b1 2 Rk , then h2 c2 b2 xT ðb þ cÞ ¼ 0 is equivalent to xT1 c1 ¼ xT1 b1 . Therefore, by invariance property of permutation matrix norm, we have 2
2
2
2
kA~1 x b ck ¼ kPh Pck ¼ kh1 c1 k þ kh2 c2 k ; then kA~1 x b ck ¼ minc2Sb is equivalent to 8 < kh1 c1 k2 ¼ min; s:t: xT1 c1 ¼ xT1 b1 ; : c1 P 0
ð4:10Þ
and kh2 c2 k ¼ min : c2 2Rn k 0
ð4:11Þ
Obviously, the minimum solution of (4.11) is c~2 ¼ ½h2 þ :
ð4:12Þ
(4.10) is a quadratic programming problem and can be transformed into a linear complementarity problem. So the minimum solution c~1 of this problem can be obtained by Lemke complementary principal element algorithm in c~1 limited steps (see [6]). Let c~ ¼ P , c~ is the very minimum solution of (4.5). c~2 Algorithm 1. For different cases of x, judge whether set Sb is nonempty by Theorem 4.2. If Sb is nonempty, goto 2, otherwise there’s no solution. 2. Compute A~1 ¼ ðA~ A~T Þ=2 and h ¼ A~1 x b. x1 k 3. Find a permutation matrix P to satisfy Px ¼ 0 , x12 R and each entry h1 b1 of x1 is nonzero. Compute Ph ¼ , h1 2 Rk , Pb ¼ , b1 2 Rk subseh2 b2 quently. 4. Solve (4.10) by Lemke Complementary Principal Element Algorithm in solving Linear Complementarity Problem to get c~1 . c~ 5. Gain c~2 through (4.12) and compute c~ ¼ P 1 further. c~2
496
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
6. Make a SVD for x and calculate xþ further. 7. Compute A^ according to (4.4). Example 2. Given vectors x ¼ ð1; 2; 0; 1; 0; 0; 4ÞT , b ¼ ð4; 2; 9; 2; 10;
6; 1ÞT and matrix 0
1 B 0 B B 4 B ~ A¼B B 1 B 2 B @ 0
2
2 0 4 3
1 5 0 4 1 0 2 1 0 4
3 2 6 1
2 4 1 0 2 4 10 2 0 6
4 6
1 2
2 1 0
1 0 0C C 2C C 4C C; 0C C 2A 2
find matrix A^ in matrix set SA ¼ fAjAx P b; A 2 ASR77 g to satisfy k A^ A~k ¼ min kA A~k: 8 A2SA
The result obtained by computing in computer is A^ ¼ 0
0 1:663636 B 0 B 1:663636 B B 0:136364 0:272727 B B B 2:818182 1:427273 B B 0:454545 0:909091 B B @ 0:272727 0:545455
0:672727
1
0:136364 0:272727 0
2:818182 1:427273
0:136364
0:454545
0:909091 0
0:272727 0:545455 0
0:136364 0 0
0 0:454545
0:272727
0:454545 0 0
0:272727 0 0
0:545455
0:418182
1:818182
1:090909
0:672727
1 0:545455
1
C C C C C C
0:418182 C C
1:818182 C C C 1:090909 A 0
Acknowledgement Research supported by National Natural Science Foundation of China (grant no. 19871024).
References [1] S.Q. Zhou, H. Dai, Inverse Problem of Algebra Eigenvalue, Henan Science and Technology Press, 1991. [2] C.R. Rao, S.K. Mitra, Generalized Inverse of Matrices and Its Applications, Wiley and Sons, New York, 1971. [3] L. Zhang, The Approximation on the Closed Convex Cone and Its Numerical Application, Ann. Hunan Math. 6 (2) (1986) 16–22.
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
497
[4] C.W. Cryer, The solution of a quadratic programming problem using systematic overrelaxation, SIAM J. Control 9 (1971) 385–392. [5] D.X. Xie, L. Zhang, Least-squares solutions of inverse problems for antisymmetric matrices, J. Eng. Math. 10 (4) (1993) 25–34. [6] M.S. Bazaraa, C.M. Shetty, Nonlinear Programming: Theory and Algorithms, John Wiley and Sons, 1979.
The best approximation of matrices under inequality constraints Xi-Yan Hu a
a,*
, Lei Zhang b, Qin Zhang
a
Department of Applied Mathematics, Hunan University, Changsha 410082, China b Hunan Computing Center, Changsha 410012, China
Abstract In this paper, a class of best approximation problems of matrices under inequality constraints are firstly presented and studied. For real matrices, symmetric matrices and antisymmetric matrices respectively, we have given the conditions for the existence of solutions and expressions of general solutions. Furthermore, the best approximation solutions and algorithms of them are provided. Ó 2002 Elsevier Science Inc. All rights reserved. Keywords: Best approximation; Inequality constraints; Inverse problem; Linear complementarity problem
1. Introduction Although a series of perfect results have been given for the best approximation problems of matrices under spectral constraints [1], the problems under inequality constraints have not been considered. In this paper, we will discuss these problems. Let Rn , Rn0 denote the sets of n-dimensional real vectors, nonnegative vectors, and Rnm , SRnn , ASRnn , ORnn denote the sets of real n m matrices, real n n symmetric matrices, real n n antisymmetric matrices and orthogonal n n matrices respectively. k k stands for the Frobenius norm of a matrix. Aþ represents the Moore-Penrose generalized inverse matrix of A and I is an
*
Corresponding author. E-mail address: [email protected] (X.-Y. Hu).
0096-3003/02/$ - see front matter Ó 2002 Elsevier Science Inc. All rights reserved. PII: S 0 0 9 6 - 3 0 0 3 ( 0 2 ) 0 0 1 5 3 - 4
488
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
identity matrix. Define inner product ðA; BÞ ¼ trðBT AÞ in Rnm and the folT lowing symbols are introduced for vector f ¼ ðf1 ; f2 ; . . . ; fn Þ : T ½f þ ¼ ðf1 ; f2 ; . . . ; fn Þ ; T
½f ¼ ðf~1 ; f~2 ; . . . ; f~n Þ ;
fi ¼ maxð0; fi Þ;
i ¼ 1; 2; . . . ; n;
f~i ¼ minð0; fi Þ;
i ¼ 1; 2; . . . ; n;
then f ¼ ½f þ þ ½f . Now we can present the best approximation problems of matrices under inequality constraints as follows: Problem I. Given 0 6¼ x, b 2 Rn and S Rnn , find A 2 S such that Ax P b. Problem II. Given A~ 2 Rnn , find A^ 2 SA such that k A^ A~k ¼ min kA A~k; A2SA
where SA is the set of solutions of Problem I. In this paper, we study three cases when S ¼ Rnn , SRnn , ASRnn respectively. The conditions for existence and expressions of general solutions of Problem I are obtained, and the expression of solution of Problem II and the algorithm to get it are also given. For convenience of discussion in the later sections, here we give the singularvalue decomposition (SVD) of a nonzero vector x 2 Rn as follows: r x¼U ¼ ru1 ; ð1:1Þ 0 pffiffiffiffiffiffiffi where U ¼ ðu1 ; U2 Þ 2 ORnn , r ¼ xT x > 0, u1 2 Rn1 . It is obvious to see that: xþ ¼ r 1 uT1 ;
I xxþ ¼ I u1 uT1 ¼ U2 U2T :
ð1:2Þ
2. Case S ¼ Rnn Lemma 2.1. Given 0 6¼ x; b 2 Rn , the general solution of equation Ax ¼ b is A ¼ bx þ Y ðI xxþ Þ, 8 Y 2 Rnn [2]. With the help of nonnegative parameter vector c, problem Ax P b is equivalent to equation Ax ¼ b þ c, c 2 Rn0 . Furthermore, by Lemma 2.1 we can draw the following conclusion. Theorem 2.1. Given S ¼ Rnn , there must exist solutions to Problem I and solution set SA can be expressed as SA ¼ fA ¼ ðb þ cÞxþ þ Y ðI xxþ Þj 8 c 2 Rn0 ; 8 Y 2 Rnn g:
ð2:1Þ
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
489
Theorem 2.2. If S ¼ Rnn , then there exists a unique best approximation solution to Problem II and it can be expressed as A^ ¼ ðb þ c~Þxþ þ A~ðI xxþ Þ;
c~ ¼ ½A~x b þ :
ð2:2Þ
Proof. From (2.1) we know that solution set SA of Problem I is a closed convex set, so that there exists a unique solution to Problem II. By (1.1), (1.2) and (2.1), we get A~ A ¼ A~ ðb þ cÞxþ Y ðI xxþ Þ ¼ ½A~ r 1 ðb þ cÞuT YU2 U T UU T 1
2
¼ ½ðA~u1 ; A~U2 Þ r 1 ðb þ c; 0Þ ð0; YU2 Þ U T : By invariance property of orthogonal matrix norm, it follows that kA~ Ak2 ¼ kA~u1 r 1 ðb þ cÞk2 þ kA~U2 YU2 k2 ; therefore kA~ Ak ¼ minA2SA is equivalent to kA~u1 r 1 ðb þ cÞk ¼ minn ;
ð2:3Þ
kA~U2 YU2 k ¼ min : nn
ð2:4Þ
c2R0
Y 2R
Noticing (1.1), when c~ ¼ ½A~x b þ , (2.3) reaches its minimum. It is also obvious that when Y ¼ A~ þ guT1 , 8 g 2 Rn , (2.4) reaches its minimum. Substitute (2.1) with above results to gain (2.2) and prove the theorem. 3. Case S ¼ SRnn Lemma 3.1. Given 0 6¼ x; b 2 Rn , there must exist solutions to equation Ax ¼ b in SRnn and solution set S0 can be expressed as [1] S0 ¼ fA ¼ bxþ þ ðbxþ ÞT ðxþ ÞT bT xxþ þ ðI xxþ ÞY ðI xxþ Þj 8 Y 2 SRnn g:
ð3:1Þ
Similar to Section 2, we can get the following result by introducing nonnegative parameter vector c and applying Lemma 3.1. Theorem 3.1. If S ¼ SRnn , then there must exist solutions to Problem I and solution set SA can be expressed as T
T
T
SA ¼ fA ¼ ðb þ cÞxþ þ ½ðb þ cÞxþ ðxþ Þ ðb þ cÞ xxþ þ ðI xxþ ÞY ðI xxþ Þj 8 c 2 Rn0 ; 8 Y 2 SRnn g:
ð3:2Þ
490
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
It is easy to find SA is a closed convex set, so that Problem II has a unique best approximation solution. Let A~1 ¼ ðA~ þ A~T Þ=2 and notice (1.1), (1.2), for A in (3.2) we have kA~ Ak2 ¼ kðA~ A~T Þ=2k2 þ kA~1 Ak2
A~ A~T 2 ¼ kðA~ A~T Þ=2k þ kU T ðA~1 AÞU k ¼ 2 ! uT A~ u r 1 uT ðb þ cÞ uT A~ U r 1 ðb þ cÞT U 2 2 1 1 1 1 1 1 2 þ U2T A~1 u1 r 1 U2T ðb þ cÞ U2T ðA~1 Y ÞU2 2 ! uT ½c ðA~ x bÞ ½c ðA~ x bÞ T U 2 A~ A~T 1 1 2
2 1 ¼ þr U2T ½c ðA~1 x bÞ 2 0 2
2
þ kU2T ðA~1 Y ÞU2 k2 ;
therefore minA2SA kA~ Ak is equivalent to uT ½c ðA~1 x bÞ ½c ðA~1 x bÞ T U2 ¼ min; 1 U T ½c ðA~ x bÞ c2Rn 0 1 0 2 kU2T ðA~1 Y ÞU2 k ¼ minnn : Y 2SR
ð3:3Þ ð3:4Þ
It is obvious that Y ¼ A~1 þ au1 uT1 ; 8 a 2 R
ð3:5Þ
is the minimum solution of (3.4). For studying the minimum problem of (3.3), we define T u1 h hT U2 n USRnn ¼ ; 8 h 2 R U2T h 0 T u1 h hT U2 nn n 8 h 2 R USR0 ¼ 0 U2T h 0 to orthogonal matrix U ¼ ðu1 ; U2 Þ in (1.1). Obviously, when orthogonal matrix U is definite, the elements in USRnn and USR0nn are one-to-one corresponding with elements in Rn and Rn0 respectively. Furthermore, USRnn is a subspace and USRnn is a closed convex cone. If ele0 ments in USRnn ðUSRnn Þ are denoted by capital letters in English, we denote 0 elements in Rn ðRn0 Þ with lowercase letters. Now, we consider the following problem: Problem M. Given H 2 USRnn ðh 2 Rn Þ, find G 2 USRnn ðg 2 Rn0 Þ such that 0 nn kG H k ¼ minD2USR0 kD H k
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
491
Note 1. If h ¼ A~1 x b in Problem M, g is the minimum solution to (3.3). Lemma 3.2. If V is a Hilbert space, k kv is inner product norm after h; i is defined as inner product in V. K V is a nonempty closed convex cone, we can know that there exists a unique t0 2 K ? , t1 2 K, t2 2 K for arbitrary t 2 V such that [3] t ¼ t0 þ t1 t2 ;
ht1 ; t2 i ¼ 0
where K is the dual cone of K in K
and ??
kt t1 kv 6 kt skv ; 8 s 2 K
.
By (1.2), for arbitrary F, H 2 USRnn , T T T u1 h hT U2 u1 f ðF ; H Þ ¼ trH T F ¼ tr T U2 h 0 U2T f T
¼ h ð2I
f T U2 0
ð3:6Þ
u1 uT1 Þf :
Lemma 3.3. In subspace USRnn , we have
? ðiÞ USR0nn ¼ f0g;
?? ¼ USRnn ; ðiiÞ USR0nn T
u1 f f T U2 nn T n ðiiiÞ USR0 ¼ f ¼ ðI þ u1 u1 Þy; 8 y 2 R0 ; U2T f 0
nn in USRnn . where ðUSRnn 0 Þ is the dual cone of USR0
Proof. (i) and (ii) are easy to prove. Now we prove (iii). By the
definition of dual . Hence, by (3.6), cone, ðUSR0nn Þ ¼ F 2 USRnn jðF ; GÞ P 0; 8 G 2 USRnn 0 we find for arbitrary F 2 ðUSR0nn Þ , ðF ; GÞ ¼ gT ð2I u1 uT1 Þf P 0, 8 g 2 Rn0 . By the arbitrary property of g 2 Rn0 , ð2I u1 uT1 Þf P 0 is obtained. Suppose 2y ð2I u1 uT1 Þf P 0, then f ¼ ðI þ u1 uT1 Þy, 8 y 2 Rn0 . This ends the lemma. We describe the following result immediately by applying Lemmas 3.2 and 3.3. Theorem 3.2. Given H 2 USRnn ðh 2 Rn Þ, there must exist a unique G 2 USR0nn T n ðg 2 Rn0 Þ, F 2 ðUSRnn 0 Þ ðf ¼ ðI þ u1 u1 Þy; 8 y 2 R0 Þ, such that H ¼ G F ðh ¼ g f ¼ g ðI þ u1 uT1 ÞyÞ;
ð3:7Þ
0 ¼ ðG; F Þ ¼ gT ð2I u1 uT1 Þf ¼ 2gT y;
ð3:8Þ
and GðgÞ is the solution of Problem M.
492
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
By (3.7) and (3.8), the problem of solving g; y 2 Rn0 can be transformed into linear complementarity problem: Problem L. Given u1 ; h, find y; g 2 Rn0 such that g ¼ h þ ðI þ u1 uT1 Þy P 0: gT y ¼ 0:
ð3:9Þ
Based on Theorems 3.1, 3.2 and Note 1, by (3.3)–(3.5) we can get such result as follows: Theorem 3.3. If S ¼ SRnn , then Problem II has a unique best approximation solution and it can be expressed as A^ ¼ ðb þ gÞxþ þ ½ðb þ gÞxþ T ðxþ ÞT ðb þ gÞT xxþ þ ðI xxþ ÞA~1 ðI xxþ Þ; where A~1 ¼ ðA~ þ A~T Þ=2, g is the solution of Problem M when h ¼ A~1 x b. Note 2. In Problem L, because ðI þ u1 uT1 Þ is a symmetric positive definite matrix, we can solve it by projective point successive over relaxation (PPSOR) method [4]. Algorithm 1. 2. 3. 4. 5.
Compute matrix A~1 ¼ ðA~ þ A~Þ=2. Compute h ¼ A~1 x b. Make a SVD for x, calculate xþ and u1 further. Solve Problem L through PPSOR method to get g. Calculate the best approximation solution A^ by (3.10). T
Example 1. Given vectors x ¼ ð 2; 1; 0; 5; 4; 2; 3Þ , b ¼ ð0; 2; 1; 1; 2; 5; 0Þ and matrix 1 0 4 2 1 4 0 0 6 B0 0 5 12 10 2 2 C C B C B B3 1 2 6 3 0 2 C C B A~ ¼ B 0 1 0 C C; B 0 2 4 1 C B 0 6 9 0
2
3
5 C B C B @ 2 4 0 3 1 0 2 A 4 0 6 2
1 0 2 find matrix A^ in matrix set SA ¼ f AjAx P b; A 2 SR77 g to satisfy k A^ A~k ¼ min kA A~k: 8 A2SA
T
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
493
The result obtained by computing in computer is A^ ¼ 0
4:000000
B B 1:000000 B B B 0:667966 B B B 2:000000 B B B 0:346667 B B B 1:000000 @ 5:000000
1:000000
0:667966
2:000000
0:346667
1:000000
0:000000
3:661017
7:000000
7:826667
1:000000
3:661017
2:000000
1:694915
3:355932
1:322034
7:000000
1:694915
1:000000
0:866667
2:000000
7:826667
3:355932
0:866667
3:386667
0:653333
1:000000
1:322034
2:000000
0:653333
0:000000
1:000000
5:983051
1:000000
2:480000
1:000000
5:000000
1
C 1:000000 C C C 5:983051 C C C 1:000000 C C: C
2:480000 C C C 1:000000 C A 2:000000
4. Case S ¼ ASRnn Lemma 4.1. If x; b 2 Rn , then there exists A 2 ASRnn to satisfy equation Ax ¼ b if and only if xT b ¼ 0. Furthermore, if solutions exist, general solutions can be expressed as [5] A ¼ bxþ ðxþ ÞT bT þ ðI xxþ ÞY ðI xxþ Þ; 8 Y 2 ASRnn :
ð4:1Þ
Sb ¼ fcjxT ðb þ cÞ ¼ 0;
ð4:2Þ
Let c 2 Rn0 g;
similar to Section 2, we can draw the following conclusion by introducing nonnegative parameter vector c and applying Lemma 4.1. Theorem 4.1. If S ¼ ASRnn , the necessary and sufficient condition for the existence of solutions of Problem I is that set Sb is nonempty. If solutions exist, general solution set is T T SA ¼ fA ¼ ðb þ cÞxþ ðxþ Þ ðb þ cÞ þ ðI xxþ ÞY ðI xxþ Þ 8 c 2 Sb ; 8 Y 2 ASRnn g:
ð4:3Þ
Theorem 4.2. If x P 0 ðx 6 0Þ, Sb is nonempty if and only if xT b 6 0 ðxT b P 0Þ. If x is common, Sb must be nonempty. Proof. When x P 0, if Sb is nonempty, i.e. exists c 2 Rn0 to satisfy xT ðb þ cÞ ¼ 0. Because xT c P 0, xT b ¼ xT c 6 0 is deduced. On the other hand, if xT b 6 0, xT c ¼ xT b P 0, it is apparent to know that this equation has nonnegative vector solution c. It is similar to prove the case when x 6 0. When x has not only positive entries but also negative entries, it is obvious for the equation xT c ¼ xT b to have nonnegative vector solution c. The theorem is proved.
494
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
Theorem 4.3. Given x; b 2 Rn , A~ 2 Rnn , S ¼ ASRnn , if Sb is nonempty, there exists a unique best approximation solution to Problem II and it can be expressed as T T A^ ¼ ðb þ c~Þxþ ðxþ Þ ðb þ c~Þ þ ðI xxþ ÞA~1 ðI xxþ Þ;
ð4:4Þ
where A~1 ¼ ðA~ A Þ=2 and c~ is the minimum solution to the following problem ~T
kA~1 x b c~k ¼ min kA~1 x b ck: c2Sb
ð4:5Þ
Proof. SA in (4.3) is obviously a closed convex set, therefore Problem II has a unique solution. Let A~1 ¼ ðA~ A~T Þ=2, noticing (1.1) and (1.2), for A in SA we have 2 2 2 kA~ Ak ¼ kðA~ þ A~T Þ=2k þ kA~1 Ak 2 2 ¼ kðA~ þ A~T Þ=2k þ kU T ðA~1 AÞU k A~ þ A~T 2 ¼ 2 !2 T uT1 A~1 U2 þ r 1 ðb þ cÞ U2 uT1 A~1 u1 þ U2T A~1 u1 r 1 U2T ðb þ cÞ U2T ðA~1 Y ÞU2
¼ kðA~ þ A~T Þ=2k2 þ 2kU2T A~1 u1 r 1 U2T ðb þ cÞk2 þ kU T ðA~1 Y ÞU2 k2 ; 2
ð4:6Þ the reason for the appearance of last expression is that because A1 2 ASRnn , T uT1 A~1 u1 ¼ 0 and ðuT1 A~1 U2 Þ ¼ U2T A~1 u1 . T Since x ðb þ cÞ ¼ 0, i.e. r 1 uT1 ðb þ cÞ ¼ 0, ! 2 uT A~ u r 1 uT ðb þ cÞ 2 T~
1 T 1 1 1 1 U2 A1 u1 r U2 ðb þ cÞ ¼ U2T A~1 u1 r 1 U2T ðb þ cÞ 2 2 ¼ kA~1 u1 r 1 ðb þ cÞk ¼ r 2 kA~1 x b ck :
ð4:7Þ From (4.6) and (4.7), we know that kA~ Ak ¼ minA2SA is equivalent to kA~1 x b ck ¼ min;
ð4:8Þ
kU2T ðA~1 Y ÞU2 k ¼ minnn :
ð4:9Þ
c2Sb
Y 2ASR
Substituting (4.3) with the minimum solutions of (4.8) and (4.9), we get (4.4) and end the proof.
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
495
Now if Sb is nonempty, we begin to analyse and consider the algorithm of solving the minimum solution c~ in (4.5). For arbitrary x 2 Rn , there exists a permutation matrix P such that x1 Px ¼ , where x1 2 Rk and each entry of x1 is nonzero. Let h,A~1 x b, 0 c1 b1 h1 k k Ph ¼ , h1 2 R , Pc ¼ , c1 2 R , Pb ¼ , b1 2 Rk , then h2 c2 b2 xT ðb þ cÞ ¼ 0 is equivalent to xT1 c1 ¼ xT1 b1 . Therefore, by invariance property of permutation matrix norm, we have 2
2
2
2
kA~1 x b ck ¼ kPh Pck ¼ kh1 c1 k þ kh2 c2 k ; then kA~1 x b ck ¼ minc2Sb is equivalent to 8 < kh1 c1 k2 ¼ min; s:t: xT1 c1 ¼ xT1 b1 ; : c1 P 0
ð4:10Þ
and kh2 c2 k ¼ min : c2 2Rn k 0
ð4:11Þ
Obviously, the minimum solution of (4.11) is c~2 ¼ ½h2 þ :
ð4:12Þ
(4.10) is a quadratic programming problem and can be transformed into a linear complementarity problem. So the minimum solution c~1 of this problem can be obtained by Lemke complementary principal element algorithm in c~1 limited steps (see [6]). Let c~ ¼ P , c~ is the very minimum solution of (4.5). c~2 Algorithm 1. For different cases of x, judge whether set Sb is nonempty by Theorem 4.2. If Sb is nonempty, goto 2, otherwise there’s no solution. 2. Compute A~1 ¼ ðA~ A~T Þ=2 and h ¼ A~1 x b. x1 k 3. Find a permutation matrix P to satisfy Px ¼ 0 , x12 R and each entry h1 b1 of x1 is nonzero. Compute Ph ¼ , h1 2 Rk , Pb ¼ , b1 2 Rk subseh2 b2 quently. 4. Solve (4.10) by Lemke Complementary Principal Element Algorithm in solving Linear Complementarity Problem to get c~1 . c~ 5. Gain c~2 through (4.12) and compute c~ ¼ P 1 further. c~2
496
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
6. Make a SVD for x and calculate xþ further. 7. Compute A^ according to (4.4). Example 2. Given vectors x ¼ ð1; 2; 0; 1; 0; 0; 4ÞT , b ¼ ð4; 2; 9; 2; 10;
6; 1ÞT and matrix 0
1 B 0 B B 4 B ~ A¼B B 1 B 2 B @ 0
2
2 0 4 3
1 5 0 4 1 0 2 1 0 4
3 2 6 1
2 4 1 0 2 4 10 2 0 6
4 6
1 2
2 1 0
1 0 0C C 2C C 4C C; 0C C 2A 2
find matrix A^ in matrix set SA ¼ fAjAx P b; A 2 ASR77 g to satisfy k A^ A~k ¼ min kA A~k: 8 A2SA
The result obtained by computing in computer is A^ ¼ 0
0 1:663636 B 0 B 1:663636 B B 0:136364 0:272727 B B B 2:818182 1:427273 B B 0:454545 0:909091 B B @ 0:272727 0:545455
0:672727
1
0:136364 0:272727 0
2:818182 1:427273
0:136364
0:454545
0:909091 0
0:272727 0:545455 0
0:136364 0 0
0 0:454545
0:272727
0:454545 0 0
0:272727 0 0
0:545455
0:418182
1:818182
1:090909
0:672727
1 0:545455
1
C C C C C C
0:418182 C C
1:818182 C C C 1:090909 A 0
Acknowledgement Research supported by National Natural Science Foundation of China (grant no. 19871024).
References [1] S.Q. Zhou, H. Dai, Inverse Problem of Algebra Eigenvalue, Henan Science and Technology Press, 1991. [2] C.R. Rao, S.K. Mitra, Generalized Inverse of Matrices and Its Applications, Wiley and Sons, New York, 1971. [3] L. Zhang, The Approximation on the Closed Convex Cone and Its Numerical Application, Ann. Hunan Math. 6 (2) (1986) 16–22.
X.-Y. Hu et al. / Appl. Math. Comput. 137 (2003) 487–497
497
[4] C.W. Cryer, The solution of a quadratic programming problem using systematic overrelaxation, SIAM J. Control 9 (1971) 385–392. [5] D.X. Xie, L. Zhang, Least-squares solutions of inverse problems for antisymmetric matrices, J. Eng. Math. 10 (4) (1993) 25–34. [6] M.S. Bazaraa, C.M. Shetty, Nonlinear Programming: Theory and Algorithms, John Wiley and Sons, 1979.