Typical ranks for m×n×(m-1)n tensors with m⩽n

Linear Algebra and its Applications 438 (2013) 953–958

Contents lists available at SciVerse ScienceDirect

Linear Algebra and its Applications journal homepage: w w w . e l s e v i e r . c o m / l o c a t e / l a a

Typical ranks for m × n × (m − 1)n tensors with m  n < Toshio Sumi a,∗ Toshio Sakata a Mitsuhiro Miyazaki b a b

Kyushu University, 4-9-1, Shiobaru, Minami-ku, Fukuoka 815-8540, Japan Kyoto University of Education, Fushimi-ku, Kyoto 612-8522, Japan

ARTICLE INFO

ABSTRACT

Article history: Received 27 June 2010 Accepted 22 July 2011 Available online 27 August 2011

In various application fields, tensor type data are used recently and then a typical rank is important. There may be more than one typical ranks over the real number field. It is well known that the set of 2 × n × n tensors has two typical ranks n, n + 1 for n  2, that the set of 3 × 4 × 8 tensors has two typical ranks 8, 9, and that the set of 4 × 4 × 12 tensors has two typical ranks 12, 13. In this paper, we show that the set of m × n × (m − 1)n tensors with m  n has two typical ranks (m − 1)n, (m − 1)n + 1 if m  ρ(n), where ρ is the Hurwitz–Radon function defined as ρ(n) = 2b + 8c for nonnegative integers a, b, c such that n = (2a + 1)2b + 4c and 0  b < 4. © 2011 Elsevier Inc. All rights reserved.

Submitted by V. Mehrmann AMS classification: 15A69 15A72 14Q99 14M12 14M99 11E39 05B20 05B15 Keywords: Typical rank Tensor Bilinear map Orthogonal design

1. Introduction An m × n × p tensor over a field F is an element of the tensor product of three vector spaces Fm , F and Fp . Thus every tensor can be expressed as a sum of tensors of the form a ⊗ b ⊗ c for a ∈ Fm , b ∈ Fn , and c ∈ Fp . The rank of a tensor T is the minimum number r of rank one tensors which express T as a sum, and it is denoted by rank F T. The rank depends on the field. The set T (m, n, p; F) of all n

< The authors were supported partially by Grant-in-Aid for Scientific Research (B) (No. 20340021) of the Japan Society for the Promotion of Science. ∗ Corresponding author. E-mail addresses: [email protected] (T. Sumi), [email protected] (T. Sakata), [email protected] (M. Miyazaki). 0024-3795/$ - see front matter © 2011 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.laa.2011.08.009

954

T. Sumi et al. / Linear Algebra and its Applications 438 (2013) 953–958

m × n × p tensors is Fmnp as a set. We denote an m × n × p tensor A by (A1 ; . . . ; Ap ) where A1 , . . . , Ap are m × n matrices whose entries are in F. Then rank F A is the smallest number r such that there are an m × r matrix P, an n × r matrix Q and diagonal r × r matrices D1 , . . . , Dp such that Ak = PDk Q T for each k = 1, . . . , p. This decomposition is well known as the CANDECOMP/PARAFAC decomposition in r dimensions [8,5]. PARAFAC analysis is used in various fields (cf. [14,9,17,22,15,23]). Now let F be the real number field R or the complex number field C. An integer r is called “typical rank” if the set of rank r tensors has a positive Lebesgue measure in T (m, n, p; F). The typical rank may be not necessarily unique. In fact m and m + 1 are typical ranks for T (m, m, 2; R) for an integer m  2 [21]. So we denote the set of typical ranks for T (m, n, p; F) by typical_rankF (m, n, p). The typical rank over the complex number field is unique, which is equal to the minimal number of the typical rank over the real number field. The typical rank has attracted the interest of several researchers [13,18,4,12]. mnp Every typical rank of m × n × p tensor is between m+n+p−2 and min{mn, np, pm} [10]. Atkinson and Lloyd [3] showed that the typical rank for 3 × n × 2n tensors with 2  n is 2n over the complex number field. ten Berge [20] determined the typical rank for m × n × p tensors with 2  m  n  p and (m − 1)n < p over the real number field, that is, typical_rankR (m, n, p) = min{p, mn}. Also he studied a typical rank for m × n × p tensors with p = (m − 1)n and showed that it is either {p} or {p, p + 1}. He gave typical_rankR (3, 4, 8) = {8, 9}. Moreover, Friedland [6] studied about several kind of ranks and showed that typical_rankR (4, 4, 12) = {12, 13}. Let ρ(n) be the Hurwitz–Radon function defined as ρ(n) = 2b + 8c, where a, b, c are nonnegative integers such that n = (2a + 1)2b+4c and 0  b < 4 [16,11]. In this note, we show the following theorem by developing ten Berge’s technique. Theorem 1.1. Suppose that 2

 m  n. It holds that

typical_rankR (m, n, (m − 1)n) for m

= {(m − 1)n, (m − 1)n + 1}

 ρ(n).

In Section 2 we define an absolutely nonsingular tensor. This concept gives a sufficient condition that typical_rankR (m, n, (m − 1)n) = {(m − 1)n, (m − 1)n + 1} (see Theorem 3.4). 2. Absolutely nonsingular tensors We denote by T (m, n, p) the set of all m × n × p tensors over the real number field. In this note, two n × n × p tensors (A1 ; . . . ; Ap ) and (B1 ; . . . ; Bp ) are called equivalent if there exist nonsingular matrices P and Q such that Ai = PBi Q for i = 1, . . . , p. Note that if two tensors are equivalent the ranks of them are same. An n × n × p tensor (A1 ; . . . ; Ap ) is called absolutely nonsingular if the equation ⎞ ⎛ p  xk Ak ⎠ = 0 det ⎝ k=1

implies (x1 , . . . , xp )T = 0. Note that if (A1 ; . . . ; Ap ) is an absolutely nonsingular tensor with p >   1, then either det( xk Ak ) > 0 for any (x1 , . . . , xp )T ∈ Rp  {0} or det( xk Ak ) < 0 for any (x1 , . . . , xp )T ∈ Rp  {0}. A tensor equivalent to an absolutely nonsingular tensor is also an absolutely nonsingular tensor. This result appeared in [19]. If an n × n × p tensor (A1 ; . . . ; Ap ) is absolutely nonsingular then so is (A1 ; . . . ; Aq ) for q  p. Let (A1 ; . . . ; Ap ) be an n × n × p tensor. We define a bilinear map f : Rp × Rn → Rn as f (x, y )

=

p  n  i=1 j=1

xi yj aij ,

where x = (x1 , . . . , xp )T , y = (y1 , . . . , yn )T , and Ak = (ak1 , . . . , akn ) for 1  k  n. A bilinear map g : Rp × Rn → Rn is called nonsingular if g (x, y) = 0 implies x = 0 or y = 0 and is called normed if

T. Sumi et al. / Linear Algebra and its Applications 438 (2013) 953–958

955

||g (x, y)|| = ||x||||y ||, where ||x|| denotes the Euclidean norm of x. Any normed bilinear map is nonsingular. The theory of orthogonal designs [7] is also related with a nonsingular bilinear map. It is clear that Lemma 2.1. (A1 ; . . . ; Ap ) is an absolutely nonsingular tensor if and only if f is a nonsingular bilinear map. There exists a normed bilinear map Rρ(n) × Rn nonsingular bilinear map Rp × Rn → Rn , then p 

→ Rn [16,11,7]. Furthermore, if there exists a ρ(n) [1,2]. Therefore, we have

Theorem 2.2. There exists an n × n × p absolutely nonsingular tensor if and only if p

 ρ(n).

Let f : Rm × Rm → Rm be a nonsingular bilinear map. It occurs only if m = 1, 2, 4 or 8. For the bilinear map g : Rm × Rm → Rm given by g (x, y ) = f (y , x), f is nonsingular if and only if g is nonsingular. Thus, the next proposition follows directly from Lemma 2.1. Proposition 2.3. Let T = (A1 ; . . . ; Am ) be an m × m × m tensor. Set Ai = (ai1 , . . . , aim ) for i = 1, …, m, Bi = (a1i , . . . , ami ) for i = 1, …, m and T  = (B1 ; . . . ; Bm ). Then T is absolutely nonsingular if and only if so is T  . Therefore absolute nonsingularity does not depend on the direction for slices. Finally we show that the set of absolutely nonsingular tensors is an open set in the Euclidean topology. The following lemma is a key. Lemma 2.4. Let f : S × T → R be a continuous map and s0 ∈ S satisfying that f (s0 , t ) > 0 for any t ∈ K, where K is a compact subset of T. Then there is an open set U in S such that s0 ∈ U and f (s, t ) > 0 for any (s, t ) ∈ U × K. Proof. For each t ∈ K, we take an open subset Ut of S and Vt of T such that s0 ∈ Ut , t ∈ Vt and f (s, v) > 0 for any (s, v) ∈ Ut × Vt . Since K is compact, there are t1 , . . . , tn ∈ K such that Vt1 ∪ Vt2 ∪· · ·∪ Vtn ⊃ K. Put U = Ut1 ∩ Ut2 ∩ · · · ∩ Utn . Then U is an open set of S with s0 ∈ U. Suppose (s, t ) ∈ U × K. Then there is j such that t ∈ Vtj . Since (s, t ) ∈ Utj × Vtj , we see that f (s, t ) > 0.  Theorem 2.5. Let M (n) be the set of all n × n matrices over R, m a continuous map defined as ⎞ ⎛ m  xk Xk ⎠ . f (x1 , . . . , xm , X1 , . . . , Xm ) = det ⎝

 2 and let f : Rm × M (n)m → R be

k=1

Let T ∈ M (n)m . Suppose that f (x, T ) > 0 for any x ∈ Rm  {0}. Then there is an open subset U of M (n)m such that T ∈ U, and f (x, X ) > 0 for any (x, X ) ∈ (Rm  {0}) × U. Proof. Let S m−1 = {x ∈ Rm | ||x|| = 1}, where || · || denotes the Euclidean norm. By Lemma 2.4, there is an open set U in M (n)m such that T ∈ U and f (x, X ) > 0 for any x ∈ S m−1 and X ∈ U. 1 1 Suppose (x, X ) ∈ (Rm  {0}) × U. Since x ∈ S m−1 , it holds that f ( x, X ) > 0. Therefore ||x|| ||x|| 1 f (x, X ) = ||x||n f ( x, X ) > 0.  ||x|| By Theorem 2.5, for an absolutely nonsingular tensor T, there is an open neighborhood of T consisting of absolutely nonsingular tensors. Remark that if f (x, T ) > 0 for any x ∈ Rm  {0} then n is even since f (−x, T ) = (−1)n f (x, T ). 3. Typical rank and absolute nonsingularity In this section we prove Theorem 1.1. The following lemmas are well known.

956

T. Sumi et al. / Linear Algebra and its Applications 438 (2013) 953–958

Lemma 3.1. Let 2

 m  n  p  mn. If r is a typical rank of T (m, n, p) then r  p.

Proof. Let A1 , . . . , Ap be m × n matrices. Then generically, they are linearly independent, that is the dimension d of the vector space spanned by them is p. Since the rank of (A1 ; . . . ; Ap ) is greater than or equal to the dimension d, we conclude the assertion.  Lemma 3.2. typical_rankR (m, n, p)

= typical_rankR (n, m, p) = typical_rankR (p, n, m).

The next theorem is due to results in [20]. Theorem 3.3 [20]. Let 2  m  n. typical_rankR (m, n, p) is equal to {mn} for p n(m − 1) < p < mn, and either {p} or {p, p + 1} for p = n(m − 1).

 mn, {p} for

Now we prove the following Theorem 3.4. Let 2

 m  n. If T (n, n, m) has an absolutely nonsingular tensor then

typical_rankR (m, n, (m − 1)n)

= {(m − 1)n, (m − 1)n + 1}.

Proof. Let p = (m − 1)n. By Theorem 3.3 (cf. [20, Result 3]), a typical rank of T (m, n, p) is either {p} or {p, p + 1}. Thus it suffices to show that a number other than p appears as a typical rank of T (m, n, p). We show that there is an open set U such that rank X > p for X ∈ U. Since T (n, n, m) has an absolutely nonsingular tensor and the absolute nonsingularity is preserved by the equivalence relation, we may assume that there is an absolutely nonsingular tensor (En ; A1 ; . . . ; Am−1 ). Take an n × p × m tensor X = (X1 ; . . . ; Xm ) defined as ⎞ ⎛ X1 ⎟ ⎜ ⎟ ⎜ ⎜ .. ⎟ ⎜ . ⎟ = Ep ⎟ ⎜ ⎠ ⎝ Xm−1 and Xm = (A1 , . . . , Am−1 ). By Lemma 2.5 there is an open neighborhood V of Xm such that En , A1 , . . . , Am−1 has no singular matrix except a zero matrix for any (A1 , . . . , Am−1 ) ∈ V . Consider a continuous ⎛ ⎞ Ep map ϕ from an open neighborhood of ⎝ ⎠ to M (n, p), the set of all n × p matrices, given by Xm ⎛ ⎜ ⎜ ⎜ ⎜ ⎝

X1

.. .

X

m

⎞ ⎟ ⎟ ⎟ ⎟ ⎠

⎛

→

⎞−1  ⎜ X1 ⎟  ⎜ .. ⎟ ⎜ . ⎟ . Xm ⎝ ⎠  Xm −1 ⎛

Set U ⎛

= ϕ −1 (V ). We show that rank (X1 ; . . . ; Xm ) >

⎞  X 1 ⎜ ⎟ ⎜ .. ⎟ ⎜ . ⎟ is nonsingular it holds that rank X  ⎝ ⎠  Xm −1 a tensor given by



⎞

⎜ X1 ⎟ ⎜ . ⎟ p for ⎜ .. ⎟ in U. Set X  ⎝ ⎠  Xm

= (X1 ; . . . ; Xm ). Since

 p. Assume that rank X  = p. Let Z = (Z1 ; . . . ; Zm ) be

T. Sumi et al. / Linear Algebra and its Applications 438 (2013) 953–958

⎛ ⎛ ⎜ ⎜ ⎜ ⎜ ⎝

⎞

⎛

Z1

X1

⎞⎛

X1

⎞−1

⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟⎜ ⎟ .. ⎟ ⎟ = ⎜ .. ⎟ ⎜ .. ⎟ . ⎟ ⎜ . ⎟⎜ . ⎟ ⎠

Zm

⎝ ⎠⎝ ⎠   Xm Xm −1

=

⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝

957

⎞

En En

..

. En

Y1 Y2

. . . Ym−1

⎟ ⎟ ⎟ ⎟ ⎟ ⎟. ⎟ ⎟ ⎟ ⎟ ⎠

Then rank X  = rank Z and (Y1 , . . . , Ym−1 ) ∈ V . There are an n × p matrix P and a p × p matrix Q and diagonal matrices Dk (k = 1, . . . , m), such that Zk = PDk Q for k = 1, . . . , m. Since ⎛ ⎞ ⎛ ⎞ Z1 PD1 ⎜ ⎟ ⎜ ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎟ ⎜ ⎟ Ep = ⎜ ⎜ .. ⎟ = ⎜ .. ⎟ Q ⎝ ⎠ ⎝ ⎠ Zm−1 PDm−1 the matrix Q is nonsingular. Putting R ⎛ ⎞ is equivalent to R

=

⎜ ⎜ ⎜ ⎜ ⎝

PD1

.. .

= Q −1 , consider an equation Zk R = PDk for k = 1, . . . , m. This

⎟ ⎟ ⎟ and Zm R ⎟ ⎠

= PDm . Therefore Y1 PD1 + · · · + Ym−1 PDm−1 − PDm = O.

PDm−1

Putting Di

= Diag(di1 , . . . , dip ), for a jth column vector u of P it holds that

(d1j Y1 + · · · + dm−1,j Ym−1 − dmj En )u = 0, which implies that d1j = · · · = dm−1,j = dmj = 0. Then D1 = · · · = Dm−1 = 0 and thus R which is a contradiction against that R is nonsingular. Therefore rank X  = p.  By Theorem 3.4, for 2

=O

 m  ρ(n), it holds that

typical_rankR (m, n, (m − 1)n)

= {(m − 1)n, (m − 1)n + 1}.

This completes the proof of Theorem 1.1. We note that Theorem 1.1 covers that the typical rank for T (3, 4, 8) is {8, 9} by ten Berge [20] and that the typical rank for T (4, 4, 12) is {12, 13} by Friedland [6]. However the typical ranks for T (3, 2(2k + 1), 4(2k + 1)) and for T (4, 2(2k + 1), 6(2k + 1)), k  1 are still open. Acknowledgement The authors are deeply grateful to the referees for their carefully reading the manuscript and giving many valuable comments and suggestions. References [1] J.F. Adams, Vector fields on spheres, Ann. Math. 75 (2) (1962) 603–632. [2] J.F. Adams, Peter D. Lax, Ralph S. Phillips, On matrices whose real linear combinations are non-singular, Proc. Amer. Math. Soc. 16 (1965) 318–322., Correction, 17 (1966) 945–947. [3] M.D. Atkinson, S. Lloyd, Bounds on the ranks of some 3-tensors, Linear Algebra Appl. 31 (1980) 19–31. [4] P. Comon, J.M.F. ten Berge, L. De Lathauwer, J. Castaing, Generic and typical ranks of multi-way arrays, Linear Algebra Appl. 430 (11–12) (2009) 2997–3007. [5] Lieven De Lathauwer, A link between the canonical decomposition in multilinear algebra and simultaneous matrix diagonalization, SIAM J. Matrix Anal. Appl. 28 (3) (2006) 642–666. (electronic). [6] Shmuel Friedland, On the generic and typical ranks of 3-tensors, 2011. http://arxiv.org/abs/0805.3777v5. [7] Anthony V. Geramita, Jennifer Seberry, Orthogonal Designs, Quadratic Forms and Hadamard Matrices, Lecture Notes in Pure and Applied Mathematics, vol. 45, Marcel Dekker Inc., New York, 1979.

958

T. Sumi et al. / Linear Algebra and its Applications 438 (2013) 953–958

[8] Richard A. Harshman, Foundations of the parafac procedure: models and conditions for an explanatory multimodal factor analysis, UCLA Working Pap. Phonet. 16 (1970) 1–84. [9] Richard A. Harshman, Sungjin Hong, Margaret E. Lundy, Shifted factor analysis — part I: models and properties, J. Chemomet. 17 (2003) 363–378. [10] Thomas D. Howell, Global properties of tensor rank, Linear Algebra Appl. 22 (1978) 9–23. [11] A. Hurwitz, Über die Komposition der quadratischen Formen, Math. Ann. 88 (1–2) (1922) 1–25. [12] Tamara G. Kolda, Brett W. Bader, Tensor decompositions and applications, SIAM Rev. 51 (3) (2009) 455–500. [13] Joseph B. Kruskal, Three-way arrays: rank and uniqueness of trilinear decompositions with, application to arithmetic complexity and statistics, Linear Algebra Appl. 18 (2) (1977) 95–138. [14] Edmund R. Malinowski, Factor Analysis in Chemistry, John Wiley & Sons, 2002. [15] D. Muti, S. Bourennane, Survey on tensor signal algebraic filtering, Signal Process. Arch. 87 (2) (2007) 237–249. [16] Von J. Radon, Lineare Scharen orthogonaler Matrizen, Abh. Math. Sem. Univ. Hamburg 1 (1922) 1–14. [17] Age K. Smilde, Rasmus Bro, Paul Geladi, Multi-way Analysis: Applications in the Chemical Sciences (Translated by K.A. Hirsch), John Wiley & Sons, 2004. [18] V. Strassen, Rank and optimal computation of generic tensors, Linear Algebra Appl. 52/53 (1983) 645–685. [19] T. Sumi, M. Miyazaki, T. Sakata, About the maximal rank of 3-tensors over the real and the complex number field, Ann. Inst. Statist. Math. 62 (4) (2010) 807–822. [20] Jos M.F. ten Berge, The typical rank of tall three-way arrays, Psychometrika 65 (4) (2000) 525–532. [21] Jos M.F. ten Berge, Henk A.L. Kiers, Simplicity of core arrays in three-way principal component analysis and the typical rank of p × q × 2 arrays, Linear Algebra Appl. 294 (1–3) (1999) 169–179. [22] M.A.O. Vasilescu, D. Terzopoulos, Multilinear independent components analysis, in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, 2005, pp. 547–553. [23] Hai-Long Wu, Jin-Fang Nie, Yong-Jie Yu, Ru-Qin Yu, Multi-way chemometric methodologies and applications: a central summary of our research work, J. Chemomet. 17 (2003) 274–286.