Iteration of holomorphic maps on Lie balls

Advances in Mathematics 264 (2014) 114–154

Contents lists available at ScienceDirect

Advances in Mathematics www.elsevier.com/locate/aim

Iteration of holomorphic maps on Lie balls Cho-Ho Chu School of Mathematical Sciences, Queen Mary, University of London, London E1 4NS, UK

a r t i c l e

i n f o

Article history: Received 16 September 2013 Accepted 7 July 2014 Available online xxxx Communicated by Gang Tian MSC: 32H50 32M15 17C65 58C10 46L70 Keywords: Lie ball Holomorphic iteration Bounded symmetric domain Spin factor JB*-triple

a b s t r a c t We investigate iterations of fixed-point free holomorphic selfmaps on a Lie ball of any dimension, where a Lie ball is a bounded symmetric domain and the open unit ball of a spin factor which can be infinite dimensional. We describe the invariant domains of a holomorphic self-map f on a Lie ball D when f is fixed-point free and compact, and show that each limit function of the iterates (f n ) has values in a onedimensional disc on the boundary of D. We show that the Möbius transformation ga induced by a nonzero element a in D may fail the Denjoy–Wolff-type theorem, even in finite dimension. We determine those which satisfy the theorem. © 2014 Elsevier Inc. All rights reserved.

1. Introduction Since the seminal works of Julia [17], Denjoy [10] and Wolff [28,29], there has been extensive literature on iteration of holomorphic maps on various complex domains in finite or infinite dimensions. In particular, iteration of holomorphic maps on the Euclidean balls and infinite dimensional Hilbert balls has been widely studied (see for example, E-mail address: [email protected]. http://dx.doi.org/10.1016/j.aim.2014.07.008 0001-8708/© 2014 Elsevier Inc. All rights reserved.

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

115

[7,8,13,15,18,23,25]). These domains are bounded symmetric domains of rank one. Finite dimensional bounded symmetric domains have been classified by É. Cartan [5]. Among the irreducible bounded symmetric domains of rank two, Lie balls stand out as one of the four classical series of Cartan domains and are an important mathematical model for physics [12]. This motivates our study of holomorphic iteration on a Lie ball. To investigate iteration of a holomorphic map f : D −→ D on a bounded domain D in a complex Banach space, one studies the asymptotic behaviour of the iterates (f n ). For this, the invariant domains of f play a useful role and are by themselves interesting objects of study. These are defined to be the domains S in D satisfying f (S) ⊂ S, where S denotes the closure of S. While one can establish the existence of invariant domains via a limit of hyperbolic balls, and observe that a limit function of (f n ) takes values in certain subset of the closure D, the problem is to describe these domains and limit sets explicitly in such a way that definitive conclusions can be drawn. In the case of a Hilbert ball or some other domains, the latter task has been accomplished and definitive results, for example, an analogue of the Denjoy–Wolff theorem, are established. Our objective in this paper is to perform similar tasks for Lie balls. We give in Theorem 4.4, Theorem 4.6 and Section 6 an explicit description of these invariant domains, called horospheres, and the limit sets for a fixed-point free compact holomorphic self-map f on a Lie ball D, thereby generalising Wolff’s theorem in [29] for the open unit disc in C. Indeed, these horospheres are affinely homeomorphic to D, but we show in Proposition 6.8 that the intersection of their closures is either a point or a one-dimensional disc in the boundary of D, in contrast to the case of Hilbert balls where the intersection is always a single boundary point. Consequently, we prove in Theorem 6.17 that every limit function of the iterates (f n ) is either constant with value in the boundary of D or takes values in a one-dimensional disc of the boundary. We show for instance, that not all Möbius transformations on a Lie ball D satisfy the Denjoy–Wolff-type theorem and determine, in Theorem 5.1, which ones do. Actually, the limit function of the iterates of a Möbius transformation can take values at every point of a one-dimensional disc in the boundary of D. More generally, given a fixed-point free holomorphic self-map f on D with one convergent orbit, we show in Proposition 6.2 that either all limit functions of (f n ) are the same constant function with value in the boundary of D or the image of each limit function is contained in a unique one-dimensional disc on the boundary. Besides Möbius transformations, we give some examples on the three-dimensional Lie ball. We make essential use of the Jordan algebraic structures of a spin factor to derive iteration results for a Lie ball. Every other infinite dimensional irreducible finite-rank bounded symmetric domain is realisable as the open unit ball of the space L(Cp , H) of linear operators from Cp to an infinite dimensional Hilbert space H. The Jordan structure of L(Cp , H) is markedly different and one needs different techniques to treat iteration for these domains. Let Z be a Banach space. The closure of a set E in Z is always denoted by E. Let U be the open unit ball of Z. We denote by

116

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

  ∂U = z ∈ Z : z = 1 the boundary of D. A subset B of U is said to be strictly contained in U , in symbols B  U , if inf{z − p : z ∈ B, p ∈ Z\U } > 0. n-times    n Given a map f : U −→ U , we denote by f = f ◦ · · · ◦ f the n-th iterate of f , for n = 1, 2, . . . . We equip the space C(U, U ) of continuous maps f : U −→ U with the topology of locally uniform convergence so that a sequence (fn ) converges to f in this topology if, and only if, it converges uniformly on any open ball B strictly contained in U . Let H(U, U ) be the subspace of C(U, U ) consisting of holomorphic maps f : U −→ Z with f (U ) ⊂ U . Then H(U, U ) is closed in C(U, U ). Given a sequence (fn ) in H(U, U ), a function h : U −→ U is called a limit function of (fn ) if there is a subsequence (fnk ) of (fn ) converging to h locally uniformly. A sequence in H(U, U ) may not have any limit function. In fact, it has been observed in [21, Example 3.2] that there is no infinite dimensional domain which is taut in the sense of Wu [30]. We call a sequence (fn ) in H(U, U ) normal if every subsequence of (fn ) contains a convergent subsequence in H(U, U ). This notion of normality is a natural extension of the finite dimensional one (cf. [1]). Remark 1.1. We note that, if a normal sequence (fn ) in H(U, U ) has a unique limit function h, then (fn ) converges locally uniformly to h. It has been shown in [21, Example 3.1] that there exists a sequence (ϕn ) of biholomorphic maps on the open unit ball of the Hilbert space 2 , which has no convergent subsequence. For separable Banach spaces, however, it follows directly from [4, Theorem 4.4] that we have the following weaker normality result. Lemma 1.2. Let U be the open unit ball of a separable reflexive Banach space V . Then any sequence (fn ) in H(U, U ) admits a subsequence (fnk ) which converges pointwise to a holomorphic map h : U −→ V , with respect to the weak topology, that is, (fnk (x)) converges weakly to h(x) for all x ∈ U . Definition 1.3. Given a sequence (fn ) in H(U, U ), a function h ∈ H(U, U ) is called a weak limit function of (fn ) if there is a subsequence (fnk ) such that (fnk (x)) converges weakly to h(x) for all x ∈ U . Remark 1.4. If the sequence (fn ) in Lemma 1.2 has a unique weak limit function h, then (fn ) converges pointwise to h in the weak topology. Indeed, for each x ∈ U and  in the dual V ∗ , the complex sequence ( ◦ fn (x)) is bounded. Given a limit point α = limk  ◦ fnk (x) of the complex sequence, there is a subsequence (fmk ) of (fnk ) converging pointwise to h weakly. It follows that α = (h(x)) and hence ( ◦ fn (x)) must converge to its unique limit point (h(x)). Hence (fn ) converges pointwise to h in the weak topology.

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

117

Throughout the paper, D will always denote a Lie ball. Given a holomorphic self-map f on D without fixed-point, we study the limit functions of its iterates (f n ) if they exist, which is the case if D is finite dimensional or f is a compact map, as defined in Section 4. In fact, in both cases, the sequence (f n ) is normal [7, Lemma 1]. 2. Spin factors A Lie ball D is the open unit ball of a spin factor V . Finite dimensional Lie balls are the type IV Cartan classical domains. Holomorphic iteration on D is closely related to the underlying structure of the spin factor V . For this reason and for completeness, we begin by deriving some properties of V for later use. The spin factors form an important class of JB*-triples. To facilitate our discussion, we first introduce some basic properties of JB*-triples and refer to [6,26] for more details and undefined terminology. The seminal work of Kaup [19] shows that a complex Banach space Z is a JB*-triple if, and only if, its open unit ball U is a symmetric domain, this is equivalent to the condition that Z admits a continuous triple product {·, ·, ·} : Z 3 −→ Z, called a Jordan triple product, which is symmetric and linear in the outer variables, but conjugate linear in the middle variable, and satisfies (i) {x, y, {a, b, c}} = {{x, y, a}, b, c} − {a, {y, x, b}, c} + {a, b, {x, y, c}}; (ii)  exp(it(a  a)) = 1 for all t ∈ R, where a  a : Z → Z is defined by (a  a)(·) = {a, a, ·}; (iii) a  a has non-negative spectrum; (iv) a  a = a2 for a, b, c, x, y ∈ Z. Let Z be a JB*-triple with open unit ball U . Given b, c ∈ Z, the box operator b  c : Z −→ Z and the Bergmann operator B(b, c) : Z −→ Z are continuous linear maps defined by (b  c)(x) = {b, c, x}

  B(b, c)(x) = x − 2(b  c)(x) + b, {c, x, c}, b

(x ∈ Z).

We always have a  b ≤ ab for a, b ∈ Z. Let L(Z) be the Banach algebra of bounded linear self-maps on Z and let 1 ∈ L(Z) be the identity operator on Z. If b  c < 1, then the inverses (1 ± b  c)−1 exist in L(Z). For a ∈ U , the Möbius transformation ga : U −→ U , induced by a, is given by

118

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

ga (z) = a + B(a, a)1/2 (1 + z  a)−1 (z)

(z ∈ U )

where the Bergmann operator B(a, a) has positive spectrum and   B(a, a)−1/2  =

1 1 − a2

(cf. [6, Proposition 3.2.13]). Let a ∈ U and ga : U −→ U be the induced Möbius transformation. Then ga has a continuous extension to the closure U , which will still be denoted by ga , such that ga (∂U ) = ∂U . If (ak ) is a sequence in U converging to some point ξ in the boundary ∂U , then it has been shown in [20] that the locally uniform limit limk gak exists, which will be denoted by gξ . We have gξ (y) = limk gak (y) = 1 for all y ∈ U . The Möbius transformations provide a useful description of the Kobayashi distance in U . For x, y ∈ U , the Kobayashi distance κ(x, y) can be expressed as   κ(x, y) = tanh−1 g−y (x) = κ(y, x). By the Schwarz–Pick lemma which states that  

  g−f (w) f (z)  ≤ g−w (z)

(z, w ∈ U )

(2.1)

(cf. [6, Lemma 3.2.15]), a holomorphic map f : U −→ U is κ-nonexpansive, that is,

κ f (x), f (y) ≤ κ(x, y)

(x, y ∈ U ).

We will denote by D = {z ∈ C : |z| < 1} the open unit disc in the complex plane. For each α ∈ D, the Möbius transformation induced by α is denoted by og α : D −→ D which has the form og α (z) =

α+z 1 + αz

(z ∈ D).

Lemma 2.1. Given two points a and b in the open unit ball U of a JB*-triple, we have  

  g−a (x) ≤ og α g−b (x)

(x ∈ U )

where α = g−a (b). Proof. Let x ∈ U and let α = g−a (b). Then  



g−a (x) = tanh κ(x, a) ≤ tanh κ(x, b) + κ(b, a)  

  = tanh tanh−1 g−b (x) + tanh−1 g−a (b) =



 g−b (x) + g−a (b) = og α g−b (x) . 1 + g−b (x)g−a (b)

2

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

119

The following formula has been proved in [24, Proposition 3.1] (see also [6, Lemma 3.2.17]). Lemma 2.2. For any two points a and b in the open unit ball of a JB*-triple, we have   1 = B(a, a)−1/2 B(a, b)B(b, b)−1/2 . 1 − g−b (a)2 A nonzero element e in a JB*-triple Z is called a tripotent if {e, e, e} = e in which case we have e = 1. Two tripotents e and c are said to be triple orthogonal (to each other) if e  c = 0, which implies c  e = 0 and also   αe + βc = sup |α|, |β|

(α, β ∈ C)

(cf. [6, p. 38; 183]). A tripotent e ∈ Z induces a Peirce decomposition Z = Z0 (e) ⊕ Z1 (e) ⊕ Z2 (e) where each Zj (e), called the Peirce j-space, is an eigenspace  Zj (e) =

z ∈ Z : (e  e)(z) =

j z 2

(j = 0, 1, 2)

of the operator e  e, and is the range of the contractive projection Pj (e) : Z −→ Z given by P0 (e) = B(e, e);



P1 (e) = 4 e  e − (e  e)2 ;

P2 (e) = 2(e  e)2 − e  e.

We call Pj (e) the Peirce j-projection and refer to [6, p. 32] for more detail. A tripotent e is called minimal if Z2 (e) = Ce, and is called maximal if Z0 (e) = {0}. In a JB*-triple Z, the extreme points of the closed unit ball U are exactly the maximal tripotents [6, p. 186]. A spin factor is a JB*-triple V equipped with a conjugation ∗ : V −→ V , which is a conjugate linear isometry of period 2, and a complete inner product ·,· satisfying

 a∗ , b∗ = b, a

and {a, b, c} =



1 a, b c + c, b a − a, c∗ b∗ 2

(2.2)

for all a, b, c ∈ V . Let V be a spin factor in the sequel and let  · h be the inner product norm of V . The spin factor norm  ·  satisfies    

 a3 = {a, a, a} =  a, a a − a, a∗ a∗ /2

120

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

where | a, a∗ | ≤ ah a∗ h = a2h . Hence we have 1 a2h a ≤ a2h a − 2

  1  3 a, a∗ a∗  ≤ a3 ≤ a2h a 2 2

and ah ≤

√

2a ≤

√ 3ah

(a ∈ V ).

(2.3)

We note that a spin factor V also carries another JB*-triple structure induced by its underlying Hilbert space structure, with the Jordan triple product {a, b, c}h =

1 a, b c + c, b a 2

(a, b, c ∈ V ).

These two JB*-triple structures are non-isomorphic although (2.3) shows that the two norms  ·  and  · h of V are equivalent. We denote by Vh the Hilbert space (V, ·,· ) equipped with the triple product {·, ·, ·}h . The weak topologies of V and Vh are equivalent. In particular, a sequence (vn ) weakly converges to some v in V if, and only if, v, x = limn vn , x for all x ∈ V . The two Banach spaces V and Vh have very different geometry. Let D and Dh be the open unit balls of V and Vh , respectively. By (2.3) and a later observation, we have √ Dh ⊂ D ⊂ 2Dh . In contrast to the Hilbert space Vh , the spin factor V is not strictly convex since the minimal tripotents of V have unit norm, but are not extreme points of the closed unit ball D. We include a proof of the following lemma for completeness. Lemma 2.3. Let V be a spin factor and Vh the underlying Hilbert space. Then (i) the tripotents of V are either minimal or maximal; (ii) the minimal tripotents of V are exactly the extreme points v of the closed unit ball Dh of Vh , satisfying v, v ∗ = 0. Also, we have v  v ∗ = 0 if v is a minimal tripotent. Proof. (i) Let a ∈ V be a tripotent. Then we have a = {a, a, a} = a, a a −

 1 a, a∗ a∗ . 2

If a, a∗ = 0, then a2h = a, a = 1 and {a, V, a} = Ca. Hence a is a minimal tripotent of V and an extreme point of Dh . If a, a∗ = 0, then a∗ = λa where    2( a, a − 1)   = 1. |λ| =  a, a∗ 

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

Let e =

√

λa. Then we have a = e∗ =

√

121

¯ and λe



¯ ∗= λa



¯ = λλa

√

λa = e.

If v ∈ Z0 (a), then {a, a, v} = 0 and

 0 = 2{e, e, v} = e, e v + v, e e − e, v ∗ e∗ = e, e v implies v = 0. Hence a is a maximal tripotent of V . (ii) We have already shown that the minimal tripotents of V are extreme points v of Dh satisfying v, v ∗ = 0. Conversely, given an extreme point v ∈ Dh with v, v ∗ = 0, we have v, v = 1 and therefore {v, v, v} = v, v v −

1 ∗ ∗ v, v v = v. 2

Hence v is a tripotent of V and is minimal since {v, V, v} = C v. Finally, given a minimal tripotent v and any x ∈ V , we have  1 ∗ 





v, v x + x, v ∗ v − v, x∗ v = 0. v, v ∗ , x = 2



2

We note that two triple orthogonal tripotents e and u in a spin factor V are also orthogonal in the Hilbert space Vh since e, u = {e, e, e}, u = e, {e, e, u} = 0. Each v ∈ V has a spectral representation v = αe + βu

(α ≥ β ≥ 0)

where e and u are triple orthogonal minimal tripotents in V , and v = α (cf. [9, Proposition 3.6] and [6, Theorem 1.2.34]). This representation is unique if α > β. In the above decomposition, u is a complex scalar multiple of e∗ . Indeed, {e, e, u} = 0 implies

 e, e u − e, u∗ e∗ = 0 and u = e, u∗ e∗ . We can therefore write the spectral representation of v in the form v = α1 e + α2 e∗



α2 ∈ C, v = α1 ≥ |α2 |

(2.4)

and the representation is unique if α1 > |α2 |. We note that α2 can be 0 and v2h = α12 + |α2 |2 ≥ α12 = v2 . This implies Dh ⊂ D. We can express the spin factor norm in terms of the inner product. Given v = α1 e + α2 e∗ in (2.4), we have v, v = α12 + |α2 |2 and v, v ∗ = 2α1 α2 . This gives  v, v +

 2 v, v 2 −  v, v ∗  = 2α12 = 2v2 .

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

122

The Lie ball D can therefore be written as    2

1 D = v ∈ V : v, v + v, v 2 −  v, v ∗  < 1 . 2

(2.5)

Since | v, v ∗ | ≤ v, v , one can also represent D in the form  D=

2  1  1  v ∈ V : 1 − v, v +  v, v ∗  > 0,  v, v ∗  < 1 . 2 2

Example 2.4. In some literature, the factor 1/2 is not included in the defining condition (2.2) for a spin factor (e.g. [12,27]). If one adopts this definition, some straightforward scaling would have to be made to our results above. For instance, the fraction 1/2 should be dropped from the description of the Lie ball in (2.5). Let n > 2 and equip Cn with the standard inner product and involution z = (z1 , . . . , zn ) ∈ Cn → z = (z 1 , . . . , z n ) ∈ Cn . If we define the triple product without 1/2: {z, z, z} = 2 z, z z − z, z z then Cn is a spin factor in this alternative definition and the open unit ball D has the form  2     D = z ∈ Cn : 1 − 2 z, z +  z, z  > 0,  z, z  < 1 which is another common representation of a type IV Cartan domain (cf. [16,22]). According to (2.4), a maximal tripotent v has the form v = e + αe∗ where e is a minimal tripotent and |α| = 1. It follows that v ∗ = αv and vh = note that the above representation of a maximal tripotent is not unique.

√

2. We

Lemma 2.5. Let a ∈ V \{0} satisfy a + λa∗ = 0 for some |λ| = 1. Then a/a is a maximal tripotent. Proof. Let a have the spectral decomposition a = α1 e + α2 e∗ as in (2.4). Then we have 0 = a + λa∗ = (α1 + α2 λ)e + (α2 + α1 λ)e∗ which implies α2 = −α1 λ and a = α1 (e − λe∗ ) where e − λe∗ is a maximal tripotent and α1 = a. 2 Lemma 2.6. Let v ∈ V be a maximal tripotent. Then v  v : V −→ V is the identity map.

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

123

Proof. As noted earlier, we have v ∗ = λv for some |λ| = 1. For each a ∈ V , we have  1 1 1 v, v a + a, v v − v, a∗ v ∗ 2 2 2 λ ∗ 1 a, v v = a. = a + a, v v − 2 2 2

v  v(a) = {v, v, a} =

Using the spectral decomposition of v = α1 e +α2 e∗ , we have a useful expression of the Bergmann operator B(v, v) for later computation. First, the triple orthogonal minimal tripotents e and e∗ give rise to a joint Peirce decomposition of V : 

V =

Vij

0≤i≤j≤2

where each Peirce space Vij is the range of a joint Peirce projection Pij : V −→ V . In fact, we have

P00 = P0 (e)P0 e∗ ; P11 = P2 (e);

P12





P01 = P0 e∗ P1 (e); P02 = P0 (e)P1 e∗



= P1 (e)P1 e∗ ; P22 = P2 e∗

(cf. [6, p. 38]). For a minimal tripotent e ∈ V , the Peirce projections are given by

 P0 (e) = ·, e∗ e∗ ;

P2 (e) = ·, e e 

P1 (e) = I − ·, e e − ·, e∗ e∗ = P1 e∗

and they are self-adjoint operators on the Hilbert space Vh . Hence for any z ∈ V , we have      z, e  = P2 (e)z  ≤ z in the spin norm. We note that P2 (e∗ ) = P0 (e) and

P1 (e)v

∗



= P1 (e) v ∗

(v ∈ V ).

(2.6)

Since e, e∗ = 0, a simple computation gives P00 = P01 = P02 = 0 and P12 = P1 (e). Hence the Bergmann operator B(v, v) for v = α1 e + α2 e∗ has the simple form B(v, v) =





1 − |αi |2 1 − |αj |2 Pij 1≤i≤j≤2

2



= 1 − |α1 |2 P2 (e) + 1 − |α1 |2 1 − |α2 |2 P1 (e)

2 + 1 − |α2 |2 P0 (e). In particular, we have

(2.7)

124

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154





B(v, v)1/2 (v) = 1 − |α1 |2 α1 e + 1 − |α2 |2 α2 e∗ B(v, v)−1/2 (v) =

(2.8)

∗

α1 e α2 e + . 2 1 − |α1 | 1 − |α2 |2

(2.9)

3. Boundary of a Lie ball Holomorphic iteration on a Lie ball is closely related to its boundary structure. To pave the way for the study of iteration, we describe in this section the boundary structure of a Lie ball. Let D be the open unit ball of a spin factor V and let T = {α ∈ C : |α| = 1}. By the previous discussion, we first observe that the boundary ∂D = {v ∈ V : v = 1} of D has the following description:   

∂D = e + αe∗ : e is a minimal tripotent, |α| ≤ 1 = e + De∗ e

where the union runs through the minimal tripotents e in V and the set of maximal  tripotents is the union e (e + Te∗ ). In fact, each e + De∗ is a boundary component of D, as defined below. The concept of a boundary component of a convex domain U in a Banach space Z has been introduced and studied in [20]. A subset C ⊂ U is called a (holomorphic) boundary component of U if the following conditions are satisfied: (i) C = ∅; (ii) for each holomorphic map f : D −→ Z with f (D) ⊂ U , either f (D) ⊂ C or f (D) ⊂ U \C; (iii) C is minimal with respect to (i) and (ii). The interior U is the unique open boundary component, all others are contained in the boundary ∂U (cf. [20]). For each a ∈ U , we denote by Ka the boundary component containing a. The following lemma has been proved in [20, Proposition 4.3]. Lemma 3.1. Let U be the open unit ball of a JB*-triple Z. Given a tripotent e ∈ Z, we have Ke = e + Z0 (e) ∩ U ⊂ ∂U . For each z ∈ Ke , we have Ke = lima→z ga (U ), where ga is the Möbius transformation induced by a ∈ U and the limit exists in the locally uniform topology. For a Lie ball D in a spin factor V , we have a very simple description of the boundary components of D. We first note that Ke = {e} if e is a maximal tripotent of V . If e is a minimal tripotent, then P0 (e) = ·, e∗ e∗ gives V0 (e) = Ce∗ and hence Ke = e + De∗ . Together with D, these are all the boundary components of D. Two boundary components of D are either equal or disjoint. In fact, the closures of two distinct boundary components in ∂D can meet at only one point.

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

125

Lemma 3.2. Let C and K be two boundary components in ∂D of a Lie ball D. If the intersection C ∩ K of the closures C and K contain two distinct points, then C = K. Proof. By the above remark, we may assume C = e + De∗ and K = u + Du∗ for some minimal tripotents e and u. Let x, y ∈ C ∩ K be two distinct points. Then x = e + βe∗ = u + γu∗ and y = e + β  e∗ = u + γ  u∗ , where β, β  are distinct complex numbers in D. If |β| < 1, then |γ| < 1 since 1 + |β|2 = x2h = 1 + |γ|2 . In this case, we have x ∈ C ∩ K which gives C = K. Likewise |β  | < 1 implies C = K. Let β, β, γ, γ  ∈ T. By convexity of C and K, we have  e+

   1 1 1 1 β + β  e∗ = u + γ + γ  u∗ ∈ C ∩ K 2 2 2 2

since β = β  and γ = γ  . Hence C = K.

2

The relevance of the boundary components to iteration can be seen from the following simple fact. Lemma 3.3. Let h : D −→ D be a holomorphic map. Then its image h(D) is contained entirely in one boundary component of D. Proof. Let z ∈ D and define a holomorphic map ψz : D −→ D by ψz (λ) = h(λz)

(λ ∈ D).

Then ψz (D) is contained in the boundary component Kψz (0) = Kh(0) . Since z ∈ D was arbitrary, this shows h(λz) ∈ Kh(0) for all λ ∈ D and z ∈ D, that is, h(D) ⊂ Kh(0) . 2 Remark 3.4. We see from the above lemma that if two holomorphic maps h, g : D −→ D have a common value h(z0 ) = g(z0 ) at some z0 ∈ D, then both of their images are contained in the same boundary component of D. 4. Invariant horospheres For iteration of a self-map f , a natural and fruitful approach is to study its invariant domains. In this section, we generalise Wolff’s theorem on invariant domains in the complex disc D [29] to Lie balls. To include infinite dimensional domains, we consider compact self-maps. A self-map ψ on a domain U in a complex Banach space is called compact if the closure of ψ(U ) is compact. In finite dimensions, all continuous self-maps on bounded domains are compact. Let D be a Lie ball in a spin factor V and let f : D −→ D be a compact holomorphic map without fixed point. Choose an increasing sequence (αk ) in (0, 1) with limit 1. Then αk f maps D strictly inside itself and by the fixed-point theorem of Earle and

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

126

Hamilton [11], αk f (zk ) = zk for some zk ∈ D. Note that zk = 0. Since f (D) is relatively compact, we may assume, by choosing a subsequence if necessary, that (zk ) converges to a point ξ ∈ D. Since f has no fixed point in D, the point ξ must lie in the boundary ∂D = D\D of D. By (2.4), each zk ∈ D has a spectral decomposition zk = α1k dk + α2k d∗k

(4.1)

where dk is a minimal tripotent and α1k = zk  → 1 as k → ∞. Since dk , d∗k = 0, we have

 zk , zk∗ = 2α1k α2k

which implies that (α2k ) converges to α = ξ, ξ ∗ /2 and |α| ≤ 1. By weak compactness of D and by choosing a subsequence, we may assume (dk ) weakly converges to some d ∈ D. It follows that (d∗k ) weakly converges to d∗ . Since (zk ) norm converges to ξ, we have ξ = d + αd∗

(4.2)

and d = 0. Observe that 2

+ |α2k |2 = 1 + |α|2 ξ, ξ = lim zk , zk = lim α1k k

k

where





 ξ, ξ = d + αd∗ , d + αd∗ = d, d + |α|2 d∗ , d∗ + 2 Re α d∗ , d . Hence we have

 1 + |α|2 = 1 + |α|2 d, d + 2 Re α d∗ , d .

(4.3)

Since dk , zk dk = α1k dk , taking limit yields

 d = d, ξ d = d, d d + α d, d∗ d which gives 

 1 = d, d + α d, d∗ = d, d + α d∗ , d .

(4.4)

From (4.3) and (4.4), we obtain d, d = 1 or |α| = 1. If d, d = 1, then the sequence (dk ) actually norm converges to d in the Hilbert space Vh , and hence norm converges to d in V as well, in which case d is a minimal tripotent.

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

127

If |α| = 1, then ξ = d + αd∗ = αξ ∗ and ξ is a maximal tripotent by Lemma 2.5. Hence ξ = v + αv ∗ for some minimal tripotent v ∈ ∂D. We retain the notation in (4.1) in the sequel. Given c ∈ V and r > 0, we denote by D(c, r) the open ball of radius r, centred at c. As before, ga denotes the Möbius transformation induced by a ∈ D. Fix a point y ∈ D and let rk = g−zk (y) where rk < 1 and limk rk = 1. We define a closed Kobayashi ball centred at zk to be the set   Dk [y] = z ∈ D : κ(z, zk ) ≤ tanh−1 rk which equals       z ∈ D : g−zk (z) ≤ g−zk (y) = gzk (w) : w ≤ rk .



Analogous to [24, Proposition 2.3], for x ∈ D and 0 < r < 1, the formula

gz (rx) = 1 − r2 B(rz, rz)−1/2 (z) + rB(z, z)1/2 B(rz, rz)−1/2 grz (x)

(4.5)

Dk [y] = ck (y) + rk B(zk , zk )1/2 B(rk zk , rk zk )−1/2 (D)

(4.6)

gives

where ck (y) = (1 − rk2 )B(rk zk , rk zk )−1/2 (zk ) and by (2.9), we have  ck (y) =

   1 − rk2 1 − rk2 α1k dk + α2k d∗k . 2 1 − rk2 α1k 1 − rk2 |α2k |2

(4.7)

Since the two bracketed non-negative sequences in (4.7) are bounded above by 1, we can pick t, s ∈ [0, 1] such that t2 = lim sup k→∞

1 − rk2 2 1 − rk2 α1k

and s2 = lim sup k→∞

1 − rk2 . 1 − rk2 |α2k |2

(4.8)

Choosing subsequences if necessary, we may replace the upper limits above by limits. It follows that   2 2 1 − rk2 1 − α1k rk2 (1 − α1k ) = 1 − t2 lim lim = lim 1 − 2 α2 = k→∞ 2 2 k→∞ 1 − rk k→∞ 1 − rk2 α1k 1 − rk2 α1k 1k

(4.9)

1 − |α2k |2 2 2 |α |2 = 1 − s . k→∞ 1 − rk 2k

(4.10)

and lim

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

128

We note that both s and t depend on the chosen point y ∈ D. Following [2], we define a closed horosphere S(ξ, y) at ξ as the limit of a sequence of closed Kobayashi balls in the following sense:   S(ξ, y) = x ∈ D : x = lim xk and xk ∈ Dk [y] . k

Observe that zk ∈ Dk [y] and ξ, y ∈ S(ξ, y). The interior of S(ξ, y) is called an open horosphere (or horosphere) at ξ and is denoted by S0 (ξ, y). We will show that S0 (ξ, y) is nonempty and is a convex domain in D. Lemma 4.1. Let y ∈ D and let s, t be given in (4.8). Then we have 0 ≤ s ≤ t and t ∈ (0, 1). Moreover, 1 + y 1 − y ≤ t2 ≤ . 2 2 These bounds are best possible. Proof. Evidently, we have 0 ≤ s ≤ t since α1k ≥ |α2k | in (4.8). For y = 0, we have rk = g−zk (0) = zk  = α1k and t2 = 1/2. Hence the bounds are best possible. Next we find a strictly positive lower bound for t. We observe from Lemma 2.2 that  2 1 − rk2 = 1 − g−zk (y) =

1 B(y, y)−1/2 B(y, z

−1/2  k )B(zk , zk )

which gives 1 − rk2 2 α2 k→∞ 1 − rk 1k

t2 = lim = lim

k→∞

≥ lim

1 2 + (1 − α2 )B(y, y)−1/2 B(y, z )B(z , z )−1/2  α1k k k k 1k

k→∞ α2 1k

=

1 + B(y, y)−1/2 B(y, zk )

1 >0 1 + B(y, y)−1/2 B(y, ξ)

where α1k = zk  and B(zk , zk )−1/2  = (1 − zk 2 )−1 . Since     

B(y, ξ) = I − 2y  ξ + y, {ξ, ·, ξ}, y  ≤ 1 + 2y + y2 = 1 + y 2 , we have 2   B(y, y)−1/2 B(y, ξ) ≤ (1 + y) = 1 + y 2 1 − y 1 − y

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

129

and t2 ≥

1 − y > 0. 2

Now we show t2 ≤ (1 + y)/2. We need only consider y = 0. Making use of (4.5), (2.7) and (2.9), one finds z ∈ D such that   y g−zk (y) = g−zk y y



−1/2 = − 1 − y2 B yzk , yzk (zk )

−1/2 + yB(zk , zk )1/2 B yzk , yzk (z) (1 − y2 )α1k dk (1 − y2 )α2k d∗k − 2 2 1 − y α1k 1 − y2 |α2k |2  2 2 )(1 − |α |2 ) (1 − α1k y(1 − α1k )P2 (dk )z 2k P1 (dk )z + + y 2 2 2 2 1 − y α1k (1 − y α1k )(1 − y2 |α2k |2 )

= −

+

y(1 − |α2k |2 )P0 (dk )z . 1 − y2 |α2k |2

Hence we have 2   

 (1 − y2 )α1k y(1 − α1k ) rk = g−zk (y) ≥ P2 (dk ) g−zk (y)  ≥ − 2 2 1 − y2 α1k 1 − y2 α1k

which gives 1 − rk ≤

(1 − α1k )(1 + α1k y2 + (1 + α1k )y) . 2 1 − y2 α1k

It follows that 2 (1 + rk )(1 − α1k 1 − rk2 )(1 + α1k y2 + (1 + α1k )y) ≤ 2 2 )(1 + α )(1 − y2 α2 ) 1 − rk2 α1k (1 − rk2 α1k 1k 1k

and 2

1 − rk2 2 (1 + y) ≤ 1 − t . 2 α2 k→∞ 1 − rk 1 − y2 1k

t2 = lim Therefore we have

t2 ≤

1 + y < 1. 2

2

We now show that the invariance of the horospheres is a simple consequence of the Schwarz–Pick lemma.

130

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

Lemma 4.2. Let f : D −→ D be a compact holomorphic map, without fixed point. Then there is a sequence (zk ) in D converging to a boundary point ξ ∈ ∂D such that, for each y ∈ D, we have f (S(ξ, y) ∩ D) ⊂ S(ξ, y) ∩ D. Proof. Let (zk ) be the sequence constructed in (4.1), converging to a boundary point ξ ∈ ∂D. Write fk = αk f in this construction, which has fixed point zk . Now let x ∈ S(ξ, y) ∩D with x = limk xk and xk ∈ Dk [y]. By the Schwarz–Pick lemma (2.1), we have    

 

  g−z fk (xk )  = g−f (z ) fk (xk )  ≤ g−z (xk ) ≤ g−z (y) k k k k k and hence fk (xk ) ∈ Dk [y]. It follows that f (x) = limk f (xk ) = limk fk (xk ) ∈ S(ξ, y) ∩ D. 2 It follows from the above lemma that, given a limit function h of the iterates (f n ) of a fixed-point free compact holomorphic self-map f on D, we have h(y) ∈ S(ξ, y) for each y ∈ D. Therefore we need a useful description of the closed horosphere S(ξ, y) to deduce iteration results. From the previous discussion, we have ξ = d + αd∗ , where d ∈ ∂D is a minimal tripotent if |α| < 1 in which case, d is the norm limit of the sequence (dk ) of tripotents. If |α| = 1, then ξ = v + αv ∗ is a maximal tripotent and v is a minimal tripotent. Each x ∈ S(ξ, y) has the form x = limk xk with xk ∈ Dk [y], where by (4.6), we have

xk = ck (y) + rk B(zk , zk )1/2 B(rk zk , rk zk )−1/2 xk for some xk ∈ D. To express limk xk explicitly, it suffices to find a subsequential limit limnk xnk . For this purpose and to avoid cumbersome notation in what follows, we often replace a sequence by a convergent subsequence without changing notation when appropriate. Let y ∈ D. We first show that, by choosing a subsequence, we may assume that the sequence (ck (y)) in (4.7) norm converges to c(y) := t2 d + αs2 d∗ where s and t are defined in (4.8), both depend on y. This is obvious if |α| < 1, by norm convergence of (dk ). Let |α| = 1. We have the weak limit c(y) = (weak) lim ck (y) = t2 d + αs2 d∗ . k

For norm convergence, we make use of the spectral representation ξ = v + αv ∗ . Let

 zk = P2 (v)zk + P0 (v)zk = zk , v v + zk , v ∗ v ∗ . Then lim zk = P2 (v)ξ + P0 (v)ξ = ξ. k

Also, limk zk , v = ξ, v = 1 and limk zk , v ∗ = ξ, v ∗ = α.

(4.11)

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

131

We can write ck (y) = gzk (uk ) for some uk ∈ D where g−zk (ck (y)) ≤ rk since ck (y) ∈ gzk (D(0, rk )). By Lemma 2.1, we have 

 



uk  = g−zk ck (y)  ≤ og ωk g−zk ck (y)  ≤ og ωk (rk ) where ωk = g−zk (zk ) < 1. Let ρk = og ωk (rk ) =

rk + g−zk (zk ) . 1 + rk g−zk (zk )

Then 0 < ρk < 1 and ρk only depends on zk , zk and rk . Moreover, (ρk ) converges to 1 which is the only limit point of the sequence. Using (4.5) as before, we have ck (y) = gzk (ρk uk /ρk )



−1/2 



1/2

−1/2 

zk + ρk B zk , zk B ρk zk , ρk zk uk = 1 − ρ2k B ρk zk , ρk zk for some uk ∈ D, where



−1/2 

1 − ρ2k B ρk zk , ρk zk zk     2

 1 − ρk 1 − ρ2k zk , v v + zk , v ∗ v ∗ . = 2 2 2 ∗ 2 1 − ρk | zk , v | 1 − ρk | zk , v |

(4.12)

We may assume the two bracketed non-negative sequences in (4.12) converge to ρ2 and η 2 , respectively, by picking subsequences if necessary. This gives



−1/2 

lim 1 − ρ2k B ρk zk , ρk zk zk = ρ2 v + η 2 αv ∗ .

k→∞

By (2.7), we have

1/2

−1/2 

B zk , zk B ρk zk , ρk zk uk    



(1 − | zk , v |2 )(1 − | zk , v ∗ |2 ) 1 − | zk , v |2 P P1 (v) uk (v) u + = 2 k 2 1 − ρk | zk , v |2 (1 − ρ2k | zk , v |2 )(1 − ρ2k | zk , v ∗ |2 )  

1 − | zk , v ∗ |2 P0 (v) uk + 2 ∗ 2 1 − ρk | zk , v | where P2 (v)(uk ) = uk , v v and P0 (v)(uk ) = uk , v ∗ v ∗ . Again, we may assume (uk ) weakly converges to some u ∈ D, by choosing a subsequence. Therefore

1/2

−1/2 

lim ρk B zk , zk B ρk zk , ρk zk uk k 









= 1 − ρ2 P2 (v) u + 1 − ρ2 1 − η 2 P1 (v) u + 1 − η 2 P0 (v) u

1/2 

u = B ρv + ηv ∗ , ρv + ηv ∗

132

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

and hence (ck (y)) norm converges to ρ2 v +αη 2 v ∗ +B(ρv +ηv ∗ , ρv +ηv ∗ )1/2 (u ). It follows that (ck (y)) norm converges to t2 d + αs2 d∗ as well in the case of |α| = 1. Remark 4.3. The above arguments also reveal that, if s = t for some y ∈ D, then the norm convergence of the two different linear combinations of dk and d∗k in (4.1) and (4.7) actually implies that (dk ) norm converges to d, and hence d is also a minimal tripotent in this case. We now give a useful description of the closed horosphere S(ξ, y), which also shows that each closed horosphere is affinely homeomorphic to the closed ball D. Theorem 4.4. Let f : D −→ D be a fixed-point free compact holomorphic map on a Lie ball D. Then there is a boundary point ξ = e + αe∗ ∈ ∂D where e ∈ ∂D is a minimal tripotent and |α| ≤ 1 such that, for each y ∈ D, the closed horosphere S(ξ, y) is a weakly compact convex set given by

1/2 S(ξ, y) = t2 e + αs2 e∗ + B te + se∗ , te + se∗ (D) where 1 − y ≤ 2t2 ≤ 1 + y and s ∈ [0, t]. If |α| < 1, then s = 0 and we have S(ξ, y) = t2 e + B(te, te)1/2 (D). Proof. Let ξ = d + αd∗ be the boundary point given in (4.2), where |α| ≤ 1. For each y ∈ D, an element x ∈ S(ξ, y) is a norm limit x = limk xk with xk = ck (y) + rk B(zk , zk )1/2 B(rk zk , rk zk )−1/2 (wk )

(4.13)

for some wk ∈ D. We have shown c(y) = lim ck (y) = t2 d + αs2 d∗ k

where s2 and t2 are the limits defined in (4.8), both depend on y. By Lemma 4.1, one has t ∈ (0, 1) and s ∈ [0, t], where 1 − y ≤ 2t2 ≤ 1 + y. Assume, by choosing a subsequence if necessary, that (wk ) weakly converges to some w ∈ D. By (2.7), we have B(zk , zk )1/2 B(rk zk , rk zk )−1/2 (wk )    2 2 )(1 − |α |2 ) (1 − α1k 1 − α1k 2k P (d )(w ) + = 2 k k 2 2 )(1 − r 2 |α |2 ) P1 (dk )(wk ) 1 − rk2 α1k (1 − rk2 α1k k 2k   1 − |α2k |2 P0 (dk )(wk ). + 1 − rk2 |α2k |2

(4.14)

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

133

We need to consider two cases: (i) d is a minimal tripotent; (ii) d is not a minimal tripotent. Given that d is a minimal tripotent, the sequence (dk ) norm converges to d. Hence we have, for any y ∈ D, lim rk B(zk , zk )1/2 B(rk zk , rk zk )−1/2 (wk ) k 





= 1 − t2 P2 (d)(w) + 1 − t2 1 − s2 P1 (d)(w) + 1 − s2 P0 (d)(w)

1/2 (w) = B td + sd∗ , td + sd∗ and therefore

1/2 (w). x = t2 d + αs2 d∗ + B td + sd∗ , td + sd∗ Conversely, given z ∈ D, we have

1/2 t2 d + αs2 d∗ + B td + sd∗ , td + sd∗ (z)

= lim ck (y) + rk B(zk , zk )1/2 B(rk zk , rk zk )−1/2 (z) ∈ S(ξ, y) k→∞

since ck (y) + rk B(zk , zk )1/2 B(rk zk , rk zk )−1/2 (z) ∈ Dk [y]. It follows that

1/2 S(ξ, y) = t2 d + αs2 d∗ + B td + sd∗ , td + sd∗ (D)

(4.15)

which is a weakly compact convex set, where B(td + sd∗ , td + sd∗ )1/2 : V → V is a linear homeomorphism. Further, if |α| < 1, then s = 0 and we just have S(ξ, y) = t2 d + B(td, td)1/2 (D). Now consider case (ii) when d is not a minimal tripotent. Then by Remark 4.3, we must have s = t for all y ∈ D. Hence |α| = 1, but we only know that (dk ) weakly converges to d in (4.14). Nevertheless, by boundedness and choosing a subsequence, we may assume the sequences wk , dk and wk , d∗k converge to, say μ ∈ D and θ ∈ D, respectively. Since s = t, the sequence in (4.14) weakly converges to





1 − t2 μd + w − μd − θd∗ + θd∗ = 1 − t2 w. It follows that, for each y ∈ D and x ∈ S(ξ, y) from (4.13), we have

x = t2 d + αt2 d∗ + 1 − t2 w = t2 ξ + B(tξ, tξ)1/2 (w)

1/2 = t2 v + αs2 v ∗ + B tv + sv ∗ , tv + sv ∗ (w) where ξ = v + αv ∗ . Conversely, given z ∈ D, we have

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

134

t2 ξ + B(tξ, tξ)1/2 (z)

= lim ck (y) + rk B(zk , zk )1/2 B(rk zk , rk zk )−1/2 (z) ∈ S(ξ, y). k→∞

This proves

1/2 S(ξ, y) = t2 v + αs2 v ∗ + B tv + sv ∗ , tv + sv ∗ (D)

(4.16)

in case (ii), which also shows that S(ξ, y) is a weakly compact convex set. To conclude, we relabel d by e in (4.15) when d is a minimal tripotent. If d is not a minimal tripotent, then we have case (ii) and one can relabel v by e in the representation ξ = v + αv ∗ and in (4.16) to complete the proof. 2 Corollary 4.5. Let f : D −→ D be a fixed-point free compact holomorphic map on a Lie ball D in a spin factor V . Let ξ = e + αe∗ be the boundary point in Theorem 4.4. Then for each y ∈ D, the horosphere S0 (ξ, y) is a convex invariant domain of f in D, given by

1/2 S0 (ξ, y) = t2 e + αs2 e∗ + B te + se∗ , te + se∗ (D) for some t ∈ (0, 1) and s ∈ [0, t]. The closure S0 (ξ, y) is the closed horosphere S(ξ, y) and S0 (ξ, y) is the interior of S(ξ, y) ∩ D. If |α| < 1, then s = 0 and S0 (ξ, y) = t2 e + B(te, te)1/2 (D). Proof. Let S(ξ, y) = t2 e + αs2 e∗ + B(te + se∗ , te + se∗ )1/2 (D) be the closed horosphere given in Theorem 4.4. Since the map F : V −→ V defined by

1/2 F = t2 e + αs2 e∗ + B te + se∗ , te + se∗ is an affine homeomorphism, F (D) is the interior of F (D) = S(ξ, y), that is, F (D) = S0 (ξ, y) which is a convex domain and we have S0 (ξ, y) = F (D) = S(ξ, y). Since t2 e + αs2 e∗ = F (0) ∈ D and F restricts to a holomorphic map F |D : D −→ D, Lemma 3.3 implies that F (D) ⊂ D and therefore F (D) is the interior of S(ξ, y) ∩ D. Now it follows readily from Lemma 4.2 that f (S0 (ξ, y)) ⊂ S(ξ, y) and hence S0 (ξ, y) is an invariant domain of f in D. 2 Given a fixed-point free compact holomorphic self-map f on a Lie ball D ⊂ V , we have seen that the horosphere S(ξ, y) is constructed from the Kobayashi balls of radii tanh−1 (rk ) = tanh−1 g−zk (y). Using different radii, one can construct a family of horospheres S(ξ, τ ) parameterised by positive real numbers τ > 0, which enables us to generalise Wolff’s theorem for D to a Lie ball D. Indeed, given τ > 0, we can find a sequence (sk ) in (0, 1) such that

τ 1 − zk 2 = 1 − s2k

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

135

from some k onwards, and as before, one can use the Kobayashi balls

Dk [τ ] = gzk D(0, sk ) to define the horosphere   S(ξ, τ ) = x ∈ D : x = lim xk and xk ∈ Dk [τ ] . k

Then we have ξ ∈ S(ξ, τ ) and f (S(ξ, τ ) ∩ D) ⊂ S(ξ, τ ) ∩ D, analogous to Lemma 4.2. For y ∈ D, we show below that S(ξ, y) = S(ξ, λ) where    1 − rk2 zk 2 1 − rk2 t2 1 − rk2 = λ := lim = lim >0 2 k 1 − zk 2 k 1 − zk 2 1 − rk zk 2 1 − t2 and t2 =

λ . 1+λ

Since |α2k | ≤ α1k = zk , we may assume, by choosing a subsequence, that converges to some σ ∈ [0, 1]. It follows that

1−zk 2 1−|α2k |2

1 − rk2 2 |α |2 k 1 − rk 2k     2 1 − rk 1 − zk 2 1 − |α2k |2 = lim k 1 − zk 2 1 − |α2k |2 1 − rk2 |α2k |2

= λσ 1 − s2

s2 = lim

and s2 =

σλ 1+σλ .

Therefore the description of S(ξ, y) in Theorem 4.4 can be rewritten as

S(ξ, y) = S(ξ, λ) = where λ =



With τ =



1/2 ασλ ∗ λ e+ e + B λ e + σ  e∗ , λ e + σ  e∗ (D) 1+λ 1 + σλ



λ  1+λ , σ = 1−s2k 1−zk 2 and

lim k

(4.17)

σλ 1+σλ

and σ = 0 if |α| < 1.

limk sk = 1, we deduce likewise that

τ 1 − s2k = 1 − s2k zk 2 1+τ

and

lim k

στ 1 − s2k . = 1 − s2k |α2k |2 1 + στ

As in (4.6) and (4.7), the Kobayashi ball Dk [τ ] has the form Dk [τ ] = ck (τ ) + sk B(zk , zk )1/2 B(sk zk , sk zk )−1/2 (D)

136

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

with  ck (τ ) =

   1 − s2k 1 − s2k α1k dk + α2k d∗k . 1 − s2k zk 2 1 − s2k |α2k |2

Now an analogous computation as in the proof of Theorem 4.4 leads to the following generalised version of Wolff’s theorem for Lie balls. Theorem 4.6. Let f : D −→ D be a fixed-point free compact holomorphic map on a Lie ball D. Then there is a boundary point ξ ∈ ∂D such that for each τ > 0, there is a horosphere S0 (ξ, τ ) ⊂ D, which is a convex invariant domain of f and satisfies f (S(ξ, τ ) ∩ D) ⊂ S(ξ, τ ) ∩ D where S(ξ, τ ) = S0 (ξ, τ ). Further, ξ = e + αe∗ for some minimal tripotent e ∈ ∂D with |α| ≤ 1 and

1/2 αστ ∗ τ e+ e + B τ  e + σ  e∗ , τ  e + σ  e∗ (D) 1+τ 1 + στ   τ στ for some σ ∈ [0, 1], where τ  = 1+τ and σ  = 1+στ . If |α| < 1, we have S0 (ξ, τ ) =

S0 (ξ, τ ) =



1/2 τ e + B τ  e, τ  e (D). 1+τ

We observe from (4.17) above that the constant s, which depends on y, is positive if and only if 1 − zk 2 >0 k→∞ 1 − |α2k |2

σ = lim

where σ does not depend on y. This also reveals that, if σ > 0, then for all y  ∈ D, we have sy > 0 in the horosphere S(ξ, y  ) = t2y e +αs2y e∗ +B(ty e +sy e∗ , ty e +sy e∗ )1/2 (D). We note that σ > 0 implies |α| = limk |α2k |2 = 1. Definition 4.7. Given a fixed-point free compact holomorphic self-map f a Lie ball D, we call the boundary point ξ = e +αe∗ = limk zk in Theorem 4.4, where e is a minimal tripo1−zk 2 tent and |α| ≤ 1, a Wolff point of f . It is called non-degenerate if lim supk 1−|α 2 > 0 in 2k | which case, |α| = 1 and by choosing a subsequence, we may assume σ = limk

1−zk 2 1−|α2k |2

> 0.

5. Iterates of Möbius transformations Before studying the iteration of an arbitrary holomorphic self-map on a Lie ball D, let us first consider, as an example, the special case of a Möbius transformation on D which will guide us to the general case. Let a ∈ D\{0}. We investigate in this section the dynamics of the Möbius transformation ga : D −→ D which is fixed-point free. Unlike the Hilbert ball, ga may fail the Denjoy–Wolff-type theorem even in finite dimensions. Let a ∈ D with spectral representation a = αe + βe∗ , where e is a minimal tripotent and α = a ≥ |β| ≥ 0. As before, og ω : D −→ D denotes the Möbius transformation on

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

137

the complex disc D induced by ω ∈ D. Given p, q ∈ D, we have







(e  e) e∗  e∗ pe + qe∗ = e∗  e∗ (e  e) pe + qe∗ = 0. It follows from (2.7) that

ga pe + qe∗



−1

pe + qe∗ = a + B(a, a)1/2 I + pe + qe∗  αe + βe∗



¯ ∗  e∗ −1 pe + qe∗ = a + B(a, a)1/2 I + pαe  e + q βe

¯ ∗  e∗ = a + B(a, a)1/2 I − pαe  e + q βe





¯ ∗  e∗ 2 − · · · pe + qe∗ + pαe  e + q βe





¯ ∗ + p3 α2 e + q 3 β¯2 e∗ − · · · = a + B(a, a)1/2 pe + qe∗ − p2 αe + q 2 βe





= a + B(a, a)1/2 1 − pα + p2 α2 − · · · pe + 1 − q β¯ + q 2 β¯2 − · · · qe∗   pe qe∗ 1/2 + = a + B(a, a) 1 + pα 1 + q β¯ = αe + βe∗ + =

(1 − |α|2 )pe (1 − |β|2 )qe∗ + 1 + pα 1 + q β¯

β+q ∗ α+p e+ e = og α (p)e + og β (q)e∗ . 1 + pα 1 + q β¯

Hence the iterates (gan (pe + qe∗ ))n converge to the point  ξ=

e+

β ∗ |β| e ∗

e + qe

if β = 0 if β = 0.

(5.1)

The celebrated Denjoy–Wolff theorem states that the iterates (f n ) of any fixed-point free holomorphic self-map f on D converge locally uniformly to a constant map taking value at the boundary of D. For a finite dimensional Lie ball, we now show that the Möbius transformation ga induced by an element a satisfies a Denjoy–Wolff-type theorem if, and only if, a has spectral decomposition αe + βe∗ with α, β = 0. This follows from the more general result below. Theorem 5.1. Let D be a Lie ball and let a ∈ D\{0} with spectral representation a = αe + βe∗ for some minimal tripotent e ∈ D. Let h : D −→ D be a limit function of the iterates (gan ) of the Möbius transformation ga . Then we have h(D) ⊂ ∂D and β ∗ (i) h(D) = {e + |β| e } if β = 0. (ii) h(D) = e + D e∗ if β = 0.

138

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

β ∗ Further, if D is in a separable spin factor, then (gan ) converges pointwise to e + |β| e in the weak topology, for β = 0. If D is finite dimensional and if β = 0, then (gan ) converges locally uniformly to h.

Proof. Let ξ be the boundary point given in (5.1). Then h(pe + qe∗ ) = limn gan (pe + qe∗ ) = ξ and therefore the image h(D) is contained in the boundary component Kξ . β ∗ If β = 0, then ξ = e + |β| e is a maximal tripotent and hence we have Kξ = {ξ} = h(D). If β = 0, then Kξ = e + D e∗ . For every p, q ∈ D, we have h(pe + qe∗ ) = e + qe∗ . It follows that h(D) = e + D e∗ . If D is in a separable spin factor, then (gan ) admits a weak limit function by Lemma 1.2. If β = 0, then the map h in (i) is the unique weak limit function and hence (gan ) converges pointwise to h in the weak topology, by Remark 1.4. If D is finite dimensional, then (gan ) is normal. Given β = 0, all limit functions of (gan ) coincide with the constant map h in (i), and hence (gan ) converges locally uniformly to h by Remark 1.1. 2 It follows from the above theorem that, for each nonzero element a = αe + βe∗ in a finite dimensional Lie ball D, the iterates (gan ) converge locally uniformly to a constant map with value e + βe∗ /|β| if, and only if, β = 0. In view of the fact that the boundary components of the boundary ∂B of a Hilbert ball B are the singletons {ζ} with ζ ∈ ∂B, if one reformulates the Denjoy–Wolff theorem for B as asserting that all limit functions of the iterates of a fixed-point free compact holomorphic self-map on B have values in a unique boundary component of ∂B, then one may regard Theorem 5.1 as a generalisation of the Denjoy–Wolff theorem for Möbius transformations. 6. Iteration of holomorphic maps We begin by extending the iteration result for Möbius transformations in Theorem 5.1 to other holomorphic self-maps on a Lie ball. In the following result, we note that the Hilbert ball Dh is not strictly contained in D. In fact, the intersection ∂Dh ∩∂D contains the set of minimal tripotents by Lemma 2.3. Proposition 6.1. Let f : D −→ D be a fixed-point free compact holomorphic map on a Lie ball D in a spin factor V . Let Dh be the open unit ball of the underlying Hilbert space Vh . If f (D) ⊂ Dh , then (f n ) converges locally uniformly to a constant map with value at a minimal tripotent e ∈ ∂D. Proof. Given f (D) ⊂ Dh , the restriction of f to Dh is a compact holomorphic self-map on Dh and by the Denjoy–Wolff-type theorem in [7], there is a unique minimal tripotent e ∈ ∂Dh such that the iterated sequence (f |nDh ) of the restriction f |Dh converges to the constant function g(·) = e on Dh , uniformly in the norm  · h , on an open  · h -ball

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

139

B  Dh . The same convergence also holds in the spin factor norm  · . The ball B contains a  · -open ball B  . It follows that (f n ) converges uniformly to g on B   D, that is, (f n ) converges locally uniformly to g on D. 2 Proposition 6.2. Let D be a Lie ball in a spin factor V and f : D −→ D a fixed-point free holomorphic map. If an orbit (f n (z0 )) converges for some point z0 ∈ D, then either (i) every limit function of (f n ) is the constant map ζ with value limn f n (z0 ) which is a maximal tripotent in ∂D, or (ii) every limit function h has image h(D) contained in e + De∗ for a unique minimal tripotent e ∈ ∂D. In case (i), (f n ) converges pointwise, in the weak topology, to ζ if V is separable. Without separability, (f n ) converges locally uniformly to ζ if (f n ) is normal. Proof. Given that (f n (z0 )) converges to some point v ∈ D, all limit functions have a common value v = limn f n (z0 ) at z0 and hence, by Remark 3.4, their images are contained in one single boundary component K of D. Since f has no fixed-point in D, we must have v ∈ ∂D and hence K ⊂ ∂D. It follows that either K = {e} for a maximal tripotent e or K = e + De∗ for some minimal tripotent e. In the latter case, we have h(D) ⊂ e + De∗ for every limit function h of the iterates (f n ) and clearly, e is unique. This gives (ii). In the case of K = {e}, we have v = e and all subsequences of (f n ) converge to the same constant function ζ with value e. This gives (i), in which case it follows from Remark 1.1 that (f n ) converges locally uniformly to ζ if (f n ) is normal. Without normality but assuming V is separable in case (i), the sequence (f n ) has a unique weak limit function ζ and by Remark 1.4, it converges pointwise to ζ in the weak topology. 2 Remark 6.3. We see from the previous example of Möbius transformations that both cases in Proposition 6.2 can occur. Moreover, one can have h(D) = e + D e∗ in the second case. We now consider a compact fixed-point free holomorphic map f : D −→ D, without any other assumption. In what follows, we fix such a map f for which there exists, by Theorem 4.4, a Wolff point ξ = e + αe∗ = limk zk = limk (α1k dk + α2k d∗k ) = d + αd∗ = v + αv ∗ , where e ∈ ∂D is a minimal tripotent and e = d if d is a minimal tripotent. Otherwise, e = v and ξ = lim zk k

 where zk = zk , v v + zk , v ∗ v ∗ .

We retain these notations throughout the section and make use of the following result from [23, Theorem 3.1].

140

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

Lemma 6.4. Given a compact holomorphic map ψ : U −→ U on the open unit ball U of a complex Banach space, the following conditions are equivalent. (i) ψ has a fixed point in U . (ii) There is a subsequence (ψ nk ) of (ψ n ) such that supk ψ nk (z) < 1 for some z ∈ U . Lemma 6.5. Let h = limk f nk be a limit function of {f n } on a Lie ball D. Then we have h(D) ⊂ ∂D. Proof. Let z ∈ D. Since f has no fixed point in D, we have supk f nk (z) = 1 by the above lemma. Since f nk (z) < 1 for all nk , there is a subsequence (f mk (z)) of (f nk (z)) such that limk f mk (z) = 1. It follows that h(z) = limk f mk (z) = 1. This proves h(D) ⊂ ∂D. 2 Let y ∈ D and S(ξ, y) be the closed horosphere in Theorem 4.4. By Lemma 6.5 and the f -invariance of S(ξ, y) ∩ D from Lemma 4.2, we have h(y) ∈ S(ξ, y) ∩ ∂D for every limit function h of (f n ). By Lemma 3.3, we also have h(D) ⊂ Ku ⊂ ∂D for some boundary component Ku . To determine the image h(D), we therefore look into the intersection S(ξ, y) ∩ ∂D and the boundary component Ku . In fact, these two sets are closely related. We first consider S(ξ, y) ∩ ∂D where

1/2 S(ξ, y) = t2 e + αs2 e∗ + B te + se∗ , ye + se∗ (D). Lemma 6.6. Let f : D −→ D be a fixed-point free compact holomorphic map on a Lie ball D with a Wolff point ξ = e + αe∗ ∈ ∂D. Then for each y ∈ D, we have Kξ ⊂ S(ξ, y) ∩ ∂D. If |α| < 1, then Ke ⊂ S(ξ, y). If |α| = 1 and s > 0, then Ke \S(ξ, y) = ∅ and 

   μ − αs2    ≤ 1 ⊂ S(ξ, y) e + μe :  1 − s2  ∗

in which case we have

e + Te∗ ∩ S(ξ, y) = {ξ}.

Proof. Let ξ = e + αe∗ = limk zk be constructed in Theorem 4.4. Fix y ∈ D. We note that Kξ = gξ (D). Let x ∈ D. Then x ≤ rk from some k onwards, for which we have gzk (x) ∈ gzk ( D(0, rk ) ) = Dk [y]. Hence gξ (x) = limk gzk (x) ∈ S(ξ, y). This proves Kξ ⊂ S(ξ, y) ∩ ∂D.

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

141

If |α| < 1, then Ke = Kξ . Let |α| = 1 and s > 0. Let F : D −→ S(ξ, y) be the affine homeomorphism given in Corollary 4.5. We have e + μe∗ ∈ S(ξ, y) if, and only if, F −1 (e + μe∗ ) ≤ 1 where



−1/2





μ − αs2 ∗ F −1 e + μe∗ = B te + se∗ , te + se∗ e . 1 − t2 e + μ − αs2 e∗ = e + 1 − s2 It follows that e + μe∗ ∈ S(ξ, y) if, and only if,    μ − αs2     1 − s2  ≤ 1. If |μ| = 1, this inequality forces μ = α and hence

  e + Te∗ ∩ S(ξ, y) = e + αe∗

which also implies Ke \S(ξ, y) = ∅ since S(ξ, y) is closed.

2

Remark 6.7. We note that it is possible to have e ∈ S(ξ, y) for |α| = 1 and s > 0. Indeed, we have shown s2 ≤ (1 + y)/2 before and for y = 0, we have s2 ≤ 1/2 and e ∈ S(ξ, 0) αs2 since | 1−s 2 | ≤ 1. Proposition 6.8. Let f : D −→ D be a fixed-point free compact holomorphic map on a Lie ball D with a Wolff point ξ = e + αe∗ ∈ ∂D. Then we have 

S(ξ, τ ) =

τ >0



∞ 

S(ξ, n) =

n=1

{ξ}

if ξ is non-degenerate

Ke

otherwise.

Proof. The first equality follows from the fact that {S(ξ, τ )}τ >0 is a nested family, that is, S(ξ, τ ) ⊂ S(ξ, τ  ) for τ > τ  , since τ=

1 − s2k 1 − (sk )2  > τ = 1 − zk 2 1 − zk 2

implies sk < sk and Dk [τ ] = gzk ( D(0, sk ) ) ⊂ Dk [τ  ]. If ξ is not non-degenerate, then σ = 0 in Theorem 4.6 and we have S(ξ, τ ) =

 

1/2 τ e+B τ /(1 + τ ) e, τ /(1 + τ ) e (D). 1+τ

It follows that Ke =

 

1/2 τ e+B τ /(1 + τ ) e, τ /(1 + τ ) e (Ke ) ⊂ S(ξ, τ ). 1+τ

142

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

Conversely, given x ∈

∞ n=1

S(ξ, n), we have, for each n,

  1/2 

 n n n e+ 1− wn , e e + 1 − x= P1 (e)(wn ) + wn , e∗ e∗ 1+n 1+n 1+n for some wn ∈ D. Let (wnk ) be a subsequence of (wn ) converging weakly to some point w ∈ D. Letting nk → ∞, we obtain

 x = e + w, e∗ e∗ ∈ K e which proves the assertion when ξ is not non-degenerate. If ξ is non-degenerate, then σ > 0 in the horosphere S(ξ, τ ) =

As before, given x ∈

∞ n=0

αστ ∗ τ e+ e 1+τ 1 + στ   +B τ /(1 + τ ) e + στ /(1 + στ ) e∗ ,  

1/2 τ /(1 + τ ) e + στ /(1 + στ ) e∗ (D). S(ξ, n), we have

  ασn ∗ n n e+ e + 1− wn , e e x= 1+n 1 + σn 1+n     

 n ασn σn 1− 1− P1 (e)(wn ) + 1 − wn , e∗ e∗ + 1+n 1 + σn 1 + σn for some wn ∈ D. Again, taking subsequential limit yields x = e + αe∗ = ξ. This concludes the proof. 2 For a limit function h of (f n ), we now consider the boundary component Ku which contains h(D), where u is a tripotent. There are two possibilities: either u is a maximal tripotent in which case Ku = {u} and h is a constant function, or u is a minimal tripotent in which case Ku = u + Du. We note from Theorem 5.1 that h(D) need not be closed. We first consider the case when h is a constant function. Lemma 6.9. Let f be a fixed-point free compact holomorphic self-map on a Lie ball D, with the Wolff point ξ = e + αe∗ given in Theorem 4.4. Let h be a limit function of (f n ). Then h(D) ∩ (e + De∗ ) = ∅. Moreover, ξ ∈ h(D) if ξ is non-degenerate. Proof. For each n = 1, 2, . . . , pick yn in the horosphere S0 (ξ, n) in Theorem 4.6. Then h(yn ) ∈ S(ξ, n) and there exists wn ∈ D such that

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

h(yn ) =

143

  ασn ∗ n n e+ e + 1− wn , e e 1+n 1 + σn 1+n     

 n ασn σn 1− 1− P1 (e)(wn ) + 1 − wn , e∗ e∗ . + 1+n 1 + σn 1 + σn

Let (wnk ) be a subsequence of (wn ) with a weak limit w ∈ D. If σ = 0, then we have

 e + w, e∗ e∗ = lim h(ynk ) ∈ h(D). k

where | w, e∗ | = P0 (e)(w) ≤ 1. If σ > 0, then e + αe∗ = lim h(ynk ) ∈ h(D). k

2

Proposition 6.10. Let f be a fixed-point free compact holomorphic self-map on a Lie ball D, with Wolff point ξ = e + αe∗ . If h is a constant limit function of (f n ), then h(D) = {e + βe∗ } for some |β| ≤ 1. Further, we have h(D) = {ξ} if ξ is non-degenerate. Proof. This follows from Lemma 6.9 since h(D) = h(D) if h is constant. 2 Example 6.11. In the above result, it can happen that |β| < 1. Consider the threedimensional Lie ball D in C3 , with the standard inner product and involution described in Example 2.4, but with triple product {z, z, z} = z, z z − z, z z/2, as in (2.2). Pick a minimal tripotent e ∈ ∂D. We have the Peirce decomposition C3 = C e ⊕ C e ⊕ V1 (e) where dim V1 (e) = 1 and hence V1 (e) = C z for some z ∈ D. For a linear combination w = λe + μe + γz in D, we have |λ| = P2 (e)(w) < 1 and |μ| = P0 (e)(w) < 1. Let ψ : D −→ D be a holomorphic function with a fixed point β ∈ D such that ψ is not biholomorphic. Then the iterates (ψ n ) converge locally uniformly to the constant β (cf. [3]). Define a fixed-point free holomorphic map f : D −→ D by f (λe + μe + γv) = og 1/2 (λ)e + ψ(μ)e where og 1/2 (λ)e + ψ(μ)e = max{|og 1/2 (λ)|, |ψ(μ)|} < 1. Then f n (λe + μe + γv) = og n1/2 (λ)e + ψ n (μ)e and (f n ) converges locally uniformly to the constant map h(·) = e + βe. To see that ξ = e + βe is the Wolff point in our construction before, let 0 < αk ↑ 1 and let λk ∈ D be a fixed point of αk og 1/2 : D −→ D. Let μk be a fixed point of αk ψ.

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

144

Then zk = λk e + μk e is a fixed point of αk f . Taking subsequence, we may assume (λk ) and (μk ) converge to λ and μ in D, respectively. Then ξ = limk zk = λe + μe. Since the extended Möbius transformation og 1/2 has fixed points at ±1 and h(D) ⊂ e + D e, we must have λ = 1. Since h(D) = {e + β e} with |β| < 1, we cannot have |μ| = 1, by Lemma 6.9. As β is the unique fixed point of ψ, it follows that μ = β. We now consider the case h(D) ⊂ u + Du∗ for some minimal tripotent u. Let y ∈ D. Then h(y) ∈ (u + Du∗ ) ∩ S(ξ, y) ∩ ∂D. Hence we need to examine this intersection. The following series of arguments will lead to the relationship between u and ξ = e + αe∗ in Lemma 6.12. Let x ∈ S(ξ, y) ∩ ∂D and let x = u + βu∗ for some minimal tripotent u with β ∈ D. Then we have and x = limk xk with xk ∈ Dk [y], where g−zk (xk ) ≤ rk . Let xk have spectral decomposition xk = β1k vk + β2k vk∗



1 > xk  = β1k ≥ |β2k | .

(6.1)

Then limk xk  = x = 1 and limk β2k = x, x∗ /2 = β. Since |β| < 1, analogous to the case of ξ = limk zk = limk (α1k dk + α2k d∗k ) and picking a subsequence if necessary, we may assume the sequence (vk ) norm converges to a minimal tripotent, which must be u by uniqueness of the spectral decomposition of x. We will make use of the identities

1 − zk 2 B(zk , zk )−1/2 P2 (dk )    1 − zk 2 1 − zk 2 = P2 (dk ) + P1 (dk ) + P0 (dk ) P2 (dk ) 1 − |α2k |2 1 − |α2k |2 = P2 (dk )

(6.2)

and analogously,

P2 (vk ) 1 − xk 2 B(xk , xk )−1/2 = P2 (vk ),

(6.3)



P0 (vk ) 1 − |β2k |2 B(xk , xk )−1/2 = P0 (vk ).

(6.4)

It follows from Lemma 2.2, (6.2) and (6.3) that 1 − rk2 zk 2 1 − rk2 ≥

1 − zk 2 1 − g−zk (xk )2

=

(1 − xk 2 )B(xk , xk )−1/2 B(xk , zk )(1 − zk 2 )B(zk , zk )−1/2  1 − xk 2

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

≥

P2 (vk )(1 − xk 2 )B(xk , xk )−1/2 B(xk , zk )(1 − zk 2 )B(zk , zk )−1/2 P2 (dk ) 1 − xk 2

=

P2 (vk )B(xk , zk )P2 (dk ) . 1 − xk 2

145

Let S(ξ, y) = t2 e + αs2 e∗ + B(te + se∗ , te + se∗ )1/2 (D). We need to consider two cases: (i) s < t and (ii) s = t. Let s < t. Then (dk ) in (4.1) norm converges to the tripotent e by Remark 4.3. We show   P2 (u)B(x, ξ)P2 (e) = 0. Since the sequence (

2 1−rk zk 2 ) 2 1−rk

(6.5)

converges to 1/t2 as k → ∞, we have

  2 2  

P2 (vk )B(xk , zk )P2 (dk ) ≤ 1 − xk 2 1 − rk zk  −→ 0 1 − rk2 and     P2 (u)B(x, ξ)P2 (e) = lim P2 (vk )B(xk , zk )P2 (dk ) = 0. k→∞

(6.6)

From this we deduce that  B(x, ξ)e, u u = P2 (u)B(x, ξ)P2 (e)(e) = 0

where



B(x, ξ)e = B u + βu∗ , e + αe∗ e



  = e − 2 u + βu∗  e + αe∗ (e) + x, {ξ, e, ξ}, x   = e − 2{u, e, e} − 2β u∗ , e, e + {x, e, x}.

(6.7)

Hence

 0 = B(x, ξ)e, u



    = e, u − 2 {u, e, e}, u − 2β u∗ , e, e , u + {x, e, x}, u

  = e, u − 2 {u, e, e}, u + {x, u, x}, e 2   2 = e, u − 1 −  e, u  +  e, u∗  + u, e

2  2  =  e, u∗  − 1 − e, u 

(6.8)

  ¯ ∗ 1 − e, u = λ e, u∗ = e, λu

(6.9)

which implies

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

146

¯ ∗ = 1 = u + λu∗ , e . The latter gives e∗ , u + λu∗ = λ. for some |λ| = 1 and e, u + λu We have



u + λu∗ = P2 (e) u + λu∗ + P1 (e) u + λu∗ + P0 (e) u + λu∗



 = u + λu∗ , e e + P1 (e) u + λu∗ + u + λu∗ , e∗

= e + P1 (e) u + λu∗ + λe∗ . Therefore 2   

2

2 2 = u + λu∗ h = 1 + P1 (e) u + λu∗ h + |λ|2 = 2 + P1 (e) u + λu∗ h which implies P1 (e)(u + λu∗ ) = 0 and u + λu∗ = e + λe∗ .

(6.10)

It follows that    

 u = u, u + λu∗ , u = {u, e, u} + u, λe∗ , u = u, e u + λ u, e∗ u and



 1 = u, e + λ u, e∗ = u, e + λ e, u∗ . Comparing this with (6.9), one finds e, u = u, e ∈ R.

(6.11)

Now we consider the case s = t. We note that |α| = 1 and ξ is a maximal tripotent in this case. Following the previous notation, we have    1 − rk2 zk 2 1 − rk2 s2 1 − rk2 zk 2 = lim = lim = 1. 2 |α |2 k 1 − rk k 1 − rk2 1 − rk2 |α2k |2 t2 2k This gives     1 − zk 2 1 − rk2 zk 2 1 − rk2 |α2k |2 1 − t2 1 − zk 2 = lim = lim = 1. k 1 − |α2k |2 k 1 − rk2 zk 2 1 − rk2 |α2k |2 1 − |α2k |2 1 − s2 We have





1 − zk 2 B(zk , zk )−1/2 (zk ) = 1 − zk 2 B(zk , zk )−1/2 α1k dk + α2k d∗k   1 − zk 2 α2k d∗k = α1k dk + 1 − |α2k |2

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

147

where     2   α1k dk + 1 − zk  α2k d∗k − ξ    2 1 − |α2k |     1 − zk 2 ∗   ≤ α1k dk + α2k dk − ξ + 1 − |α2k |dk  −→ 0 as k → ∞. 1 − |α2k |2 Hence, as before, we have (1 − xk 2 )B(xk , xk )−1/2 B(xk , zk )(1 − zk 2 )B(zk , zk )−1/2  1 − rk2 zk 2 ≥ 2 1 − rk 1 − xk 2 ≥

P2 (vk )B(xk , zk )(1 − zk 2 )B(zk , zk )−1/2 (zk ) 1 − xk 2

and    

P2 (u)B(x, ξ)ξ  = limP2 (vk )B(xk , zk ) 1 − zk 2 B(zk , zk )−1/2 (zk ) k

 

1 − rk2 zk 2 ≤ lim 1 − xk 2 = 0. k 1 − rk2 It follows from Lemma 2.6 that

  0 = B(x, ξ)ξ, u = ξ − 2{x, ξ, ξ} + {x, ξ, x}, u

 = ξ, u − 2 x, u + {x, u, x}, ξ = ξ, u − 2 + u, ξ

which gives Re ξ, u = 1. Since | ξ, u | ≤ 1, we must have ξ, u = Re ξ, u = 1, that is,

 e, u + α e∗ , u = 1

(6.12)

which is analogous to (6.9) where the role of λ is now played by α, with the role of e and u interchanged. Therefore we also have, as in (6.10) and (6.11), that e + αe∗ = u + αu∗ ,

e, u = u, e

(6.13)

in the case of s = t. The previous discussion establishes the following lemma. Lemma 6.12. Let D be a Lie ball and let ξ = e + αe∗ be the Wolff point in Theorem 4.4. Given y ∈ D and a minimal tripotent u ∈ ∂D such that (u + Du∗ ) ∩ S(ξ, y) = ∅, we have 

e, u = u, e = 1 − λ e, u∗ for some λ ∈ T.

and

u + λu∗ = e + λe∗

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

148

Example 6.13. The identities alone in Lemma 6.12 are insufficient to yield u = e or u = e∗ . Let D ⊂ C3 be the Lie ball in Example 6.11, with involution z → z. The following two different minimal tripotents  e=

 1+i 1−i , ,0 , 2 2

 u=



1 + i −1 + i , ,0 2 2

satisfy e, u = 0,

e, u = i and u + i u = e + i e.

We also note that e + e = −iu + i u. The closures K u and K −iu do not intersect, but they both meet K e . Lemma 6.14. Let D be a Lie ball and let ξ = e + αe∗ be the Wolff point in Theorem 4.4. Let u ∈ ∂D be a minimal tripotent satisfying

 e, u = u, e = 1 − λ e, u∗ for some λ ∈ T. Then for any x = u + βu∗ with |β| ≤ 1, we have

B(x, ξ)e = λ(λ − β)2 1 − e, u u∗ . Proof. The given identities imply u + λu∗ = e + λe∗ as in (6.10). We have   B(x, ξ)e = e − 2{u, e, e} − 2β u∗ , e, e + {x, e, x} 



= e − u, e e − u + u, e∗ e∗ − β u∗ , e e − βu∗ + β e, u e∗ + {x, e, x} 





= 1 − u, e − β u∗ , e e + u, e∗ + β e, u e∗ − u − βu∗ + {x, e, x} where   {x, e, x} = u + βu∗ , e, u + βu∗     = {u, e, u} + 2β u, e, u∗ + β 2 u∗ , e, u∗



 = u, e u + β u, e u∗ + β u∗ , e u − βe∗ + β 2 u∗ , e u∗ . Hence







B(x, ξ)e = 1 − e, u − β e∗ , u e − β − β e, u − u, e∗ e∗







− 1 − e, u − β e∗ , u u − β 1 − e, u − β e∗ , u u∗ where, from the hypothesis,

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

149



1 − e, u − β e∗ , u = 1 − e, u − βλ 1 − e, u





= (1 − βλ) 1 − e, u = λ(λ − β) 1 − e, u

and





β − β e, u − u, e∗ = β − β e, u − λ 1 − e, u = (β − λ) 1 − e, u . It follows that



B(x, ξ)e = λ(λ − β) 1 − e, u e + (λ − β) 1 − e, u e∗



− λ(λ − β) 1 − e, u u − βλ(λ − β) 1 − e, u u∗



= λ(λ − β) 1 − e, u e + λe∗ − u − βu∗

= λ(λ − β)2 1 − e, u u∗ . 2

(6.14)

Proposition 6.15. Let D ⊂ V be a Lie ball and let ξ = e + αe∗ be the Wolff point in Theorem 4.4. Given y ∈ D and a minimal tripotent u ∈ ∂D satisfying (u + Du∗ ) ∩ S(ξ, y) = ∅, we have

u + Tu∗ ∩ S(ξ, y) ⊂ e + Te∗ Proof. Let x = u + βu∗ ∈ S(ξ, y) with |β| = 1, where

1/2 S(ξ, y) = t2 e + αs2 e∗ + B te + se∗ , te + se∗ (D) and t ∈ (0, 1). Like (6.1), we write x = limk xk = limk β1k vk + β2k vk∗ where vk is a minimal tripotent and limk |β2k | = |β| = 1. If s = t above, then S(ξ, y) = t2 ξ + (1 − t2 )(D) and x = t2 ξ + (1 − t2 )(w) for some w ∈ D. Since x is an extreme point of D, we have x = ξ = e + αe∗ . If s < t, then the minimal tripotents (dk ) in (4.1) norm converges to the minimal tripotent d = e. Let (vnk ) be a subsequence of (vk ) weakly converging to some w ∈ Dh . Then we have x = w + βw∗ and



 w, u + β w, u∗ = w, x = lim vnk , β1nk vnk + β2nk vn∗ k = 1. k

(6.15)

Using (6.4) and computation similar to that after it, we obtain 1 − zk 2 1 − rk2 zk 2 ≥ 2 1 − rk 1 − g−zk (xk )2 =

(1 − |β2k |2 )B(xk , xk )−1/2 B(xk , zk )(1 − zk 2 )B(zk , zk )−1/2  1 − |β2k |2

≥

P0 (vk )B(xk , zk )P2 (dk ) . 1 − |β2k |2

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

150

It follows that   2 2  

P0 (vk )B(xk , zk )P2 (dk ) ≤ 1 − |β2k |2 1 − rk zk  −→ 0. 1 − rk2 The norm convergence of (dnk ) to e gives      B(x, ξ)e, w∗  = lim B(xn , zn )dn , vn∗  k k k k k   = limP0 (vnk )B(xnk , znk )P2 (dnk )dnk  = 0. k

By (6.6), we also have      B(x, ξ)e, w  = lim B(xn , zn )dn , vn  k k k k k   = limP2 (vnk )B(xnk , znk )P2 (dnk )dnk  = 0. k

It follows that B(x, ξ)e, x = 0 and hence, by (6.8),

  B(x, ξ)e, βu∗ = B(x, ξ)e, x − u = 0.

Now one deduces from Lemma 6.14 that

 λ(λ − β)2 1 − e, u = B(x, ξ)e, u∗ = 0 This implies that either β = λ or e, u = 1. The former case yields x = u + βu∗ = u + λu∗ = e + λe∗ . The latter case gives e = u, as eh = uh = 1, and then x = u + βu∗ = e + βe∗ .

2

While Lemma 6.6 reveals that not every boundary component of a minimal tripotent lies entirely in S(ξ, y), we show below that, if |α| < 1, then S(y, ξ) ∩ ∂D contains only one boundary component of a minimal tripotent, namely, Ke , where ξ = e + αe∗ . Corollary 6.16. Let D be a Lie ball and ξ = e + αe∗ the Wolff point in Theorem 4.4. Let y ∈ D and let Ku be a boundary component contained in S(y, ξ), where u is a minimal tripotent. Then we have Ku = Ke . Proof. Since K u ⊂ S(ξ, y), we can pick two distinct extreme points of K u in S(ξ, y), say u +βu∗ and u +β  u∗ , where β, β  are distinct complex numbers in T. By Proposition 6.15, we have u+βu∗ = e+λe∗ and u+β  u∗ = e+λ e∗ for some λ, λ ∈ T. Hence the intersection K u ∩ K e contains two distinct points and by Lemma 3.2, we have Ku = Ke . 2

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

151

Finally, we have the following description of the limit functions for a fixed-point free compact holomorphic self-map f on a Lie ball. Theorem 6.17. Let f : D −→ D be a fixed-point free compact holomorphic map on a Lie ball D with boundary ∂D. Then there is a point ξ = e + αe∗ ∈ ∂D, where e ∈ ∂D is a minimal tripotent and |α| ≤ 1, such that each non-constant limit function h of the iterates {f n } satisfies either of the following conditions. (i) h(D) ⊂ e + D e∗ . (ii) h(D) ⊂ u + D u∗ for some minimal tripotent u and (u + D u∗ ) ∩ (e + D e∗ ) is a singleton in e + T e∗ . If ξ is non-degenerate, the singleton in (ii) is {ξ}. If h is a constant limit function, then it takes value in e + D e∗ and moreover, this value is ξ if it is non-degenerate. Proof. Let ξ = e + α e∗ = limk zk be the Wolff point in Theorem 4.4. Given a limit function h, we have h(D) ⊂ ∂D by Lemma 6.5. Therefore h(D) is contained in a boundary component Ku in ∂D, for some tripotent u ∈ ∂D. If u is a maximal tripotent, then Ku = {u} = h(D) and by Proposition 6.10, u ∈ e + D e∗ . If u is a minimal tripotent, then h(D) ⊂ Ku = u + D u∗ . Pick y ∈ D and let S(ξ, y) be the closed horosphere in Theorem 4.4. Then h(y) ∈ (u + D u∗ ) ∩ S(ξ, y). By Lemma 6.12, we have



u + λu∗ = e + λe∗ ∈ u + D u∗ ∩ e + D e∗ for some λ ∈ T. If h fails case (i), then Ku = Ke and by Lemma 3.2, the intersection K u ∩ K e must be a singleton. Therefore h satisfies (ii). Now let ξ be non-degenerate and consider case (ii) where u = e. Let

1/2 S(ξ, y) = t2 e + αs2 e∗ + B te + se∗ , te + se∗ (D) in which s > 0 and |α| = 1. If s = t, then we already have u + αu∗ = e + αe∗ ∈ (u + D u∗ ) ∩ (e + D e∗ ) from (6.13). Let s < t. Then (dk ) in (4.1) norm converges to e. Write x = h(y) ∈ S(ξ, y) ∩ ∂D. As in (6.1), there exists |β| < 1 such that

x = u + βu∗ = lim xk = lim β1k vk + β2k vk∗ k

k

where xk ∈ Dk [y] and (vk ) norm converges to the minimal tripotent u. Analogous to (6.2), we have

1 − |α2k |2 B(zk , zk )−1/2 P0 (dk ) = P0 (dk )

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

152

and as before, 1 − |α2k |2 1 − rk2 |α2k |2 ≥ 2 1 − rk 1 − g−zk (xk )2 =

(1 − xk 2 )B(xk , xk )−1/2 B(xk , zk )(1 − |α2k |2 )B(zk , zk )−1/2  1 − xk 2

≥

P2 (vk )B(xk , zk )P0 (dk ) . 1 − xk 2

It follows that    

1 − rk2 |α2k |2 0 2   = 2 = 0. lim P2 (vk )B(xk , zk )P0 (dk ) ≤ lim 1 − xk  2 k k 1 − rk s In particular,        B(x, ξ)e∗ , u  = P2 (u)B(x, ξ)e∗  = limP2 (vk )B(xk , zk )P0 (dk )d∗k  = 0. k

Since       B(x, ξ)e∗ = e∗ − 2α u, e∗ , e∗ − 2βα u∗ , e∗ , e∗ + α2 x, e∗ , x , we obtain, using (6.9) in which |λ| = 1 and the fact that e, u ∈ R,

 0 = B(x, ξ)e∗ , u 

 2

 = e∗ , u − α e∗ , u  − α + α e, u 2 + α2 u, e∗





= λ 1 − e, u − α 1 − 2 e, u + e, u 2 − α + α e, u 2 + α2 λ 1 − e, u



= λ(1 − αλ)2 1 − e, u . It follows that λ = α since e, u = 1. This proves (u + D u∗ ) ∩ (e + D e∗ ) = {ξ}, by (6.10). Finally, the last assertion follows from Proposition 6.10. 2 Example 6.18. Let D ⊂ C3 be the three-dimensional Lie ball discussed in Example 6.11 and let C3 = Ce ⊕ C e ⊕ V1 (e) be the Peirce decomposition with respect to a minimal tripotent e ∈ D. Choose w ∈ V1 (e) to form a basis {e, e, w} for C3 and note that αe + β e + λw < 1 implies |α|, |β| < 1.

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

153

By [14, Théorème 4], one can construct a fixed-point free holomorphic self-map ψ on the bidisc D2 , where



ψ(μ, τ ) = ψ1 (μ, τ ), ψ2 (μ, τ ) (μ, τ ) ∈ D2 and ψ1 , ψ2 : D2 −→ D are holomorphic, such that the iterates (ψ n ) admit limit functions Λ1 and Λ2 with images 

 Λ1 D2 = exp(iθ), exp(iϕ) ,

  Λ2 D2 ⊂ exp(iθ) × D.

Define f : D −→ D by f (μe + τ e + ηw) = ψ1 (μ, τ )e + ψ2 (μ, τ ) e where ψ1 (μ, τ )e + ψ2 (μ, τ ) e = max{|ψ1 (μ, τ )|, |ψ2 (μ, τ )|}. Then there are two limit functions h1 and h2 of the iterates (f n ) with values in disjoint boundary components:   h1 (D) = exp(iθ) e + exp(iϕ) e ,

h2 (D) ⊂ exp(iθ) e + D e.

References [1] L.V. Ahlfors, Complex Analysis, McGraw–Hill, New York, 1966. [2] A.F. Beardom, The dynamics of contractions, Ergodic Theory Dynam. Systems 17 (1997) 1257–1266. [3] R.B. Burckel, Iterating analytic self-maps on discs, Amer. Math. Monthly 88 (1981) 396–407. [4] J. Byun, H. Gaussier, K.-T. Kim, Weak type normal families of holomorphic mappings in Banach spaces and characterization of the Hilbert ball by its automorphism group, J. Geom. Anal. 12 (2002) 581–599. [5] É. Cartan, Sur les domaines bornés homogènes de l’espace de n variables complexes, Abh. Math. Semin. Univ. Hambg. 11 (1935) 116–162. [6] C.-H. Chu, Jordan Structures in Geometry and Analysis, Cambridge Tracts in Math., vol. 190, Cambridge Univ. Press, Cambridge, 2012. [7] C.-H. Chu, P. Mellon, Iteration of compact holomorphic maps on a Hilbert ball, Proc. Amer. Math. Soc. 125 (1997) 1771–1777. [8] C.-H. Chu, M. Rigby, Iteration of self-maps on a product of Hilbert balls, J. Math. Anal. Appl. 411 (2014) 773–786. [9] T. Dang, Y. Friedman, Classification of JBW*-triple factors and applications, Math. Scand. 61 (1987) 292–330. [10] A. Denjoy, Sur l’itération des fonctions analytiques, C. R. Acad. Sci. Paris 182 (1926) 255–257. [11] C.J. Earle, R.S. Hamilton, A fixed point theorem for holomorphic mappings, Proc. Sympos. Pure Math. 16 (1969) 61–65. [12] Y. Friedman, Physical Applications of Homogeneous Balls, Birkhäuser, Basel, 2005. [13] K. Goebel, Fixed points and invariant domains of holomorphic mappings of the Hilbert ball, Nonlinear Anal. 6 (1982) 1327–1334. [14] M. Hervé, Itération des transformations analytiques dans le bicercle-unité, Ann. Sci. Éc. Norm. Super. 71 (1954) 1–28. [15] M. Hervé, Quelques propriétés des applications analytiques d’une boule à m dimensions das ellemême, J. Math. Pures Appl. 42 (1963) 117–147. [16] L.K. Hua, Harmonic Analysis of Functions of Several Complex Variables in the Classical Domains, Amer. Math. Soc., Providence, 1963.

154

C.-H. Chu / Advances in Mathematics 264 (2014) 114–154

[17] G. Julia, Mémoire sur l’itération des fonctions rationnelles, J. Math. Pures Appl. 8 (1918) 47–245. [18] J. Kapeluszny, T. Kuczumow, S. Reich, The Denjoy–Wolff theorem in the open unit ball of a strictly convex Banach space, Adv. Math. 143 (1999) 111–123. [19] W. Kaup, A Riemann mapping theorem for bounded symmetric domains in complex Banach spaces, Math. Z. 183 (1983) 503–529. [20] W. Kaup, J. Sauter, Boundary structure of bounded symmetric domains, Manuscripta Math. 101 (2000) 351–360. [21] K.-T. Kim, S.G. Krantz, Normal families of holomorphic functions and mappings on a Banach space, Expo. Math. 21 (2003) 193–218. [22] S. Kobayashi, Hyperbolic Manifolds and Holomorphic Mappings, World Scientific, Singapore, 2005. [23] A. Kryczka, T. Kuczumow, The Denjoy–Wolff-type theorem for compact KBH -nonexpansive maps on a Hilbert ball, Ann. Univ. Mariae Curie-Skłodowska, Sect. A 51 (1997) 179–183. [24] P. Mellon, Holomorphic invariance on bounded symmetric domains, J. Reine Angew. Math. 523 (2000) 199–223. [25] A. Stachura, Iterates of holomorphic self-maps on the unit ball of a Hilbert space, Proc. Amer. Math. Soc. 93 (1985) 88–90. [26] H. Upmeier, Symmetric Banach Manifolds and Jordan C*-algebras, Math. Stud., vol. 104, NorthHolland, Amsterdam, 1985. [27] H. Upmeier, Jordan Algebras in Analysis, Operator Theory and Quantum Mechanics, CBMS Reg. Conf. Ser. Math., vol. 67, Amer. Math. Soc., Providence, 1987. [28] J. Wolff, Sur l’itération des fonctions bornées, C. R. Acad. Sci. Paris 182 (1926) 200–201. [29] J. Wolff, Sur une géneralisation d’un théorème de Schwarz, C. R. Acad. Sci. Paris 182 (1926) 500–502. [30] H. Wu, Normal families of holomorphic mappings, Acta Math. 119 (1967) 193–233.