Global bifurcations of critical orbits of G -invariant strongly indefinite functionals

Nonlinear Analysis 74 (2011) 1823–1834

Contents lists available at ScienceDirect

Nonlinear Analysis journal homepage: www.elsevier.com/locate/na

Global bifurcations of critical orbits of G-invariant strongly indefinite functionals Anna Gołebiewska, ¸ Sławomir Rybicki ∗ Faculty of Mathematics and Computer Science, Nicolaus Copernicus University, PL-87-100 Toruń, ul. Chopina 12/18, Poland

article

info

Article history: Received 12 June 2010 Accepted 26 October 2010 MSC: primary 47H11 secondary 37G40 Keywords: Strongly indefinite functionals Global bifurcation of critical orbits Non-cooperative elliptic systems

abstract Let H be a separable Hilbert space which is an orthogonal representation of a compact Lie group G and let Φ : H → R be a G-invariant strongly indefinite functional of the class C 1 . To study critical orbits of Φ we have defined the degree for G-invariant strongly indefinite functionals, which is an element of the Euler ring U (G). Using this degree we have formulated the Rabinowitz alternative for G-invariant strongly indefinite functionals. The abstract results are applied to the study of global bifurcations of weak solutions of non-cooperative systems of elliptic differential equations. © 2010 Elsevier Ltd. All rights reserved.

1. Introduction Throughout this article G stands for a compact Lie group, (H, ⟨·, ·⟩) is a separable Hilbert G-representation and CGk (H, R) is the class of G-invariant functionals of the class C k , k ∈ N. Consider a functional Φ ∈ CG1 (H, R) of the form

Φ (u) =

1 2

⟨Lu, u⟩ − η(u),

(1.1)

where L : H → H is a bounded, self-adjoint, linear, Fredholm operator and η is a nonlinear operator whose gradient ∇η ∈ CG0 (H, H) is completely continuous, where CGk (H, H) is the class of G-equivariant operators of the class C k , k ∈ N ∪ {0}. We have a decomposition H = H− ⊕ H0 ⊕ H+ , where ±⟨Lu± , u± ⟩ > 0 for every u± ∈ H± and H0 = ker L. Since L is a Fredholm operator, dim H0 < ∞. If dim H− = dim H+ = ∞ then functional Φ is said to be strongly indefinite. We study G-orbits of critical points of G-invariant strongly indefinite functionals of the form (1.1). We are mainly interested in the study of connected sets of critical orbits of such functionals. The study of critical points of strongly indefinite functionals is motivated by a number of problems from mathematical physics. They appear for example in the search for periodic solutions of Hamiltonian systems, wave equations and weak solutions of non-cooperative systems of elliptic differential equations. Many topological and critical point theory tools have been developed for the study of critical points of strongly indefinite functionals. For instance the Morse theories have been defined in [1–9], for the Conley index theory see [10,11], the minimax methods have been considered in [12–15], and the spectral flow methods have been developed in [16–18]. In the presence of symmetries of a compact Lie group G critical orbits of G-invariant strongly indefinite functionals have been studied for instance in [19–22]. However, using these methods we cannot study connected sets of critical orbits of G-invariant functionals i.e. the continuation and global bifurcation of critical orbits; see [23–28] for discussion and examples.

∗

Corresponding author. Tel.: +48 56 639 43 54; fax: +48 56 622 89 79. E-mail addresses: [email protected] (A. Gołebiewska), ¸ [email protected] (S. Rybicki).

0362-546X/$ – see front matter © 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.na.2010.10.055

1824

A. Gołebiewska, ¸ S. Rybicki / Nonlinear Analysis 74 (2011) 1823–1834

Our aim is to study connected sets of critical orbits of G-invariant strongly indefinite functionals. That is why in this article we pursue another approach. The only degree theory known to the authors for G-invariant strongly indefinite functionals is the theory due to the second author [29] in the presence of SO(2)-symmetries. Therefore our first purpose here is to define the degree for G-invariant strongly indefinite functionals. The second goal is to apply this degree to the study of global bifurcations of weak solutions of non-cooperative elliptic systems. After this introduction the article is organised as follows. In Section 2 we set up notations and terminologies. The main result of this section is Theorem 2.1. Namely, we have proved that the degree ∇G -deg(−Id, B(V)) is invertible in the Euler ring U (G). This property will play a crucial role in the definition of the degree for G-invariant strongly indefinite functionals. In Section 3 we have defined the degree for G-invariant strongly indefinite functionals; see Definition 3.2. This degree does not depend on the choice of a G-equivariant approximation scheme on H; see Lemma 3.3. Properties of this degree have been formulated in Theorem 3.1. The product formula of the degree for G-invariant strongly indefinite functionals will play a crucial role in the study of isolated degenerate critical points; see Theorem 3.2. At the end of Section 3 we study global bifurcation of critical orbits of G-invariant strongly indefinite functionals. Theorem 3.3 is a version of the Rabinowitz alternative for this class of functionals. In Section 4 we illustrate the abstract theory developed in this article. This section is devoted to the study of global bifurcations of weak solutions of a non-cooperative elliptic system considered with Dirichlet boundary condition. Namely, we consider weak solutions of system (4.1) as critical orbits of a G-invariant strongly indefinite functional Φ defined by (4.2) and apply the degree for G-invariant strongly indefinite functionals to the study of global bifurcations of weak solutions of system (4.1). We have formulated sufficient conditions for the existence of global bifurcations of weak solutions of system (4.1) from the set of trivial ones; see Corollaries 4.1 and 4.2. Some properties of the bounded continua of weak solutions of system (4.1) bifurcating from the set of trivial solutions have been presented in Theorems 4.1 and 4.2. In Corollary 4.3 we have excluded one possibility in the Rabinowitz alternative. Namely, we have formulated sufficient conditions for the unboundedness of continua of weak solutions of system (4.1) bifurcating from the set of trivial ones. At the end of this section we illustrate the results proved in this section and consider a system of elliptic differential equations with symmetry group G = SO(3). In Section 5 we have discussed the relation of results obtained in this article to those which can be obtained by application of the Leray–Schauder degree. 2. Preliminaries In this section we set up notations and terminology. Moreover, we prove an important property of the degree for G-equivariant gradient maps; see [30,31]. We denote this degree briefly by ∇G -deg(·, ·) to underline the gradient and equivariant structures. It is worth to point out that the degree for G-equivariant gradient maps is an element of the Euler ring U (G); see [32,33]. This property, see Theorem 2.1, will play a crucial role in the definition of the degree for G-invariant strongly indefinite functionals. Let sub(G) be the set of closed subgroups of G. Here and subsequently W (H ) denotes the Weyl group of H ∈ sub(G). Set sub0 (G) = {H ∈ sub(G) : dim W (H ) = 0}. The conjugacy class of H ∈ sub(G) is denoted by (H ). We let sub[G] = {(H ) : H ∈ sub(G)} and sub0 [G] = {(H ) : H ∈ sub0 (G)}. Let F∗ [G] be the set G-homotopy types of pointed finite G-CW-complexes and denote by χG : F∗ [G] → U(G) the equivariant Euler characteristic. We remind that the Euler ring U (G) is a free abelian group with the basis χG G/H + , (H ) ∈ sub[G] and the unit I = χG (G/G+ ). If X is a G-CW-complex without base point then by X+ we ∑ denote a pointed G-CW-complex X+ = X ∪ {∗}. Denote by U+ (G) ⊂ U (G)(U− (G) ⊂ U (G)) a subset consisting of   + elements (H )∈sub[G] n(H ) · χG G/H such that n(H ) ≥ 0(n(H ) ≤ 0). Denote by (A(G), +, ◦) the Burnside ring  of G. We have a decomposition U (G) = A(G) ⊕ U0 (G), where (U0 (G), +) is a subgroup of (U (G), +) generated by χG G/H + such that (H ) ∈ sub[G] \ sub0 [G]. It is worth to point out that generally A(G) is not a subring of U (G) and that U0 (G) consists of nilpotent elements in U (G). Let πA(G) : U (G) → A(G) be the natural projection. Let V be an orthogonal G-representation and Ω ⊂ V be an open, bounded and G-invariant subset. Denote by CGk (V, R) the set of G-invariant C k -potentials, k ∈ N, and by CGk (V, V) the set of G-equivariant C k -maps, k ∈ N ∪ {0}. For ϕ ∈ CGk (V, R) the gradient ∇ϕ ∈ CGk−1 (V, V) is called Ω -admissible, if (∇ϕ)−1 (0) ∩ ∂ Ω = ∅. 1 ¸ has defined in [30] the degree for G-equivariant gradient maps ∇G -deg(∇ϕ, Ω ) = ∑ For Ω -admissible ϕ ∈ CG (V, R) Geba + (H )∈sub[G] ∇G -deg(H ) (∇ϕ, Ω ) · χG (G/H ) ∈ U (G). We underline that for G = SO(2) the degrees of [30,34] coincide. We have defined an infinite-dimensional generalisation of the degree for SO(2)-equivariant gradient operators of the form compact perturbation of the identity; see [34]. The same definition remains valid for every compact Lie group G. Under some additional assumptions the degree for SO(2)-equivariant gradient maps, which is a rational number, is due to Dancer [25] (see also [35], where the case of flow with the first integral was considered). Let Bα (V), Dα (V) ⊂ V be the open and closed discs of radius α centred at the origin, respectively, and Sα (V) = ∂ Dα (V). For simplicity of notation we write B(V), D(V), S (V) instead of B1 (V, 0), D1 (V, 0), S1 (V, 0), respectively. Remark 2.1. Fix L : V → V is a linear, self-adjoint, G-equivariant, isomorphism. It is of interest to compute ∇G -deg(L, B(V)) ∈ U (G). For a general compact Lie group G computation of ∇G -deg(L, B(V)) ∈ U (G) seems to be rather difficult. If σ− (L) := σ (L) ∩ (−∞, 0) = ∅ then by the homotopy invariance of the degree for G-equivariant gradient maps, see [30,31], we

A. Gołebiewska, ¸ S. Rybicki / Nonlinear Analysis 74 (2011) 1823–1834

1825

obtain ∇G -deg(L, B(V)) = ∇G -deg(Id, B(V)) = I ∈ U (G) i.e.∇G -deg(L, B(V)) is invertible in U (G). If σ− (L) ̸= ∅ then by the homotopy invariance of the degree for G-equivariant gradient maps there are sub-representations V− , V+ ⊂ V such that

∇G -deg(L, B(V)) = ∇G -deg((−Id, Id), B(V− ) × B(V+ )) = ∇G -deg(−Id, B(V− )) ⋆ ∇G -deg(Id, B(V+ )) = ∇G -deg(−Id, B(V− )). It is understood that if σ (L) = σ− (L) then V+ = {0} and ∇G -deg(Id, B(V+ )) = I. Therefore without loss of generality we can assume that L = −Id. In the theorem below we prove that ∇G -deg(−Id, B(V)) is invertible in U (G). In opinion of the authors this theorem exists in the mathematical folklore, but the proof of this theorem has never been published. Theorem 2.1. The degree ∇G -deg(−Id, B(V)) is invertible in U (G). More precisely, there are x ∈ U0 (G) and n ∈ N such that ∇G -deg(−Id, B(V))−1 = ∇G -deg(−Id, B(V)) ⋆ I + x + · · · + xn−1 . Proof. The following equality is due to Geba ¸ [30]: ∇G -deg(−Id,B(V)) = χG (S V ). Therefore to prove this theorem it is enough to prove that χG (S V ) is invertible in U (G). We know that πA(G) χG (S V⊕V ) = I; see [32,33]. Hence χG (S V⊕V ) = I − x, where x ∈ U0 (G). Since x is nilpotent, there is n ∈ N such that xn = Θ ∈ U (G). Therefore (I − x)⋆ I + x + · · · + xn−1 = I − xn = I



and consequently we obtain χG S V⊕V such that



 −1



= I + x + · · · + xn−1 . Summing up we obtain that there are x ∈ U0 (G) and n ∈ N

  −1     = χG S V⊕V ⋆ I + x + · · · + xn−1  V           ⋆ χG S ⋆ I + x + · · · + x n −1 = χ G S V ⋆ χ G S V ⋆ I + x + · · · + x n −1 ,

I = χ G S V ⊕V ⋆ χ G S V ⊕V



= χG S 

 V



which completes the proof.



Corollary 2.1. Combining Remark 2.1 and Theorem 2.1 we obtain that if L : V → V is a linear, self-adjoint, G-equivariant, isomorphism then the degree ∇G -deg(L, B(V)) is invertible in U (G). 3. Definition of degree In this section we define the degree for G-invariant strongly indefinite functionals and prove some of its basic properties. Since the definition of this invariant is done by finite-dimensional approximations of the functional Φ ∈ CG1 (Ω , R) satisfying assumptions (a1)–(a4), we start with a definition of a G-equivariant approximation scheme. Let (H, ⟨·, ·⟩) be an infinite-dimensional, separable Hilbert space which is an orthogonal G-representation. Here Γ = {πn : H → H : n ∈ N ∪ {0}} denotes a sequence of G-equivariant orthogonal projections. Definition 3.1. We say that Γ is a G-equivariant approximation scheme on H, if (1) Hn = πn (H) is a finite-dimensional sub-representation of representation H, for all n ∈ N, (2) there exists a sub-representation Hn of Hn+1 such that Hn+1 = Hn ⊕ Hn and Hn ⊥ Hn for all n ∈ N, (3) limn→∞ πn (u) = u, for all u ∈ H. Set Hn+1 ⊖ Hn = {u ∈ Hn+1 : ⟨u, v⟩ = 0 ∀v ∈ Hn }. It is clear that Hn+1 ⊖ Hn = Hn . The following assumptions we will need throughout the rest of this section: (a1) Ω ⊂ H is an open, bounded and G-invariant subset, (a2) L : H → H is a linear, bounded, self-adjoint, G-equivariant Fredholm operator satisfying the following assumptions (a21) ker L = H0 , (a22) πn ◦ L = L ◦ πn , for all n ∈ N ∪ {0}, (a3) ∇η : Ω → H is a continuous, G-equivariant, compact operator, (a4) Φ ∈ CG1 (Ω , R) satisfies the following assumptions: (a41) ∇ Φ (u) = Lu − ∇η(u), (a42) cl((∇ Φ )−1 (0)) ∩ ∂ Ω = ∅. If H′ is a G-sub-representation of H then for the simplicity of notation we write L instead of L|H′ . Since cl((∇ Φ )−1 (0)) ∩

∂ Ω = ∅, (∇ Φ )−1 (0) is compact, G-invariant and dist ((∇ Φ )−1 (0), ∂ Ω ) > 0. Choose 0 < ε ≤ dist ((∇ Φ )3 (0),∂ Ω ) and set Ωε = (∇ Φ )−1 (0) + Bε (H). It is clear that Ωϵ is open, bounded, G-invariant and that cl(Ωϵ ) ⊂ Ω . Moreover, it is easy to check that πn (Ωϵ ) ⊂ Ω for sufficiently large n ∈ N. That is why the operator L − πn ◦ ∇η ◦ πn : Ωε → H is well defined. −1

Before we define the degree for G-invariant strongly indefinite functionals let us explain the main idea of this definition. We approximate the Hilbert space H by a finite-dimensional space Hn , n = 0, the functional Φ by functionals Φn = Φ|Ωϵ ∩Hn , n = 0 and consider a sequence {∇G -deg(∇ Φn , Ωε ∩ Hn )} ⊂ U (G), with n sufficiently large. If the operator L : H → H has a finite negative spectrum then this sequence stabilises and we obtain the definition of the degree ∇G -deg(∇ Φ , Ω ) ∈ U (G). In this case the functional Φ is not strongly indefinite. In the case of a strongly indefinite L

1826

A. Gołebiewska, ¸ S. Rybicki / Nonlinear Analysis 74 (2011) 1823–1834

the sequence {∇G -deg(∇ Φn , Ωε ∩ Hn )} does not stabilise. But the difference between two of these degrees depends only on L, and we can correct this sequence. To be more precise, passing from one approximation to another, the only new part of the degree is the product by a degree of L on the orthogonal complement Hm ⊖ Hn i.e. by ∇G -deg(L, B(Hm ⊖ Hn )), which is invertible in the Euler ring U (G); see Corollary 2.1. That is why one may get a definition which is stable. Namely, we prove that the sequence {∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(∇ Φn , Ωε ∩ Hn )} ⊂ U (G) stabilises. The proof of the statement presented below follows the same lines as the one related to the passage from the Brouwer degree to the Leray–Schauder degree. Lemma 3.1. Fix a functional Φ ∈ CG1 (Ω , R) satisfying assumptions (a1)–(a4) and choose 0 < ε ≤ is n0 ∈ N such that for every m ≥ n > n0 the following equality holds

dist ((∇ Φ )−1 (0),∂ Ω ) . Then there 3

∇G -deg(L − πm ∇η, Ωε ∩ Hm ) = ∇G -deg(L − πn ∇η, Ωε ∩ Hn ) ⋆ ∇G -deg(L, Bε (Hm ⊖ Hn )). We are now in a position to define the degree for G-invariant strongly indefinite functionals. dist ((∇ Φ )−1 (0),∂ Ω )

Definition 3.2. Let Φ ∈ CG1 (Ω , R) satisfy assumptions (a1)–(a4). Fix 0 < ε ≤ and choose n > n0 , where 3 n0 is given by Lemma 3.1. Then the degree for G-invariant strongly indefinite functionals is defined as follows

∇G -deg(L − ∇η, Ω ) = ∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(L − πn ∇η, Ωε ∩ Hn ).

(3.1) dist ((∇ Φ )−1 (0),∂ Ω ) 3

What is left is to show that the definition does not depend on the choice of 0 < ε ≤ Similarly to the case of Lemma 3.1 we omit the proof of the following standard statement.

and n > n0 .

Lemma 3.2. Under the assumptions of Lemma 3.1, if moreover n0 is provided by this lemma, then for every m ≥ n > n0 the following equality holds

∇G -deg(L, B(Hm ⊖ H0 ))−1 ⋆ ∇G -deg(L − πm ∇η, Ωε ∩ Hm ) = ∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(L − πn ∇η, Ωε ∩ Hn ). It follows directly from Lemma 3.2 that the definition of the degree for G-invariant strongly indefinite functionals does not depend on the choice of n > n0 . dist ((∇ Φ )−1 (0),∂ Ω )

Fix 0 < ε1 < ε2 ≤ , choose n0 (ε1 ), n0 (ε2 ) as in Lemma 3.1 for ε = ε1 , ε2 , respectively, and put 3 n0 = max{n0 (ε1 ), n0 (ε2 )}. Without restriction of generality we can assume that (L − πn ∇η)−1 (0) ∩ cl(Ωϵ2 \ Ωϵ1 ) ∩ Hn = ∅ for every n > n0 , which implies that

∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(L − πn ∇η, Ωε2 ∩ Hn ) = ∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(L − πn ∇η, Ωε1 ∩ Hn ). We have just proved that the definition of the degree for G-invariant strongly indefinite functionals does not depend on the choice of ϵ . Remark 3.1. Suppose that G = SO(2). We underline that the degree defined in this article does not coincide with that of [29]. On the other hand it coincides with that of [34] for L = IdH : H → H. The same approach to the definition of the infinite-dimensional version of the degree for SO(2)-equivariant gradient operators of the form of compact perturbation of the identity, under some additional assumptions maps, has been performed in [25]. In the lemma below we prove the independence of the definition of the degree on the choice of the approximation scheme. Lemma 3.3. The definition of the degree for G-invariant strongly indefinite functionals does not depend on the choice of a Gequivariant approximation scheme on H.

 = { Proof. Let Γ = {πn : H → H : n ∈ N ∪ {0}} and Γ πn : H → H : n ∈ N ∪ {0}} be G-equivariant approximation schemes on H, where Hn = im πn ,  Hn = im  πn . Define  Hn = (Hn ⊖ (Hn ∩  Hn )) ⊕ ( Hn ⊖ (Hn ∩  Hn )) ⊕ (Hn ∩  Hn ). We claim that ∃n1 ∈N ∀n>n1 ∀t ∈[0,1] ∀u∈∂(Ωε ∩ Hn ) Lu − t (πn ◦ ∇η ◦ πn )(u) − (1 − t ) · ( πn ◦ ∇η ◦  πn )(u) ̸= 0.

(3.2)

Suppose, contrary to our claim, that ∀j∈N ∃nj >j ∃tj ∈[0,1] ∃uj ∈∂(Ωε ∩ Hn ) such that

πnj ◦ ∇η ◦  πnj )(uj ) = 0, Luj − tj (πnj ◦ ∇η ◦ πnj )(u) − (1 − tj ) · ( which is equivalent to uj − u0j − tj · L˜ −1 (πnj ◦ ∇η ◦ πnj )(uj ) − (1 − tj ) · L˜ −1 (π˜ nj ◦ ∇η ◦ π˜ nj )(uj ) = 0, where u0j = π0 (uj ) and L˜ = (Id, L) : ker L ⊕ im L → H. We obtain that there exist t0 , ua , ub such that t0 = limj→∞ tj , ua = limj→∞  L−1 (πnj ◦

∇η ◦ πnj )(uj ), ub = limj→∞  L−1 (π˜ nj ◦ ∇η ◦  πnj )(uj ) and limj→∞ (uj − u0j ) = t0 · ua + (1 − t0 ) · ub . Hence there exists a limit

A. Gołebiewska, ¸ S. Rybicki / Nonlinear Analysis 74 (2011) 1823–1834

1827

limj→∞ uj ∈ ∂ Ωε ∩ (∇ Φ )−1 (0), a contradiction. Consequently by the homotopy invariance of the degree for G-equivariant gradient maps, see [30,31], and (3.2) we obtain

∇G -deg(L − πn ◦ ∇η ◦ πn , Ωε ∩  Hn ) = ∇G -deg(L −  πn ◦ ∇η ◦  π n , Ωε ∩  Hn ).

(3.3)

By the product formula for the degree for G-equivariant gradient maps we obtain

∇G -deg(L − πn ◦ ∇η ◦ πn , Ωε ∩  Hn ) = ∇G -deg(L, Bε ( Hn ⊖ Hn )) ⋆ ∇G -deg(L − πn ◦ ∇η, Ωε ∩ Hn ) = ∇G -deg(L, Bε ( Hn ⊖ ( Hn ∩ Hn ))) ⋆ ∇G -deg(L − πn ◦ ∇η, Ωε ∩ Hn )

(3.4)

∇G -deg(L −  πn ◦ ∇η ◦  πn , Ωε ∩  Hn ) = ∇G -deg(L, Bε ( Hn ⊖  Hn )) ⋆ ∇G -deg(L −  πn ◦ ∇η, Ωε ∩  Hn ) = ∇G -deg(L, Bε (Hn ⊖ (Hn ∩  Hn ))) ⋆ ∇G -deg(L −  πn ◦ ∇η, Ωε ∩  Hn ).

(3.5)

and

Finally, multiplying (3.3) by ∇G -deg(L, Bε ( Hn ⊖ ( Hn ∩ Hn )))−1 ⋆ ∇G -deg(L, B(Hn ⊖ H0 ))−1 and taking into account (3.4) and (3.5) and  H0 = H0 we obtain

∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(L − πn ◦ ∇η, Ωε ∩ Hn ) = ∇G -deg(L, B( Hn ⊖  H0 ))−1 ⋆ ∇G -deg(L −  πn ◦ ∇η, Ωε ∩  Hn ), which completes the proof.



The theorem below yields information about basic properties of the degree for G-invariant strongly indefinite functionals. Theorem 3.1. Let potential Φ ∈ CG1 (H, R) satisfy assumptions (a1)–(a4). Then the degree for G-invariant strongly indefinite functionals has the following properties: (1) (a) if ∇G -deg(∇ Φ , Ω ) ̸= Θ ∈ U (G), then (∇ Φ )−1 (0) ∩ Ω ̸= ∅, (b) if Ω = Ω1 ∪ Ω2 and Ω1 , Ω2 are open, disjoint and G-invariant sets, then ∇G -deg(∇ Φ , Ω ) = ∇G -deg(∇ Φ , Ω1 ) + ∇G -deg(∇ Φ , Ω2 ), (c) if Ω1 ⊂ Ω is open and G-invariant and (∇ Φ )−1 (0) ∩ Ω ⊂ Ω1 , then ∇G -deg(∇ Φ , Ω ) = ∇G -deg(∇ Φ , Ω1 ), (d) if 0 ∈ Ω and Φ ∈ CG2 (Ω , R) is such that ∇ Φ (0) = 0 and ∇ 2 Φ (0) : H → H is a G-equivariant self-adjoint isomorphism then there is α0 > 0 such that for every α < α0 we have ∇G -deg(∇ Φ , Bα (H)) = ∇G -deg(∇ 2 Φ (0), B(H)). (2) Let assumptions (a1)–(a2) be fulfilled. Fix Φ ∈ CG1 (Ω × [0, 1], R) such that (∇u Φ )−1 (0) ∩ (∂ Ω × [0, 1]) = ∅ and ∇u Φ (u, t ) = L(u) − ∇η(u, t ), where ∇u η : Ω × [0, 1] → H is G-equivariant and compact. Then ∇G -deg(∇u Φ (·, 0), Ω ) = ∇G -deg(∇u Φ (·, 1), Ω ). dist ((∇ Φ )−1 (0),∂ Ω )

Proof. (1)(a) Suppose that ∇G -deg(∇ Φ , Ω ) ̸= Θ ∈ U (G), fix 0 < ε ≤ and choose n0 ∈ N provided by 3 n 0 −1 n Lemma 3.1. Then ∇G -deg(L, B(H ⊖ H )) ⋆ ∇G -deg(L − πn ∇η, Ωε ∩ H ) ̸= Θ , for all n > n0 . Therefore for every n > n0 we have ∇G -deg(L − πn ∇η, Ωε ∩ Hn ) ̸= Θ . By the existence property of the degree for G-equivariant gradient maps for every n > n0 there is un ∈ Ωε ∩ Hn such that Lun = (πn ◦ ∇η)(un ). Now it is easy to prove that limn→∞ un = u0 ∈ Ω ∩ (∇ Φ )−1 (0), up to subsequence, which completes the proof. (1)(b) Since Ω1 ∩ Ω2 = ∅, (∇ Φ )−1 (0) ∩ Ω1 and (∇ Φ )−1 (0) ∩ Ω2 are disjoint compact sets. Fix 0 < ε < min{dist (∂ Ω1 ,(∇ Φ )−1 (0)),dist (∂ Ω2 ,(∇ Φ )−1 (0))} 3

and choose n0 ∈ N provided by Lemma 3.1. Then, by the additivity property of the degree for G-equivariant gradient maps, for n > n0 we obtain

∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(L − πn ∇η, Ωε ∩ Hn ) = ∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(L − πn ∇η, (Ω1ε ∪ Ω2ε ) ∩ Hn ) = ∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(L − πn ∇η, Ω1ε ∩ Hn ) + ∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(L − πn ∇η, Ω2ε ∩ Hn ), which completes the proof. dist (∂ Ω ,(∇ Φ )−1 (0))

1 (1)(c) Fix 0 < ε ≤ and choose n0 ∈ N as in Lemma 3.1. Then for n > n0 , by the localisation property of 3 the degree for G-equivariant gradient maps we obtain

∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(L − πn ∇η, Ωε ∩ Hn ) = ∇G -deg(L, B(Hn ⊖ H0 ))−1 ⋆ ∇G -deg(L − πn ∇η, Ω1ε ∩ Hn ), which completes the proof. (1)(d) Since ∇ 2 Φ (0) is an isomorphism, the homotopy ∇ Ψ (u, t ) = t ∇ Φ (u) + (1 − t )∇ 2 Φ (0)u does not vanish on Sα (H) for sufficiently small α . The rest of the proof is a consequence of (2). dist (∂ Ω ,(∇ Φ )−1 (0)) (2) Fix 0 < ε ≤ . Since Lu − ∇u η(u, t ) ̸= 0 for every t ∈ [0, 1] and u ∈ ∂ Ωε , we obtain ∃n0 ∈N ∀n>n0 3 ∀u∈∂(Ωε ∩Hn ) Lu ̸= (πn ◦ ∇u η)(u, t ). Hence by the homotopy invariance property of the degree for G-equivariant gradient maps, we obtain ∇G -deg(L − πn ◦ ∇u η(·, 0), Ωε ∩ Hn ) = ∇G -deg(L − πn ◦ ∇u η(·, 1), Ωε ∩ Hn ). The rest of the proof is a direct consequence of the definition of the degree for G-invariant strongly indefinite functionals. 

1828

A. Gołebiewska, ¸ S. Rybicki / Nonlinear Analysis 74 (2011) 1823–1834

Remark 3.2. The case of variational (orthogonal) operators of the form of a compact perturbation of the identity equivariant with respect to the groups Γ × S 1 was considered in [36,37]. For Γ being an arbitrary compact Lie group, we refer to [38]. For the equivariant degree theory in the non-variational setting we refer to [36,39]). Finally the Brouwer (Leray–Schauder) degree of the equivariant maps was studied in [40]. The case of two actions was completely settled in [41]. In the following theorem we formulate the Cartesian product formula for the degree for G-invariant strongly indefinite functionals. We omit the straightforward proof of this theorem. Theorem 3.2. Let Ω1 ⊂ H1 , Ω2 ⊂ H2 be open, bounded and G-invariant subsets of G-representations H1 , H2 . Assume that the functionals Φi ∈ CG1 (Hi , R), i = 1, 2, are of the form Φi (u) = 21 ⟨Li u, u⟩ + ηi (u) and satisfy assumptions (a1)–(a4). Define a functional Φ ∈ CG1 (H, R) by Φ (u1 , u2 ) = Φ1 (u1 ) + Φ2 (u2 ) and set Ω = Ω1 × Ω2 . Then, ∇G -deg(∇ Φ , Ω1 × Ω2 ) = ∇G -deg(∇ Φ1 , Ω1 ) ⋆ ∇G -deg(∇ Φ2 , Ω2 ). In the rest of this section we need the following assumption (a5) K : H → H is a linear, bounded, self-adjoint, G-equivariant, compact operator. Let assumptions (a2), (a5) be fulfilled and let be Φ ∈ CG2 (H × R, R) of the form

Φ (u, λ) =

1 2

⟨Lu − λKu, u⟩ − η0 (u, λ),

(3.6)

where ∇η0 : H × R → H is a compact, and G-equivariant operator such that ∇η0 (u, λ) = o(‖u‖), as ‖u‖ → 0, uniformly on bounded λ-intervals. Definition 3.3. A solution of the problem

∇u Φ (u, λ) = 0,

(3.7)

of the form (0, λ) is said to be trivial. A point (0, λ0 ) ∈ H × R is said to be a bifurcation point of solutions of problem (3.7) if (0, λ0 ) ∈ cl{(u, λ) ∈ (H \ {0}) × R : ∇u Φ (u, λ) = 0}. The set of bifurcation points of solutions of problem (3.7) will be denoted by BIF (Φ ). Define P (Φ ) = {(0, λ0 ) ∈ {0} × R : ∇u2 Φ (0, λ0 ) is not an isomorphism}. By the implicit function theorem we obtain BIF (Φ ) ⊂ P (Φ ). It is of interest to formulate sufficient conditions for the existence of bifurcation points of solutions of problem (3.7). Below we define a bifurcation index. Nontrivial bifurcation index will imply a global bifurcation of solutions of problem (3.7). Definition 3.4. An element BIF G (λ0 ) ∈ U (G) defined by

BIF G (λ0 ) = lim (∇G -deg(L − (λ0 + ϵ)K , B(H)) − ∇G -deg(L − (λ0 − ϵ)K , B(H))) ϵ→0

is said to be a bifurcation index at (0, λ0 ) ∈ H × R. Since P (Φ ) does not have finite accumulation points, the bifurcation index BIF G (λ0 ) is well defined. Write N = cl{(u, λ) ∈ (H \ {0}) × R : ∇u Φ (u, λ) = 0}. Let C (λ0 ) be a continuum (closed connected component) of N such that (0, λ0 ) ∈ C (λ0 ). In the theorem below we study global bifurcations of critical orbits of G-invariant functionals of the form (3.6). The proof of this theorem is standard; see for instance [42–45]. In the classical proof it is enough to replace the Leray–Schauder degree by the degree for G-invariant strongly indefinite functionals. Theorem 3.3. Let Φ ∈ CG2 (H × R, R) be of the form (3.6). Fix (0, λ0 ) ∈ P (Φ ) such that BIF G (λ0 ) ̸= Θ ∈ U (G). Then (1) either C (λ0 ) is unbounded, (2) or C (λ0 ) is bounded and additionally the following conditions are satisfied (a) C (λ0 ) ∩ ({0} × R) = {(0, λ0 ), . . . , (0, λp )} ⊂ P (Φ ), (b) p −

BIF G (λi ) = Θ ∈ U (G).

(3.8)

i=0

Remark 3.3. The important question is how to verify the main assumption of the above theorem i.e. the condition

BIF G (λ0 ) ̸= Θ ∈ U (G). Generally speaking this a difficult task because the Euler ring U (G) has a complicated structure.

However, one can reduce this problem to an H-equivariant problem for some H ∈ sub(G). Usually the H-symmetric problem is simpler because we require less symmetries. It can happen that the structure of the Euler ring U (H ) is much simpler than that of U (G). Assume that G is connected and let H = T ⊂ G be a maximal torus. The structure of the Euler ring U (T ) is relatively simple because the torus is a commutative group; see [46]. A natural homomorphism ϕ : T → G induces a ring homomorphism ϕ ∗ : U (G) → U (T ); see [38,32,33]. It is clear that ϕ ∗ (BIF G (λ0 )) = BIF T (λ0 ) ∈ U (T ). The

A. Gołebiewska, ¸ S. Rybicki / Nonlinear Analysis 74 (2011) 1823–1834

1829

following equivalence is known BIF G (λ0 ) ̸= Θ ∈ U (G) iff BIF T (λ0 ) ∈ U (T ) ̸= Θ ∈ U (T ); see [46]. We point out that the connectivity of G is important here since, in such a case every two maximal tori are conjugate and every element of G is conjugate to an element of a maximal torus. In other words we have simplified the verification of the assumption BIF G (λ0 ) ̸= Θ ∈ U (G). Now it is enough to proceed with the verification in U (T ). We pursue this approach, with H = SO(2), in the next section to prove the existence of global bifurcations of weak solutions of problem (4.1). Another interesting question is related to the existence of the symmetry breaking phenomenon of solutions of system (4.1). In our approach we lose some information about isotropy groups of bifurcating solutions after the restriction of symmetries. We underline that in this article we restrict our attention only to the existence of global bifurcations of solutions of system (4.1). It is not our purpose to study the symmetry breaking phenomenon. We study the symmetry breaking of solutions of elliptic differential equations in [47]. 4. Bifurcations of solutions of non-cooperative elliptic systems Consider the following system of elliptic equations



A1u(x) = ∇u F (u(x), λ) u=0

in Ω , on ∂ Ω ,

(4.1)

where

Ω ⊂ RN is an open, bounded and G-invariant subset of an orthogonal G-representation RN , with smooth boundary, F ∈ C 2 (Rm × R, R), ∇u F (u, λ) = λu + ∇u g (u, λ), where ∇u g (0, λ) = 0, ∇u2 g (0, λ) = 0, for all λ ∈ R, there are C > 0 and 1 ≤ p < (N + 2)(N − 2)−1 s.t. |∇u2 F (u, λ)| ≤ C (1 + |u|p−1 ) (for N = 2 we assume that p ∈ [1, +∞)), (e5) A = diag (α1 , . . . , αm ), where αi ∈ {−1, +1}, i = 1, . . . , m. (e1) (e2) (e3) (e4)

Denote by n− , p+ the number of elements ‘‘−1’’, ‘‘+1’’ on the main diagonal of the matrix A, respectively. m 1 In this section we study global bifurcations of weak solutions of system (4.1). Consider a Hilbert space H = i=1 H0 (Ω ) ∑m with an inner product ⟨u1 , u2 ⟩H = ⟨ u , u ⟩ .( H , ⟨·, ·⟩ ) is an orthogonal G-representation with a G-action 1 1 , i 2 , i H i =1 H (Ω ) 0

defined by (gu)(x) = u(g −1 x), for all (u, g ) ∈ H × G. Define a functional Φ ∈ CG2 (H × R, R) as follows

Φ (u(x), λ) =

1

∫ − m 

2

Ω i =1

We have ⟨∇u Φ (u, λ), ϕ⟩H =

 λ −αi |∇ ui (x)|2 dx − 2

∑m

i =1

∫ Ω

|u(x)|2 dx −

∫ Ω

g (u(x), λ) dx.

(4.2)

−αi ⟨ui , ϕi ⟩H1 (Ω ) − ⟨λK (u), ϕ⟩H − ⟨∇u η0 (u, λ), ϕ⟩H for u, ϕ ∈ H.

Summing up, we obtain the following

0

∇u Φ (u, λ) = L(u) − ∇u η(u, λ) = −A(u) − λK (u) − ∇u η0 (u, λ) = (−α1 u1 − λ(−∆)−1 (u1 ), . . . , −αm um − λ(−∆)−1 (um )) − ∇u η0 (u, λ),

(4.3)

where (1) L = −A : H → H is a linear, self-adjoint, bounded, G-equivariant Fredholm operator, (2) K = ((−∆)−1 , . . . , (−∆)−1 ) : H → H is a linear, self-adjoint, compact, bounded, G-equivariant operator, (3) ∇u η0 : H × R → H is a compact, G-equivariant, gradient operator such that ∇u η0 (0, λ) = 0, ∇u2 η0 (0, λ) = 0, for every λ ∈ R. Let σ (−∆; Ω ) = {λ1 < λ2 5 · · ·} be the set of eigenvalues of the problem



−1u = λu u=0

in Ω , Moreover, denote by V−∆ (λk0 ) on ∂ Ω .

the eigenspace of −∆ corresponding to λk0 . Define a G-equivariant approximation scheme Γ = {πn : H → H : n ∈ N ∪ {0}} on H by (1) H0 =  {0},  n m (2) Hn = k=1 i=1 V−∆ (λk ), for n ∈ N, (3) πn : H → H a natural projection such that im πn = Hn , for n ∈ N. Assumptions (a2), (a5) are fulfilled and LK = KL. Set σ − (−∆; Ω ) = {−λ0 ∈ R : λ0 ∈ σ (−∆; Ω )}. Remark 4.1. Taking into account (4.3) it is easy to verify that

 (1) P (Φ ) =

{0} × σ (−∆; Ω ) {0} × σ − (−∆; Ω ) {0} × (σ (−∆; Ω ) ∪ σ − (−∆; Ω ))

1 (2) σ (K ) = {λ− : λi ∈ σ (−∆; Ω )}, i  m −1 (3) VK (λi ) = i=1 V−∆ (λi ).

if n− > 0, p+ = 0, if n− = 0, p+ > 0, if n− · p+ = ̸ 0,

1830

A. Gołebiewska, ¸ S. Rybicki / Nonlinear Analysis 74 (2011) 1823–1834

Moreover, if (0, λ0 ) ∈ P (Φ ) then

n

− (1) if λ0 > 0, then n− > 0, λ0 ∈ σ (−∆; Ω ) and ker ∇u2 Φ (0, λ0 ) = i=1 V−∆ (λ0 ),  p+ 2 (2) if λ0 < 0, then p+ > 0, −λ0 ∈ σ (−∆; Ω ) and ker ∇u Φ (0, λ0 ) = i=1 V−∆ (−λ0 ).

Set V(n) = k=1 V−∆ (λk ). It is understood that V(0) is a two-dimensional trivial G-representation. In the two lemmas below we derive formulas for the bifurcation index BIF G (λ0 ) ∈ U (G), where λ0 ∈ σ (−∆; Ω ) ∪ σ − (−∆; Ω ).

n

Lemma 4.1. Assume that n− > 0 and fix λk0 ∈ σ (−∆; Ω ). Then,

BIF G (λk0 ) = [∇G -deg (−Id, B (V(k0 − 1)))]n− ⋆

n−     −I . ∇G -deg −Id, B V−∆ (λk0 )

If additionally G = SO(2) then (1) if dim V−∆ (λk0 ) is even and n− · dim V(k0 − 1) is even then

BIF G (λk0 ) =



n−    − I ∈ U+ (G), ∇G -deg −Id, B V−∆ (λk0 )

(2) if dim V−∆ (λk0 ) is even and n− · dim V(k0 − 1) is odd then

n− 

BIF G (λk0 ) = I − ∇G -deg −Id, B V−∆ (λk0 )

∈ U− (G),    n−  (3) if n− is even then BIF G (λk0 ) = ∇G -deg −Id, B V−∆ (λk0 ) − I ∈ U+ (G). 







Proof. From Remark 4.1 it follows that ∇ 2 Φ (0, λk0 ± ϵ) = L − (λk0 ± ϵ)K : H → H is an isomorphism for sufficiently small ϵ > 0. Taking into account that σ (−∆; Ω ) ∩ (−∞, 0) = ∅ for n > k0 we obtain

 −1 ∇G -deg(∇ 2 Φ (0, λk0 ± ϵ), B(H)) = ∇G -deg(L, B(Hn )) ⋆ ∇G -deg(L − (λk0 ± ϵ)K , B(Hn )) = [∇G -deg (−Id, B (V(n)))]−p+ ⋆ ∇G -deg(−A − (λk0 ± ϵ)K , B(Hn )) ∏ = [∇G -deg (−Id, B (V(n)))]−p+ ⋆ ∇G -deg(−αi Id − (λk0 ± ϵ)(−∆)−1 , B(V(n))) α i >0

⋆

∏

∇G -deg(−αi Id − (λk0 ± ϵ)(−∆)−1 , B(V(n)))

α i <0

= [∇G -deg (−Id, B (V(n)))]−p+ ⋆ [∇G -deg (−Id, B (V(n)))]p+ ⋆ [∇G -deg(Id − (λk0 ± ϵ)(−∆)−1 , B(V(n)))]n−   n− = ∇G -deg Id − (λk0 ± ϵ)(−∆)−1 , B(V(n)) . Note that

  ∇G -deg Id − (λk0 + ϵ)(−∆)−1 , B (V(n)) = ∇G -deg (−Id, B (V(k0 )))   ∇G -deg Id − (λk0 − ϵ)(−∆)−1 , B (V(n)) = ∇G -deg (−Id, B (V(k0 − 1))) .

(4.4) (4.5)

Summing up, from (4.4) and (4.5) we obtain

BIF G (λk0 ) = [∇G -deg (−Id, B (V(k0 )))]n− − [∇G -deg (−Id, B (V(k0 − 1)))]n−

= [∇G -deg (−Id, B (V(k0 − 1)))]n− ⋆



  n−  ∇G -deg −Id, B V−∆ (λk0 ) −I .

Finally, if G = SO(2) then applying Lemmas 2.11 and 3.5 of [48] we complete the proof.



The proof of the lemma below is close in spirit to that of Lemma 4.1. That is why we omit it. Lemma 4.2. Assume that p+ > 0 and fix λk0 ∈ σ (−∆; Ω ). Then

BIF G (−λk0 ) = [∇G -deg (−Id, B(V(k0 )))]−p+ ⋆



  p+  ∇G -deg −Id, B V−∆ (λk0 ) −I .

If additionally G = SO(2) then (1) if dim V−∆ (λk0 ) is even and p+ · dim V(k0 ) is even then

BIF G (−λk0 ) =



  p+  ∇G -deg −Id, B V−∆ (λk0 ) − I ∈ U+ (G),

(2) if dim V−∆ (λk0 ) is even and p+ · dim V(k0 ) is odd then

p+ 

BIF G (−λk0 ) = I − ∇G -deg −Id, B V−∆ (λk0 )





(3) if p+ is even then BIF G (−λk0 ) =

∈ U− (G), p+  ∇G -deg −Id, B V−∆ (λk0 ) − I ∈ U+ (G). 









It is of interest to formulate easy to verify conditions which imply BIF G (λk0 ) ̸= Θ ∈ U (G). In the lemma below we formulate such conditions in terms of the of eigenspaces of the Laplace operator −∆.

A. Gołebiewska, ¸ S. Rybicki / Nonlinear Analysis 74 (2011) 1823–1834

1831

Lemma 4.3. Let G be connected. If λk0 ∈ σ (−∆; Ω ) then (1) (2) (3) (4)

if if if if

n− n− p+ p+

> 0 is even then BIF G (λk0 ) ̸= Θ ∈ U (G) iff V−∆ (λk0 ) is a nontrivial G-representation, > 0 is odd then BIF G (λk0 ) ̸= Θ ∈ U (G) iff dim V−∆ (λk0 ) is odd or V−∆ (λk0 ) is a nontrivial G-representation, > 0 is even then BIF G (−λk0 ) ̸= Θ ∈ U (G) iff V−∆ (λk0 ) is a nontrivial G-representation, > 0 is odd then BIF G (−λk0 ) ̸= Θ ∈ U (G) iff dim V−∆ (λk0 ) is odd or V−∆ (λk0 ) is a nontrivial G-representation.

Proof. First of all note that from Lemma 4.1 we obtain that

BIF G (λk0 ) = [∇G -deg (−Id, B (V(k0 − 1)))]n− ⋆

n−     −I . ∇G -deg −Id, B V−∆ (λk0 )

(1) (⇒) Suppose, contrary to our claim, that BIF G (λk0 ) ̸= Θ ∈ U (G) and that V−∆ (λk0 ) is a trivial G-representation. Since n− is even, we obtain

n−    ∇G -deg −Id, B V−∆ (λk0 ) = (−1)n− ·dim V−∆ (λk0 ) · χG (G/G+ ) = χG (G/G+ ) = I. Thus BIF G (λk0 ) = Θ ∈ U (G), a contradiction. (1)(⇐) Since group G is connected, every element of G is contained in a maximal torus. Hence there is a maximal torus T ⊂ G such that V−∆ (λk0 ) is a nontrivial T -representation. It follows that there is a homomorphism ϕ : SO(2) → G which defines a nontrivial SO(2)-action on V−∆ (λk0 ). Homomorphism ϕ induces a ring homomorphism ϕ ∗ : U (G) → U (SO(2)), see [33], such that ϕ ∗ (BIF G (λk0 )) = BIF SO(2) (λk0 ). Since V−∆ (λk0 ) is a nontrivial SO(2)-representation, BIF SO(2) (λk0 ) ̸= Θ ∈ U (SO(2)), see [48], which implies that BIF G (λk0 ) ̸= Θ ∈ U (G). (2) (⇒) Suppose, contrary to our claim, that BIF G (λk0 ) ̸= Θ ∈ U (G) and that V−∆ (λk0 ) is a trivial, even-dimensional G-representation. Therefore

   n− ∇G -deg −Id, B V−∆ (λk0 ) = (−1)n− ·dim V−∆ (λk0 ) · χG (G/G+ ) = χG (G/G+ ) = I. Hence BIF G (λk0 ) = Θ ∈ U (G), a contradiction. (2)(⇐) Consider the homomorphism ϕ : SO(2) → G defined in the proof of (1). Since V−∆ (λk0 ) is an odd-dimensional or nontrivial SO(2)-representation, BIF SO(2) (λk0 ) = ϕ ∗ (BIF G (λk0 )) ̸= Θ ∈ U (SO(2)), see [48], which implies that BIF G (λk0 ) ̸= Θ ∈ U (G). The proofs of (3) and (4) are the same as that of (1) and (2), respectively.  The following two corollaries are versions of the Rabinowitz alternative characterising continua of weak solutions of system (4.1) bifurcating from the set of trivial ones. To prove these corollaries it is enough to combine Theorem 3.3 and Lemma 4.3. Corollary 4.1. Let G be connected. If λk0 ∈ σ (−∆; Ω ) and one of the following conditions is satisfied: (1) n− > 0 is even and V−∆ (λk0 ) is a nontrivial G-representation, (2) n− > 0 is odd and V−∆ (λk0 ) is an odd-dimensional or nontrivial G-representation, then from (0, λk0 ) ∈ H × R bifurcates a continuum C (λk0 ) ⊂ H × R which (1) either is unbounded, (2) or is bounded and moreover the following conditions are satisfied: (a) C (λk0 ) ∩ {0} × R = {(0, λk0 ), . . . , (0, λkp )} ⊂ P (Φ ), ∑p (b) i=0 BIF G (λki ) = Θ ∈ U (G). Corollary 4.2. Let G be connected. If λk0 ∈ σ (−∆; Ω ) and one of the following conditions is satisfied: (1) p+ > 0 is even and V−∆ (λk0 ) is a nontrivial G-representation, (2) p+ > 0 is odd and V−∆ (λk0 ) is an odd-dimensional or nontrivial G-representation, then from (0, −λk0 ) ∈ H × R bifurcates a continuum C (−λk0 ) ⊂ H × R which (1) either is unbounded, (2) or is bounded and moreover the following conditions are satisfied: (a) C (−λk0 ) ∩ {0} × R = {(0, −λk0 ), (0, λk1 ), . . . , (0, λkp )} ⊂ P (Φ ), ∑p (b) BIF G (−λk0 ) + i=1 BIF G (λki ) = Θ ∈ U (G). In the two theorems below we characterise bounded continua of weak solutions of system (4.1) bifurcating from the trivial line. Theorem 4.1. Let G be connected and λk0 ∈ σ (−∆; Ω ). Moreover, assume that n− > 0 is even and that V−∆ (λk0 ) is a nontrivial G-representation. If the continuum C (λk0 ) ⊂ H × R is bounded, then p+ > 0 is odd and C (λk0 ) ∩ {(0, λ0 ) ∈ H × (−∞, 0) : λ0 ∈ σ − (−∆; Ω )} ̸= ∅. Proof. Note that the assumptions of Corollary 4.1 are fulfilled and therefore the continuum C (λk0 ) satisfies the assertion of this theorem. Since G is connected and V−∆ (λk0 ) is a nontrivial G-representation there is a homomorphism ϕ : SO(2) → G such that V−∆ (λk0 ) is a nontrivial SO(2)-representation with SO(2)-action given by (g · u)(x) = u(ϕ(g )−1 x), where g ∈ SO(2)

1832

A. Gołebiewska, ¸ S. Rybicki / Nonlinear Analysis 74 (2011) 1823–1834

and u ∈ V−∆ (λk0 ). The homomorphism ϕ : SO(2) → G induces a homomorphism ϕ ∗ : U (G) → U (SO(2)) (see [33]) such that ϕ ∗ (BIF G (λ0 )) = BIF SO(2) (λ0 ) ∈ U (SO(2)) for every λ0 ∈ σ (−∆; Ω ) ∪ σ − (−∆; Ω ). Note that since V−∆ (λk0 ) is a nontrivial SO(2)-representation,

BIF SO(2) (λk0 ) ̸= Θ ∈ U (SO(2));

(4.6)

see [48]. Now suppose, contrary to our claim, that the continuum C (λk0 ) is bounded and p+ = 0 or C (λk0 ) ∩ {(0, λ0 ) ∈ H × (−∞, 0) : λ0 ∈ σ − (−∆; Ω )} = ∅. By Corollary 4.1 we obtain the following C (λk0 ) ∩ ({0} × R) = {(0, λk0 ), (0, λk1 ), . . . , (0, λkp )} ⊂ P (Φ ), and p −

BIF G (λki ) = Θ ∈ U (G).

(4.7)

i=0

∑p

∑p

∗ If p+ = 0 then P (Φ ) = {0} × σ (−∆; Ω ) and by Lemma 4.1 we obtain ϕ ∗ BIF G (λki ) = i=0 ϕ i=0 BIF G (λki ) = ∑p BIF SO(2) (λki ) ∈ U+ (SO(2)). Taking into consideration the above expression and inequality (4.6) we obtain i = 0 ∑p ∈ U (SO(2)), which If p+ > 0 is even then by Lemmas 4.1 and 4.2 it follows (2) (λki ) ̸= Θ  i=0 BIF  (4.7). ∑SO  contradicts ∑p ∑p p ∗ that ϕ ∗ = BIF G (λki ) = i=0 ϕ i = 0 BIF SO(2) (λki ) ∈ U+ (SO(2)). Taking into account the i=0 BIF G (λki ) ∑p above expression and inequality (4.6) we obtain i=0 BIF SO(2) (λki ) ̸= Θ ∈ U (SO(2)), which contradicts (4.7). If C (λk0 ) ∩ {(0, λ0 ) ∈ H × (−∞, 0) : λ0 ∈ σ − (−∆; Ω )} = ∅ then C (λk0 ) ∩ {0} × R = {(0, λk0 ), (0, λk1 ), . . . , (0, λkp )} ⊂







∑p  ∑p   ∑p ∗ {0} × σ (−∆; Ω ) and by Lemma 4.1 we obtain ϕ ∗ BIF G (λki ) = i=0 BIF SO(2) (λki ) ∈ i=0 BIF G (λki ) = i =0 ϕ ∑ p U+ (SO(2)). Taking into consideration the above expression and (4.6) we obtain i=0 BIF SO(2) (λki ) ̸= Θ ∈ U (SO(2)), which contradicts (4.7).



Theorem 4.2. Let G be connected. Moreover, assume that λk0 ∈ σ (−∆; Ω ), p+ > 0 is even and that V−∆ (λk0 ) is a nontrivial G-representation. If the continuum C (−λk0 ) ⊂ H × R is bounded, then n− > 0 is odd and C (−λk0 ) ∩ {(0, λ0 ) ∈ H × (0, ∞) : λ0 ∈ σ (−∆; Ω )} ̸= ∅. Proof. The proof is in fact the same as that of Theorem 4.1.



As a direct consequence of Theorems 4.1 and 4.2 we obtain the following corollary in which we have excluded one of the possibilities of behaviour of continua C (λk0 ), C (−λk0 ) ⊂ H × R in the Rabinowitz alternative. Corollary 4.3. Assume that n− , p+ > 0 are even and fix λk0 ∈ σ (−∆; Ω ) such that V−∆ (λk0 ) is a nontrivial G-representation. Then continua C (λk0 ), C (−λk0 ) ⊂ H × R are unbounded. To illustrate the abstract results we consider system (4.1) with G = SO(3) and Ω = B3 ⊂ R3 . It is known that dim V−∆ (λk0 ) is odd for every λk0 ∈ σ (−∆; B3 ). Moreover, if dim V−∆ (λk0 ) > 1 then V−∆ (λk0 ) is a nontrivial SO(3)representation. Example 4.1. Fix λk0 ∈ σ (−∆; B3 ) such that dim V−∆ (λk0 ) > 1. Since V−∆ (λk0 ) is a nontrivial SO(3)-representation, applying Corollaries 4.1 and 4.2 we obtain that from (0, ±λk0 ) ∈ P (Φ ) ⊂ H × R bifurcates a continuum C (±λk0 ) ⊂ H × R which either is unbounded or is bounded, and moreover the following conditions are satisfied: (1) C (±λk0 ) ∩ {0} × R = {(0, ±λk0 ), . . . , (0, λkp )} ⊂ P (Φ ) ⊂ H × R, ∑p (2) i=0 BIF SO(3) (λki ) = Θ ∈ U (SO(3)). Now suppose that dim V−∆ (λk0 ) = 1. If n− is odd then from Corollary 4.1 we obtain that from the point (0, λk0 ) ∈ P (Φ ) ⊂ H × R bifurcates a continuum C (λk0 ) ⊂ H × R which either is unbounded or is bounded, and moreover the following conditions are satisfied: (1) C (λk0 ) ∩ {0} × R = {(0, λk0 ), . . . , (0, λkp )} ⊂ P (Φ ) ⊂ H × R, ∑p (2) i=0 BIF SO(3) (λki ) = Θ ∈ U (SO(3)). If p+ is odd then applying Corollary 4.2 one can formulate the same conclusion as above for the point (0, −λk0 ) ∈ P (Φ ). Example 4.2. Let n− , p+ > 0 be even. Fix λk0 ∈ σ (−∆; B3 ) satisfying dim V−∆ (λk0 ) > 1. Since V−∆ (λk0 ) is a nontrivial SO(3)-representation, applying Corollary 4.3 we obtain that continua C (±λk0 ) ⊂ H × R are unbounded. 5. Final remarks Let Φ ∈ CG1 (H, R) and Ω ⊂ H satisfy assumptions (a1)–(a4) of Section 3. Assume additionally that L : H → H is an isomorphism. Then the equation

∇ Φ (u) = L(u) − ∇η(u) = 0,

(5.1)

is equivalent to the equation

Ψ (u) = (L−1 ◦ ∇ Φ )(u) = 0.

(5.2)

A. Gołebiewska, ¸ S. Rybicki / Nonlinear Analysis 74 (2011) 1823–1834

1833

We can study the orbits of solutions of Eq. (5.1) using the degree for G-invariant strongly indefinite functionals

∇G -deg(∇ Φ , Ω ) ∈ U (G). Note that Ψ ∈ CG0 (H, H) is a G-equivariant operator of the form compact perturbation of the identity. We underline that, generally speaking, Ψ is not a gradient operator! Nevertheless to study orbits of solutions of Eq. (5.2) we can apply the Leray–Schauder degree degLS (Ψ , Ω , 0) ∈ Z.

We point out that the two above described approaches are not equivalent because we claim that it can happen that degLS (Ψ , Ω , 0) = 0 ∈ Z whereas ∇G -deg(∇ Φ , Ω ) ̸= Θ ∈ U (G). In fact if G is a torus and Ω G = ∅ then degLS (Ψ , Ω , 0) = 0 ∈ Z; see [40,41]. On the other hand if (∇ Φ )−1 (0) ∩ Ω consist of exactly one non-degenerate orbit then ∇G -deg(∇ Φ , Ω ) ̸= Θ ∈ U (G). Let system



A1u(x) = ∇u F (u(x), λ) u=0

in Ω , on ∂ Ω ,

(5.3)

satisfy assumptions of Section 4. In Section 4 we have applied the degree for G-equivariant strongly indefinite functionals to the study of global bifurcations of orbits of weak solutions of system (5.3). Namely, we have studied solutions of equation

∇u Φ (u, λ) = L(u) − ∇u η(u, λ) = 0,

(5.4)

where Φ is given by (4.2). System (5.3) is equivalent to the following system



−1u(x) = (−A)∇u F (u(x), λ) u=0

in Ω , on ∂ Ω .

(5.5)

In order to study orbits of weak solutions of system (5.5) we have to study orbits of solutions of the following equation

Ψ (u, λ) = (L−1 ◦ ∇u Φ )(u, λ).

(5.6)

Note that Ψ ∈ CG1 (H × R, H) is a family of G-equivariant operators of the form compact perturbation of the identity. We point out that, generally speaking, Ψ is not a family of gradient operators! Therefore to the study of orbits of solutions of Eq. (5.4) we cannot apply the degree for G-equivariant gradient operators of the form compact perturbation of the identity. However we can apply the Leray–Schauder degree. Also in this case the results obtained in this article for system (5.3) are stronger than the results obtained by application of the classical Rabinowitz alternative to system (5.5). Moreover, the results of obtained in this article are not consequences of [49,50], because in these articles we have studied gradient and equivariant operators of the form compact perturbation of the identity. Acknowledgements The second author was partially supported by the Ministry of Education and Science, Poland, under grant no. N201 385534 and by the Japan Society for the Promotion of Science, under fellowship L-10515. References [1] A. Abbondandolo, A new cohomology for the Morse theory of strongly indefinite functionals on Hilbert spaces, Topol. Methods Nonlinear Anal. 9 (2) (1997) 325–382. [2] A. Abbondandolo, Morse Theory for Hamiltonian Systems, in: Chapman & Hall/CRC Research Notes in Mathematics, vol. 425, Boca Raton, FL, 2001. [3] Z. Balanov, Y. Schwartzman, Morse complex, even functionals and buckling of a thin elastic plate, C. R. Acad. Sci. Paris Ser. I Math. 320 (3) (1995) 273–278. [4] Z. Balanov, Y. Schwartzman, Morse complex, even functionals and asymptotically linear differential equations with resonance at infinity, Topol. Methods Nonlinear Anal. 12 (2) (1998) 323–366. [5] K.C. Chang, Infinite-Dimensional Morse Theory and Multiple Solution Problems, in: Progr. in Nonl. Diff. Equat. and their Appl., vol. 6, Birkhäuser Boston Inc., Boston, MA, 1993. [6] W. Kryszewski, A. Szulkin, An infinite dimensional Morse theory with applications, Trans. Amer. Math. Soc. 349 (8) (1997) 3181–3234. [7] S.J. Li, J.Q. Liu, Morse theory and asymptotic linear Hamiltonian system, J. Differential Equations 78 (1) (1989) 53–73. [8] A. Szulkin, Cohomology and Morse theory for strongly indefinite functionals, Math. Z. 209 (2) (1992) 375–418. [9] A. Szulkin, Bifurcation for strongly indefinite functionals and a Liapunov type theorem for Hamiltonian systems, Differential Integral Equations 7 (1) (1994) 217–234. [10] K. Geba, ¸ M. Izydorek, A. Pruszko, The Conley index in Hilbert spaces and its applications, Studia Math. 134 (3) (1999) 217–233. [11] M. Izydorek, A cohomological Conley index in Hilbert spaces and applications to strongly indefinite problems, J. Differential Equations 170 (1) (2001) 22–50. [12] V. Benci, P. Rabinowitz, Critical point theorems for indefinite functionals, Invent. Math. 52 (3) (1979) 241–273. [13] D.G. Costa, C.A. Magalhães, A unified approach to a class of strongly indefinite functionals, J. Differential Equations 125 (2) (1996) 521–547. [14] P.H. Rabinowitz, Periodic solutions of Hamiltonian systems, Comm. Pure Appl. Math. 32 (1978) 157–184. [15] P.H. Rabinowitz, Minimax Methods in Critical Point Theory with Applications to Differential Equations, in: CBMS Regional Conf. Ser. in Math., vol. 65, AMS, Providence, RI, 1986. [16] E. Ciriza, P.M. Fitzpatrick, J. Pejsachowicz, Uniqueness of spectral flow. Nonlinear operator theory, Math. Comput. Modelling 32 (11–13) (2000) 1495–1501. [17] P.M. Fitzpatrick, J. Pejsachowicz, Spectral flow and bifurcation of critical points of strongly-indefinite functionals part I. General theory, J. Differential Equations 162 (1) (1999) 52–95.

1834

A. Gołebiewska, ¸ S. Rybicki / Nonlinear Analysis 74 (2011) 1823–1834

[18] P.M. Fitzpatrick, J. Pejsachowicz, L. Recht, Spectral flow and bifurcation of critical points of strongly-indefinite functionals part II. Bifurcation of periodic orbits of Hamiltonian systems, J. Differential Equations 163 (1) (2000) 18–40. [19] T. Bartsch, M. Clapp, Critical point theory for indefinite functionals with symmetries, J. Funct. Anal. 138 (1) (1996) 107–136. [20] V. Benci, On critical point theorey of indefinite functionals in the presence of symmetries, Trans. Amer. Math. Soc. 274 (2) (1982) 533–572. [21] M. Izydorek, Equivariant Conley index in Hilbert spaces and applications to strongly indefinite problems, Nonlinear Anal. TMA 51 (1) (2002) 33–66. [22] Z.Q. Wang, Equivariant Morse theory for isolated critical orbits and its applications to nonlinear problems, in: Part. Diff. Equat, Tianjin, 1986, in: Lecture Notes in Math., vol. 1306, Springer, Berlin, 1988, pp. 202–223. [23] A. Ambrosetti, Branching points for a class of variational operators, J. Anal. Math. 76 (1998) 321–335. [24] R. Böhme, Die Lösung der versweigungsgleichungen für nichtlineare eigenwert-probleme, Math. Z. 127 (1972) 105–126. [25] E.N. Dancer, A new degree for S 1 -invariant mappings and applications, Ann. Inst. H. Poincaré Anal. Non Linéaire 2 (1985) 473–486. [26] J. Ize, Topological bifurcation, in: M. Matzeu i, A. Vignoli (Eds.), Topological Nonlinear Analysis, Degree, Singularity and Variations, in: Progr. Nonl. Diff. Equat. Appl., vol. 15, Birkhäuser, 1995, pp. 341–463. [27] A. Marino, La biforcazione nel caso variazionale, Conf. Sem. Mat. Univ. Bari 132 (1977). [28] F. Takens, Some remarks on the Böhme–Berger bifurcation theorem, Math. Z. 125 (1972) 359–364. [29] S. Rybicki, Degree for S 1 -equivariant strongly-indefinite functionals, Nonlinear Anal. TMA 43 (8) (2001) 1001–1017. [30] K. Geba, ¸ Degree for gradient equivariant maps and equivariant Conley index, in: M. Matzeu i, A. Vignoli (Eds.), Birkhäuser, Topological Nonlinear Analysis, Degree, Singularity and Variations, in: Progr. in Nonl. Diff. Equat. and their Appl., vol. 27, Birkhäuser, 1997, pp. 247–272. [31] S. Rybicki, Degree for equivariant gradient maps, Milan J. Math. 73 (2005) 103–144. [32] T. tom Dieck, Transformation Groups and Representation Theory, in: Lecture Notes in Mathematics, vol. 766, Springer Verlag, Berlin, New York, Heidelberg, 1979. [33] T. tom Dieck, Transformation Groups, Walter de Gruyter, Berlin, New York, 1987. [34] S. Rybicki, A degree for S 1 -equivariant orthogonal maps and its applications to bifurcation theory, Nonlinear Anal. TMA 23 (1) (1994) 83–102. [35] E.N. Dancer, J.F. Toland, The index change and global bifurcation for flows with a first integral, Proc. Lond. Math. Soc. (3) 66 (3) (1993) 539–567. [36] Z. Balanov, W. Krawcewicz, H. Steinlein, Applied equivariant degree, in: AIMS Series on Diff. Equat.& Dyn. Sys., Springfield, 2006. [37] H. Ruan, S. Rybicki, Applications of equivariant degree for gradient maps to symmetric Newtonian systems, Nonlinear Anal. TMA 68 (6) (2008) 1479–1516. [38] Z. Balanov, W. Krawcewicz, H. Ruan, Periodic solutions of O(2)-symmetric variational problems: O(2) × S 1 -equivariant gradient degree approach, in: Conf. on Nonlin. Analysis and Optimization, 2008, Haifa, Israel, in: Israel Math. Con. Proc., AMS Contemporary Mathematics Series, vol. 514, 2010, pp. 45–84. [39] J. Ize, A. Vignoli, Equivariant degree theory, in: de Gruyter Series in Nonlinear Analysis and Applications, vol. 8, Walter de Gruyter, Berlin, New York, 2003. [40] E.N. Dancer, Symmetries, degree, homotopy indices and asymptotically homogeneous problems, Nonlinear Anal. TMA 6 (7) (1982) 667–686. [41] A. Kushkuley, Z. Balanov, Geometric Methods in Degree Theory for Equivariant Maps, in: Lect. Notes in Math., vol. 1632, Springer-Verlag, Berlin, 1996. [42] R.F. Brown, A Topological Introduction to Nonlinear Analysis, Birkhäuser, Boston, 1993. [43] K. Deimling, Nonlinear Functional Analysis, Springer Verlag, Berlin, Heidelberg, 1985. [44] J. Ize, Bifurcation theory for Fredholm operators, Mem. Amer. Math. Soc. 174 (1976). [45] P.H. Rabinowitz, Some global results for nonlinear eigenvalue problems, J. Funct. Anal. 7 (1971) 487–513. [46] G. López Garza, S. Rybicki, Equivariant bifurcation index, Nonlinear Anal. TMA 73 (7) (2010) 2779–2791. [47] S. Rybicki, Global bifurcations of critical orbits via equivariant Conley index (2010) (submitted for publication). [48] S. Rybicki, Bifurcations of solutions of SO(2)-symmetric nonlinear problems with variational structure, in: R. Brown, M. Furi, L. Górniewicz, B. Jiang (Eds.), Handbook of Topological Fixed Point Theory, Springer, 2005, pp. 339–372. [49] J. Gawrycka, S. Rybicki, Solutions of multiparameter systems of elliptic differential equations, Adv. Nonlinear Stud. 5 (2) (2005) 279–302. [50] J. Gawrycka, S. Rybicki, Solutions of systems of elliptic differential equations on circular domains, Nonlinear Anal. 59 (8) (2004) 1347–1367.