Sign-changing solutions of nonlinear Schrödinger system

J. Math. Anal. Appl. 481 (2020) 123478

Contents lists available at ScienceDirect

Journal of Mathematical Analysis and Applications www.elsevier.com/locate/jmaa

Sign-changing solutions of nonlinear Schrödinger system ✩ Xiangqing Liu a , Bingqin Qiu a , Zhaosheng Feng b,∗ a b

Department of Mathematics, Yunnan Normal University, Kunming, Yunnan 650500, China Department of Mathematics, University of Texas Rio Grande Valley, Edinburg, TX 78539, USA

a r t i c l e

i n f o

a b s t r a c t

Article history: Received 31 January 2019 Available online 5 September 2019 Submitted by G. Chen

In this paper, we are concerned with the existence of infinitely many sign-changing solutions of nonlinear Schrödinger systems in RN by applying the perturbation method and the method of invariant sets of descending flow. The main feature which distinguishes this paper from other related works (such as [13]) lies in the fact that coefficients of nonlinearities are dependent on x rather than constants. © 2019 Elsevier Inc. All rights reserved.

Keywords: Schrödinger system Sign-changing solutions Lipschitz continuity Perturbation Hölder’s inequality

1. Introduction Consider the Schrödinger system: 

−u + λ1 u = β11 (x)u3 + β12 (x)v 2 u, x ∈ RN , −v + λ2 v = β12 (x)u2 v + β22 (x)v 3 , x ∈ RN ,

(1.1)

where N = 2, 3, λ1 and λ2 are positive constants, and βij (i, j = 1, 2) are real functions. Such a class of systems, also known as Gross-Pitaevskii equations, has tremendous applications in scientific areas such as nonlinear optics and multispecies Bose-Einstein condensates [1–3]. A large number of theoretical issues concerning nonlinear systems arising from physics and engineering has received considerable attention, and some profound results on a priori bounds and multiple existence of positive solutions have been established, see [4–12], and the references therein. To the best of our knowledge, for system (1.1) it does not seem that the existence of sign-changing solutions in the whole space RN has been presented previously. ✩

This work is supported by NSF of China 11761082.

* Corresponding author. E-mail address: [email protected] (Z. Feng). https://doi.org/10.1016/j.jmaa.2019.123478 0022-247X/© 2019 Elsevier Inc. All rights reserved.

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

2

Recently, Liu et al. [13] considered a Schrödinger system of the form: ⎧ ⎨ ⎩

−uj + λj uj =

k  i=1

βij u2i uj , x ∈ RN ,

(1.2)

uj (x) → 0, as |x| → ∞, j = 1, · · · , k,

where λj > 0 (j = 1, ..., k), βij are constants satisfying βjj > 0 (j = 1, ..., k) and βij = βji ≤ 0 (1 ≤ i < j ≤ k). A general critical point theory was developed to deal with the existence and locations of multiple critical points produced by the minimal methods in relation to multiple invariant sets of the associated gradient flow. Later on, they extended the study to the nonlinear Schrödinger systems in RN [14]: ⎧ ⎨ ⎩

−Δuj + λj (x)uj =

k  i=1

βij u2i uj , x ∈ RN ,

(1.3)

uj (x) → 0, as |x| → ∞, j = 1, · · · , k,

where λj (x) (j = 1, · · · , k) are potential functions with finite depth, βij (i, j = 1, · · · , k) are constants satisfying βij = βji , βjj > 0 and βij ≤ 0 for i = j. It is shown that system (1.3) has an unbounded sequence of sign-changing solutions provided that potential functions λj (x) satisfy the given decay assumptions. In the present paper, we are concerned with the existence of infinitely many sign-changing solutions for a more general case of system (1.1) in the whole space RN , while βij (i, j = 1, 2) are real functions satisfying certain conditions. This makes the problem more difficult and challenging, because the existing variational methods cannot be applied to system (1.1) with variable coefficients directly [15–18]. The purpose of this study is to use Theorem 2.5 of [13] to establish some novel results on infinitely many sign-changing solutions of nonlinear Schrödinger-Poisson system. For convenience of readers, let us recall [13, Theorem 2.5] before stating our main results. [Theorem 2.5, [13]] Let X be a complete metric space with an isometric involution G, Pi , i = 1, ..., k being open subsets of X. Denote Qi = GPi , i = 1, ..., k, M = ∩ki=1 (Pi ∩ Qi ), Σ = ∩ki=1 (∂Pi ∩ ∂Qi ), and W = ∪ki=1 (Pi ∪Qi ). Let f be a G−invariant continuous functional on X. Assume that {Pi }k1 is a G-admissible family of invariant sets with respect to f at level c for c ≥ c∗ := inf u∈Σ f (u). Suppose that for any n ∈ N there exists a continuous map ϕ(n) : B nk → X satisfying (1) Denote t = (t1 , ..., tk ) ∈ B nk , t1 , ..., tk ∈ Rn . Then ϕ(n) (t) ∈ Mi := Pi ∩ Qi , if ti = 0, i = 1, ..., k; (2) ϕ(n) (∂B nk ) ∩ M = ∅; (3) c0 := max{supu∈FG f (u), supu∈ϕ(n) (∂B nk ) f (u)} < c∗ ; (4) ϕ(n) (−t) = Gϕ(n) (t), t ∈ B nk , where FG = {u|Gu = u} is the set of fixed points of G. Define cj = inf

sup f (u),

B∈Γj u∈B\W

where Γj = {B | B = ϕ(B nk \ Y ), ϕ ∈ Gn , n ≥ j, open subset − Y = Y ⊂ B nk , γ(Y ) ≤ n − j}, and Gn = {ϕ | ϕ ∈ C(B nk , X), ϕ(−t) = Gϕ(t), t ∈ B nk ; ϕ(t) ∈ Mi , if ti = 0; ϕ|∂B nk = ϕ(n) }. Then cj , j ≥ k + 1, are critical values of f with cj → ∞ and Kcj \ W = ∅. Throughout this paper, we make the following assumption:

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

3

(B) β11 (x), β22 (x) > 0 and β12 (x) ≤ 0. Note that system (1.1) has a variational structure, and the functional is defined by 

1 I(U ) = 2

(|∇u|2 + λ1 u2 + |∇v|2 + λ2 v 2 )dx RN



1 4

−

(β11 (x)u4 + 2β12 (x)u2 v 2 + β22 (x)v 4 )dx. RN

The weak form of system (1.1) is to look for (u, v) ∈ H := H 1 (RN ) × H 1 (RN ) satisfying  (∇u∇ϕ1 + λ1 uϕ1 + ∇v∇ϕ2 + λ2 vϕ2 )dx RN



=

(β11 (x)u3 ϕ1 + β12 (x)u2 vϕ2 + β12 (x)v 2 uϕ1 + β22 (x)v 3 ϕ2 )dx,

RN

for all Φ = (ϕ1 , ϕ2 ) ∈ H. We introduce the inner product  U, V  =

(∇u∇u1 + λ1 uu1 + ∇v∇v1 + λ2 vv1 )dx,

RN

where U = (u, v) and V = (u1 , v1 ) ∈ H with the induced norm  U  =

(|∇u|2 + λ1 u2 + |∇v|2 + λ2 v 2 )dx. RN

Apparently, the norm is equivalent to the usual norm in H. In the following, we use | · |p to denote the Lp norm, and C and c represent different constants used from line to line but independent of the arguments. We say that (u, v) is sign-changing if both u and v are sign-changing. Let us summarize our main results of sign-changing solutions of system (1.1). Theorem 1.1. Suppose that condition (B) holds, and (B1 ) β11 , β12 , β22 ∈ Ls (RN ), s > 1 for N = 2 and s > 3 for N = 3. Then system (1.1) has infinitely many sign-changing solutions. Theorem 1.2. When N = 3, we suppose that condition (B) holds, and the following two conditions are true: (B2 ) β11 , β12 , β22 ∈ L3 (RN ), and (B + ) β11 (x)s2 + 2β12 (x)st + β22 (x)t2 ≥ C|β12 (x)|(s2 + t2 ), x ∈ RN , s ≥ 0, t ≥ 0. Then system (1.1) has infinitely many sign-changing solutions. The rest of the paper is organized as follows. In Section 2, we prove some technical lemmas and then use them as well as the perturbation method to prove Theorem 1.1. Section 3 is dedicated to the proof of Theorem 1.2.

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

4

2. Proof of Theorem 1.1 To prove Theorem 1.1, we consider the existence of critical points of an associated perturbed functional, and then explore the existence of solutions of system (1.1) by applying a convergence argument. ∗ Denote 2∗ = +∞ for N = 2, and 2∗ = 6 for N = 3. Take 4 < p < s−1 s · 2 . For μ ∈ (0, 1], we consider the functional:  μ Iμ (U ) = I(U ) − (β1 (x)|u|p + β2 (x)|v|p )dx, U ∈ H, (2.1) p RN

where β1 (x) = β11 (x) + |β12 (x)| and β2 (x) = β22 (x) + |β12 (x)|, and so β1 , β2 ∈ Ls (RN ). Lemma 2.1. For the fixed μ > 0, Iμ satisfies the (P S) condition. Proof. Assume that {Un } ⊂ H is a (P S) sequence of Iμ , that is, |Iμ (Un )| ≤ C and Iμ (Un ) → 0 as n → ∞. We now prove that {Un } has a convergent subsequence in H. It is not difficult to see that C + o(1)Un  1 ≥Iμ (Un ) − ∇Iμ (Un ), Un  4   1 1 1 = Un 2 + − μ (β1 (x)|Un |p + β2 (x)|Vn |p )dx 4 4 p

(2.2)

RN

1 ≥ Un 2 , 4 which implies that {Un } is bounded in H. Up to a subsequence, we may assume that Un  U in H, Un → U in Ltloc (RN ) × Ltloc (RN ) for t ∈ [2, 2∗ ), and Un (x) → U (x) a.e. x ∈ RN , as n → ∞. So we have o(1) =∇Iμ (Un ) − ∇Iμ (U ), Un − U  

=Un − U 2 − β11 (x)(u3n − u3 )(un − u) + β12 (x)(u2n vn − u2 v)(vn − v) RN

 + β12 (x)(vn2 un − v 2 u)(un − u) + β22 (x)(vn3 − v 3 )(vn − v) dx 

−μ β1 (x)(|un |p−2 un − |u|p−2 u)(un − u) RN

 + β2 (x)(|vn |p−2 vn − |v|p−2 v)(vn − v) dx. Denote A = {x ∈ RN | |x| ≤ R, |β12 (x)| ≤ M }. Since β12 (x) ∈ Ls (RN ), for any ε > 0 there exist two large constants R and M such that ⎛ ⎜ ⎝



RN \A

⎞ 1s ⎟ |β12 (x)|s dx⎠ < ε.

(2.3)

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

5

By virtue of the local convergence and Hölder’s inequality, we deduce that       2 2  β12 (x)(un vn − u v)(vn − v)dx   N  R   2 2 ≤ |β12 (x)(un vn − u v)(vn − v)|dx + |β12 (x)(u2n vn − u2 v)(vn − v)|dx RN \A

⎛

A

⎞ 1s ⎛



⎜ ≤⎝

⎟ ⎜ |β12 (x)|s dx⎠ ⎝

RN \A

⎛

4s ⎟ |un | s−1 dx⎠

RN \A



⎜ ⎝

⎞ s−1 4s 4s ⎟ |vn | s−1 dx⎠

RN \A

⎞ s−1 4s



⎜ ·⎝

⎞ 2(s−1) ⎛ 4s



4s ⎟ |vn − v| s−1 dx⎠

RN \A

⎛ ⎜ +⎝

⎞ 1s ⎛



⎟ ⎜ |β12 (x)|s dx⎠ ⎝

RN \A

⎛ ⎜ ·⎝



⎞ 2(s−1) ⎛ 4s 4s ⎟ |u| s−1 dx⎠

RN \A

⎜ ⎝



⎞ s−1 4s 4s ⎟ |v| s−1 dx⎠

RN \A

⎞ s−1 4s



4s ⎟ |vn − v| s−1 dx⎠

RN \A

⎛ ⎞ 12 ⎛ ⎞ 14 ⎛ ⎞ 14    + M ⎝ |un |4 dx⎠ ⎝ |vn |4 dx⎠ ⎝ |vn − v|4 dx⎠ A

A

A

⎛ ⎞ 12 ⎛ ⎞ 14 ⎛ ⎞ 14    + M ⎝ |u|4 dx⎠ ⎝ |v|4 dx⎠ ⎝ |vn − v|4 dx⎠ A

A

A

≤cε + o(1). Due to arbitrariness of ε, we get  β12 (x)(u2n vn − u2 v)(vn − v)dx = o(1).

(2.4)

RN

Processing in a similar way, one can derive that  β12 (x)(vn2 un − v 2 u)(un − u)dx = o(1),

(2.5)

RN

 β11 (x)(u3n − u3 )(un − u)dx = o(1),

(2.6)

β22 (x)(vn3 − v 3 )(vn − v)dx = o(1).

(2.7)

RN

and  RN

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

6

Using the local convergence and Hölder’s inequality again yields       p−2 p−2   |β (x)|(|u | u − |u| u)(u − u)dx 1 n n n    N  R  ≤ |β1 (x)(|un |p−2 un − |u|p−2 u)(un − u)|dx RN \A



|β1 (x)(|un |p−2 un − |u|p−2 u)(un − u)|dx

+ A





|β1 (x)|(|un |

≤

p−1

RN \A

⎛

⎜ ·⎝



⎞ (p−1)(s−1) ps ps ⎟ |un | s−1 dx⎠

⎞ s−1 ps ps ⎟ |un − u| s−1 dx⎠

⎛



⎛ ⎜ +⎝

RN \A



ps

|u| s−1

⎤ ⎞ (p−1)(s−1) ps ⎥ ⎟ ⎥ dx⎠ ⎦

RN \A

⎡⎛



⎢ + M ⎣⎝

RN \A

·⎝

|β1 (x)|(|un |p−1 + |u|p−1 )|un − u|dx A

RN \A

⎛

)|un − u|dx +

⎞ 1s ⎡⎛  ⎟ ⎢⎜ |β1 (x)|s dx⎠ ⎢ ⎝ ⎣



⎜ ≤⎝

+ |u|

p−1

⎞ p−1 p |un |p dx⎠

⎛ +⎝

A



⎤ ⎞ p−1 p ⎥ |u|p dx⎠ ⎦

A

⎞ p1 |un − u|p dx⎠

(2.8)

A

≤ cε + o(1). Here, A = {x ∈ RN | |x| ≤ R, |β1 (x)| ≤ M }. Since ε is arbitrary, we find  β1 (x)(|un |p−2 un − |u|p−2 u)(un − u)dx = o(1).

(2.9)

β2 (x)(|vn |p−2 vn − |v|p−2 v)(vn − v)dx = o(1).

(2.10)

RN

Similarly, we have  RN

According to (2.3)-(2.10), we obtain Un − U  → 0, as n → ∞. That is, Un → U in H. Hence, Iμ satisfies the (P S) condition. 2 Denote   P = u | u ∈ H 1 (RN ), u(x) ≥ 0 a.e. x ∈ RN .

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

7

For δ > 0, we define   P1± = U | U = (u, v) ∈ H, distH 1 (RN ) (u, ±P ) < δ ,   P2± = U | U = (u, v) ∈ H, distH 1 (RN ) (v, ±P ) < δ ,   W = P1+ ∪ P1− ) ∪ (P2+ ∪ P2− ,   Σ = ∂P1+ ∩ ∂P1− ) ∩ (∂P2+ ∩ ∂P2− . For a function u, we denote u+ = max{u, 0} and u− = min{u, 0}. Define the operator by Aμ : H → H, (u, v) → (w1 , w2 ) = Aμ U satisfying ⎧ 2 3 p−2 N ⎪ ⎨ −w1 + λ1 w1 − β12 (x)v w1 = β11 (x)u + μβ1 (x)|u| u, x ∈ R , ⎪ ⎩ −w + λ w − β (x)u2 w = β (x)v 3 + μβ (x)|v|p−2 v, x ∈ RN . 2 2 2 12 2 22 2

(2.11)

The weak form of system (2.11) is to look for Ψ = (w1 , w2 ) ∈ H and Φ = (ϕ1 , ϕ2 ) ∈ H satisfying ⎧  2 3 p−2 ⎪ ⎨ RN (∇w1 ∇ϕ1 + λ1 w1 ϕ1 − β12 (x)v w1 ϕ1 )dx = RN (β11 (x)u ϕ1 + μβ1 (x)|u| uϕ1 )dx, ⎪ ⎩

(∇w2 ∇ϕ2 + λ2 w2 ϕ2 − β12 (x)u w2 ϕ2 )dx = 2

RN

(2.12)



3

RN

p−2

(β22 (x)v ϕ2 + μβ2 (x)|v|

vϕ2 )dx.

Then Aμ is well-defined and Lipschitz continuous. Lemma 2.2. There exists δ0 > 0 such that for 0 < δ < δ0 , Aμ (P1± ) ⊂ P1± and Aμ (P2± ) ⊂ P2± hold. Proof. Let U = (u, v) ∈ H, Ψ = (w1 , w2 ) = Aμ U. Taking w1+ = ϕ1 in (2.12), we have dist2H 1 (RN ) (w1 , −P ) = inf w1 − ξ2H 1 (RN ) ξ∈−P

= inf w1− + w1+ − ξ2H 1 (RN ) ≤ w1+ 2H 1 (RN ) , ξ∈−P

and 



w1+ 2H 1 (RN ) = RN



=C RN

 |∇w1+ |2 + (w1+ )2 dx ≤ C



  |∇w1+ |2 + λ1 (w1+ )2 dx

RN

  β11 (x)u3 w1+ + β12 (x)v 2 (w1+ )2 + μβ1 (x)|u|p−2 uw1+ dx

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

8





≤C

 β11 (x)(u+ )3 w1+ + μβ1 (x)(u+ )p−1 w1+ dx

RN

⎛ ≤C⎝



⎞ 1s ⎛ |β11 (x)|s dx⎠ ⎝

RN

⎛ +C⎝

⎛ ⎞ 3(s−1) 4s

 |u+ |

4s s−1

dx⎠

RN



⎞ 1s ⎛ β1 (x)s dx⎠ ⎝

RN

≤ C |u+ |3

L

|w1+ |

⎞ (p−1)(s−1) ⎛ sp



(RN )

(RN )

+ distp−1 ps

L s−1 (RN )

|w1+ |

4s L s−1

4s s−1

dx⎠

RN

|u+ | s−1 dx⎠

⎝

ps

RN

4s L s−1

= C dist3 s−1 4s

⎝

⎞ s−1 4s





⎞ s−1 sp |w1+ | s−1 dx⎠ ps

RN

(RN )

(u, −P )dist

+ |u+ |p−1 ps

L s−1 (RN )

4s

L s−1 (RN )

(u, −P )dist

ps L s−1

(RN )

|w1+ |

ps L s−1

 (RN )

(w1 , −P )

 (w1 , −P )

 (u, −P ) distH 1 (RN ) (w1 , −P ). ≤ C dist3H 1 (RN ) (u, −P ) + distp−1 H 1 (RN )

 Take δ0 > 0 such that C δ02 + δ0p−2 < 12 . Let u ∈ P1− , for 0 < δ ≤ δ0 , then we have

distH 1 (RN ) (w1 , −P ) ≤

1 distH 1 (RN ) (u, −P ). 2

Hence, we see that

Aμ (P1− ) ⊂ P1− . Similarly, we can obtain that Aμ (P1+ ) ⊂ P1+ and Aμ (P2± ) ⊂ P2± . 2 Lemma 2.3. There holds  ∇Iμ (U ), U − Aμ U  = U − Aμ U  − 2

  β12 (x) u2 (v − w2 )2 + v 2 (u − w1 )2 dx,

RN

∇Iμ (U ) ≥ U − Aμ U , and there exists a constant C = C(μ) such that

 1 ∇Iμ (U ) ≤ C U − Aμ U (1 + |Iμ (U )| 2 + U − Aμ U 2 .

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

9

Proof. Given U = (u, v), Ψ = (w1 , w2 ) = Aμ U and Ψ satisfying (2.12), for Φ = (ϕ1 , ϕ2 ) we have  ∇Iμ (U ), Φ =

(∇u∇ϕ1 + λ1 uϕ1 + ∇v∇ϕ2 + λ2 vϕ2 ) dx RN



  β11 (x)u3 ϕ1 + β12 (x)u2 vϕ2 + β12 (x)v 2 uϕ1 + β22 (x)v 3 ϕ2 dx

− RN



−μ



 β1 (x)|u|p−2 uϕ1 + β2 (x)|v|p−2 vϕ2 dx

RN



=

(2.13)

(∇u − ∇w1 )∇ϕ1 + λ1 (u − w1 )ϕ1 + (∇v − ∇w2 )∇ϕ2

RN





+ λ2 (v − w2 )ϕ2 dx −  = U − Aμ U, Φ −

  β12 (x) u2 (v − w2 )ϕ2 + v 2 (u − w1 )ϕ1 dx

RN

  β12 (x) u2 (v − w2 )ϕ2 + v 2 (u − w1 )ϕ1 dx.

RN

Taking Φ = U − Aμ U in (2.13) gives  ∇Iμ (U ), U − Aμ U  = U − Aμ U 2 −

  β12 (x) u2 (v − w2 )2 + v 2 (u − w1 )2 dx.

(2.14)

RN

Using Hölder’s inequality and (2.14), we have       2   β (x)v ϕ (u − w )dx 12 1 1    N  R

⎛ ≤⎝

1 ⎛ ⎞ 4s



|β12 (x)|s dx⎠

RN

⎛ ·⎝



⎝ ⎞ 14



⎞ 12 ⎛ |β12 (x)|v 2 (u − w1 )2 dx⎠ ⎝

RN

⎞ s−1 4s

 |ϕ1 |

4s s−1

dx⎠

RN

(2.15)

|β12 (x)|v 4 dx⎠

RN

⎛

≤C |∇Iμ (U ), U − Aμ U | 2 Φ ⎝ 1



⎞ 14 |β12 (x)|v 4 dx⎠ .

RN

Similarly, one can derive   ⎛ ⎞ 14      1  β12 (x)u2 ϕ2 (v − w2 )dx ≤ C|∇Iμ (U ), U − Aμ U | 2 Φ ⎝ |β12 (x)|u4 dx⎠ .    3  N R

R

(2.16)

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

10

It follows (2.13)-(2.16) that

|∇Iμ (U ), Φ| !

⎛

≤U − Aμ U Φ + C|∇Iμ (U ), U − Aμ U | Φ ⎝ 1 2

⎛ +⎝

⎞ 14



|β12 (x)|u4 dx⎠

⎞ 14



|β12 (x)|v 4 dx⎠

RN

"

(2.17)

RN

⎛

≤U − Aμ U Φ + C|∇Iμ (U ), U − Aμ U | Φ ⎝ 1 2



⎞ 14 |β12 (x)|(v 4 + u4 )dx⎠ .

RN

By (2.17), we obtain ⎛ 1 ∇Iμ (U ) ≤ U − Aμ U  + C|∇Iμ (U ), U − Aμ U | 2 ⎝



⎞ 14 |β12 (x)|(|v|4 + |u|4 )dx⎠ .

(2.18)

RN

Note that 

1 1 1 Iμ (U ) − U, U − Aμ U  = U 2 − 4 4 4  +

β12 (x)(u2 (v − w2 )v + v 2 (u − w1 )u)dx RN

1 1 − 4 p





(2.19) p

μ

p

(β1 (x)|u| + β2 (x)|v| )dx,

RN

and  |β12 (x)|(|u|4 + |v|4 )dx RN



 |β12 (x)|(|u| + |v| )dx + C

≤C

2

RN

⎛ ≤C ⎝

|β12 (x)|(|u|p + |v|p )dx

2

RN



⎞ 1s ⎛ |β12 (x)| dx⎠ ⎝ s

RN

⎞ s−1 s

 |U |

2s s−1

dx⎠

RN



|β12 (x)|(|u|p + |v|p )dx.

≤CU  + C 2

RN

Then, by (2.19) and (2.20) we have

(2.20)

 |β12 (x)|(|u| + |v| )dx p

+C RN

p

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

11

 U  +

|β12 (x)|(|u|4 + |v|4 )dx

2

RN

⎛

≤C ⎝U 2 +

⎞



|β12 (x)|(|u|p + |v|p )dx⎠

RN

⎛

≤C ⎝U  +



⎞

(2.21)

(β1 (x)|u| + β2 (x)|v| )dx⎠

2

p

p

RN

 ⎞      2  2  ≤C ⎝|Iμ (U )| + |U, U − Aμ U | +  β12 (x) u (v − w2 )v + v (u − w1 )u dx⎠ .  N  ⎛

R

It follows Hölder’s inequality and (2.14) that       2   β (x)v u(u − w )dx 12 1    N  R

⎛ ≤⎝



⎞ 12 ⎛ |β12 (x)|v 2 (u − w1 )2 dx⎠ ⎝

RN

⎛

1 ≤C|∇Iμ (U ), U − Aμ U | 2 ⎝



⎞ 14 ⎛ |β12 (x)|u4 dx⎠ ⎝

RN



⎞ 12



⎞ 14 |β12 (x)|v 4 dx⎠

(2.22)

RN

|β12 (x)|(|u|4 + |v|4 )dx⎠ .

RN

Similarly, we have   ⎞ 12 ⎛       β12 (x)u2 v(v − w2 )dx ≤ C|∇Iμ (U ), U − Aμ U | 12 ⎝ |β12 (x)|(|u|4 + |v|4 )dx⎠ .    3  N R

(2.23)

R

Making use of (2.21)-(2.23) leads to  U  + 2

|β12 (x)|(|u|4 + |v|4 )dx

RN

! 1 ≤C |Iμ (U )| + |U, U − Aμ U | + |∇Iμ (U ), U − Aμ U | 2 ⎛ ·⎝



⎞ 12 |β12 (x)|(|u| + |v| )dx⎠ 4

4

(2.24)

" .

RN

In view of (2.24), it follows Young’s inequality that  |β12 (x)|(|u|4 + |v|4 )dx ≤ C(|Iμ (U )| + U − Aμ U 2 + |∇Iμ (U ), U − Aμ U |). RN

For any ε > 0, there exists Cε > 0 such that

(2.25)

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

12

 1 1 1 1 ∇Iμ (U ), U − Aμ U  2 |Iμ (U )| 4 + U − Aμ U  2 + |∇Iμ (U ), U − Aμ U | 4

(2.26)

1

≤ ε∇Iμ (U ) + Cε U − Aμ U  |Iμ (U )| 2 + Cε U − Aμ U 2 + Cε U − Aμ U 3 . By (2.25), (2.18) and (2.26), we obtain ∇Iμ (U )

 1 1 1 1 ≤U − Aμ U  + C∇Iμ (U ), U − Aμ U  2 |Iμ (U )| 4 + U − Aμ U  2 + |∇Iμ (U ), U − Aμ U | 4 1

≤U − Aμ U  + ε∇Iμ (U ) + Cε U − Aμ U  |Iμ (U )| 2 + Cε U − Aμ U 2 + Cε U − Aμ U 3 . Choose ε = 12 , we have

 1 ∇Iμ (U ) ≤ CU − Aμ U  1 + |Iμ (U )| 2 + U − Aμ U 2 .

2

Lemma 2.4. Let c ∈ R and Kc = {U | U ∈ H, ∇Iμ (U ) = 0, Iμ (U ) = c}, Kc∗ = Kc \ W. Assume that N is a symmetric closed neighborhood of Kc∗ . Then there exists an ε0 > 0 such that for 0 < ε < ε0 there exists a continuous map σ : [0, 1] × H → H satisfying: (1) (2) (3) (4) (5)

σ(0, U ) = U , ∀ U ∈ H. σ(t, U ) = U , ∀ t ∈ [0, 1] and |Iμ (U ) − c| ≥ 2ε. σ(t, −U ) = −σ(t, U ), ∀ (t, U ) ∈ [0, 1] × H. σ(1, Iμc+ε \(N ∪ W )) ⊂ Iμc−ε . σ(t, P1± ) ⊂ P1± , σ(t, P2± ) ⊂ P2± , ∀t ∈ [0, 1].

In particular, if η = σ(1, ·), then (6) (7) (8) (9)

η(−U ) = −η(U ), ∀ U ∈ H. η|Iμc−2ε = Id. η(Iμc+ε \(N ∪ W )) ⊂ Iμc−ε . η(P1± ) ⊂ P1± , η(P2± ) ⊂ P2± .

Proof. For sufficiently small δ > 0, we denote N (δ) = {U | U ∈ H, dist(U, Kc ) < δ} and N (2δ) ⊂ N ∪ W. Since Iμ satisfies the (P S) condition, there exist constants ε, b1 > 0 such that ∇Iμ (U ) ≥ b1 , ∀ U ∈ Iμ−1 ([c − 2ε, c + 2ε])\N

δ  2

By virtue of Lemma 2.3, there exists b > 0 such that U − W  ≥ b and ∇Iμ (U ), U − W  ≥ bU − W  for U ∈ Iμ−1 ([c − 2ε, c + 2ε])\N ( 2δ ).

.

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

13

Let ε < ε0 < 12 bδ and define an even Lipschitz continuous function g: H → [0, 1] by  g(U ) =

0, U ∈ N ( 2δ ) or |Iμ (U ) − c| ≥ 2ε, 1, U ∈ / N (δ) and |Iμ (U ) − c| ≤ ε.

Consider the initial value problem 

dσ(t,U ) dt

= −Ψ(σ(t, U )),

(2.27)

σ(0, U ) = U ∈ H, σ−A σ

where Ψ(σ) = g(σ) σ−Aμμ σ . Set σ(t, U ) = τ ( 2ε b t, U ). It is not difficult to verify that (1)-(3) are true. For (4), we let U ∈ Iμc+ε \(N ∪ W ). If Iμ (τ (t, U )) > c −ε for 0 ≤ t ≤ then

2ε b ,

then g(τ (t, U )) = 1. If there exists t0 ∈ [0, 2ε b ] such that τ (t0 , U ) ∈ N (δ),

t0 δ ≤ τ (t0 , U ) − U  ≤

τ (s, U )ds ≤ t0 ≤

2ε < δ, b

0

which yields a contradiction. Thus, we obtain   2ε Iμ (σ(1, U )) = Iμ τ ,U b 1 = Iμ (U ) −

∇Iμ (σ(s, U )), Ψ(σ(s, U ))ds 0

g(σ(s, U )) = Iμ (U ) − σ(s, U ) − Aμ σ(s, U ) b bds ≤ Iμ (U ) − 0

∇Iμ (σ(s, U )), σ(s, U ) − Aμ σ(s, U )ds 0

2ε

1 ≤ Iμ (U ) −

1

bds 0

< c + ε − 2ε = c − ε. For (5), it is straightforward to be verified due to Aμ (P1± ) ⊂ P1± and Aμ (P2± ) ⊂ P2± .

2

Lemma 2.5. For sufficiently small δ > 0, which is independent of μ, there holds Iμ (U ) ≥ (∂P1+ ∩ ∂P1− ) ∩ (∂P2+ ∩ ∂P2− ).

δ2 2

for U ∈ Σ =

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

14

Proof. From (2.1), we can rewrite it as

Iμ (U ) =

1 1 U 2 − 2 4 −

μ p



 (β11 (x)u4 + 2β12 (x)u2 v 2 + β22 (x)v 4 )dx RN

(2.28)

(β1 (x)|u|p + β2 (x)|v|p )dx. RN

For U ∈ ∂P1+ , we have Cu−  ≥ distH 1 (RN ) (u, P ) = δ. Moreover, we can deduce that  RN

⎛ β11 (x)(u− )4 dx ≤ ⎝



⎞ 1s ⎛ |β11 (x)|s dx⎠ ⎝

RN

⎞ s−1 s



(u− ) s−1 dx⎠ 4s

RN

≤ C|u− |4

4s

L s−1 (RN )

= Cdist4 s−1 4s L

(RN )

(u, P )

≤ Cdist4H 1 (RN ) (u, P ) = Cδ 4 , and  RN

⎛ β1 (x)(u− )p dx ≤ ⎝



⎞ 1s ⎛ |β1 (x)|s dx⎠ ⎝

RN

⎞ s−1 s



(u− ) s−1 dx⎠ ps

RN

≤ C|u− |p

ps

L s−1 (RN )

= Cdistp

ps

L s−1 (RN )

(u, P )

≤ CdistpH 1 (RN ) (u, P ) = Cδ p . Using an analogous process, we can get the estimates for u+ and v ± . Substituting these estimates into (2.28), for sufficiently small δ > 0 we obtain Iμ (U ) ≥ Cδ 2 −

C 4 C p δ2 δ − δ ≥ . 4 p 2

2

Lemma 2.6. Let μn → 0 and Un ∈ H satisfy ∇Iμn (Un ) = 0 and Iμn (Un ) ≤ C for some C > 0. Then up to a subsequence, there hold Un → U in H, ∇I(U ) = 0 and I(U ) = lim Iμn (Un ). n→∞

Furthermore, if Un (n = 1, 2, · · · ) is sign-changing, then U is sign-changing too.

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

15

Proof. Note that 1 Iμn (Un ) − ∇Iμn (Un ), Un  4   1 1 1 2 − μn = Un  + (β1 (x)|un |p + β2 (x)|vn |p ) dx 4 4 p RN

1 ≥ Un 2 . 4 So {Un } is bounded in H. Similar to the proof of Lemma 2.1, we have Un → U in H. Thus we further get I(U ) = lim Iμn (Un ) and ∇I(U ) = 0. n→∞

Since Un = (un , vn ) satisfies 

−u + λ1 u − β12 (x)v 2 u = β11 (x)u3 + μn β1 (x)|u|p−2 u, x ∈ RN , −v + λ2 v − β12 (x)u2 v = β22 (x)v 3 + μn β2 (x)|v|p−2 v, x ∈ RN ,

we then have

 2 u+ n =

2 + 2 (|∇u+ n | + λ1 (un ) ) dx RN



≤

4 + p (β11 (x)(u+ n ) + β1 (un ) ) dx RN

 4 ≤ c |u+ n|

4s

L s−1 (RN )

p + |u+ n|

ps

L s−1 (RN )

4 + p ≤ c(u+ n  + un  ). − ± That is, there exists c > 0 such that u+ n  ≥ c. Similarly, we can also obtain un  ≥ c and vn  ≥ c. By ± ± passing to the limit, we have u  ≥ c and v  ≥ c. Thus, U is sign-changing. 2

Now we are ready to prove Theorem 1.1. Proof of Theorem 1.1. Let B 2n be a unit closed ball of R2n . Denote t = (t1 , t2 ) ∈ R2n with t1 = (t11 , · · · , t1n ) and t2 = (t21 , · · · , t2n ), where t1 , t2 ∈ Rn . Define a continuous map ϕ(n) : B 2n → H by ϕ(n) (t) = ϕ(n) (t1 , t2 ) = Rn (t1 u, t2 v), where Rn > 0 is large enough, u = (u1 , · · · , un ), v = (v1 , · · · , vn ), t1 u = t11 u1 + · · · + t1n un , and t2 v = t21 v1 + · · · + t2n vn . Here we require ui , vi (i = 1, 2, · · · , n) to be disjoint supports. Then we have (1) If t1 = 0, then ϕ(n) (t) = Rn (0, t2 v) ∈ P1+ ∩ P1− ; and if t2 = 0, then ϕ(n) (t) = Rn (t1 u, 0) ∈ P2+ ∩ P2− . (2) For t ∈ B 2n , ϕ(n) (−t) = Rn (−t1 u, −t2 v) = −ϕ(n) (t). (3) sup Iμ (ϕ(n) (t)) < 0 < inf Iμ (u, v). t∈∂B 2n

(u,v)∈Σ

For t = (t1 , t2 ) ∈ ∂B 2n , we have Iμ (ϕ(n) (t)) ≤ I(ϕ(n) (t))

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

16

= I(Rn (t1 u, t2 v))    1 2 |t1 ∇u|2 + λ1 |t1 u|2 + |t2 ∇v|2 + λ2 |t2 v|2 dx = Rn 2 RN

1 − Rn4 4



  β11 (x)|t1 u|4 + 2β12 (x)|t1 u|2 |t2 v|2 + β22 (x)|t2 v|4 dx

RN

≤

1 2 1 Rn (t1 u, t2 v)2 − Rn4 2 4





 β11 (x)|t1 u|4 + β22 (x)|t2 v|4 dx.

RN

In addition, for (t1 , t2 ) ∈ ∂B 2n , we know that (t1 u, t2 v) ≤ A and the function F (t1 , t2 ) =  (β11 (x)|t1 u|4 + β22 (x)|t2 v|4 )dx is positive and continuous on ∂B 2n . Thus there exists α > 0 such RN that F (t1 , t2 ) ≥ α > 0 for (t1 , t2 ) ∈ ∂B 2n . For the sufficient large Rn , we have Iμ (ϕ(n) (t)) ≤

1 2 2 1 4 A Rn − Rn α < 0 . 2 4

Define cj (μ) = inf

sup

B∈Γj (u,v)∈B\W

Iμ (u, v), μ ∈ (0, 1],

where   Γj = B | ∃ n ≥ j, B = ϕ(B 2n \Y ), ϕ ∈ Gn , −Y = Y ⊂ B 2n , γ(Y ) ≤ n − j , and  Gn = ϕ ∈ C(B 2n , H)| ϕ(−t) = −ϕ(t), ϕ(t) ∈ P1+ ∩ P1− , t1 = 0, ϕ(t) ∈ P2+ ∩ P2− ,  t2 = 0, ϕ|∂B 2n = ϕ(n) . By virtue of Lemmas 2.1-2.5 and following Theorem 2.5 of [13], we know that cj (μ) (j ≥ 3) are critical values of Iμ , Kcj (μ) \W = ∅, and cj (μ) → ∞ as j → ∞. Let cj = limμ→0 cj (μ). According to Lemma 2.6, cj (j ≥ 3) are critical values of the functional I and there exists a sequence of sign-changing critical points {Un } ⊂ H satisfying ∇I(Un ) = 0, I(Un ) = cj and cj → ∞, as j → ∞. 2 3. Proof of Theorem 1.2 In this section, we will consider the special case of s = 3 for N = 3, which plays a critical role in the proof. We impose an additional assumption on the matrix (βij (x)) (i, j = 1, 2). That is, βij (x) (i, j = 1, 2) satisfy condition (B + ). In this case, the perturbation is not needed. Parallel to Lemmas 2.1-2.2 and 2.4-2.5 in the preceding section, we can obtain the following Lemmas 3.1-3.2 and 3.4-3.5, and omit the proofs. Lemma 3.1. I satisfies the (P S) condition. The definitions of P , P1± , P2± , W , and Σ are the same as given in the preceding section. Now we define the operator A by Ψ = AU,

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

17

where Ψ = (w1 , w2 ) is given by ⎧ 2 3 N ⎪ ⎨ −w1 + λ1 w1 − β12 (x)v w1 = β11 (x)u , x ∈ R , ⎪ ⎩ −w + λ w − β (x)u2 w = β (x)v 3 , x ∈ RN , 2 2 2 12 2 22 or in it’s the weak form ⎧   2 3 1 N ⎪ ⎨ RN (∇w1 ∇ϕ1 + λ1 w1 ϕ1 )dx − RN β12 (x)v w1 ϕ1 dx = RN β11 (x)u ϕ1 dx, ∀ ϕ1 ∈ H (R ), ⎪ ⎩

RN

(∇w2 ∇ϕ2 + λ2 w2 ϕ2 )dx −

 RN

2

β12 (x)u w2 ϕ2 dx =

 RN

(3.1) β22 (x)v ϕ2 dx, ∀ ϕ2 ∈ H (R ). 3

1

N

Then, A is well-defined and Lipschitz continuous. Lemma 3.2. There exists δ0 > 0 such that for 0 < δ < δ0 , there holds A(P1± ) ⊂ P1± and A(P2± ) ⊂ P2± . Lemma 3.3. There holds  ∇I(U ), U − AU  = U − AU  − 2

  β12 (x) u2 (v − w2 )2 + v 2 (u − w1 )2 dx,

RN

∇I(U ) ≥ U − AU , and there exists a constant C > 0 such that

 1 ∇I(U ) ≤ CU − AU  1 + |I(U )| 2 + U − AU 2 . Proof. Given U = (u, v) and Ψ = (w1 , w2 ) = AU satisfying (3.1), for Φ = (ϕ1 , ϕ2 ), we have  ∇I(U ), Φ = RN

 

−

∇u∇ϕ1 + λ1 uϕ1 + ∇v∇ϕ2 + λ2 vϕ2 )dx 

 β11 (x)u3 ϕ1 + β12 (x)u2 vϕ2 + β12 (x)v 2 uϕ1 + β22 (x)v 3 ϕ2 dx

RN

 =



(∇u − ∇w1 )∇ϕ1 + λ1 (u − w1 )ϕ1 + (∇v − ∇w2 )∇ϕ2

RN



 + λ2 (v − w2 )ϕ2 dx −  = U − AU, Φ −

(3.2)

β12 (x)(u2 (v − w2 )ϕ2 + v 2 (u − w1 )ϕ1 )dx

RN

β12 (x)(u2 (v − w2 )ϕ2 + v 2 (u − w1 )ϕ1 )dx.

RN

Taking Φ = U − AU in (3.2) leads to  ∇I(U ), U − AU  = U − AU 2 − RN

It follows (3.3) and Hölder’s inequality that

  β12 (x) u2 (v − w2 )2 + v 2 (u − w1 )2 dx.

(3.3)

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

18

      2   β (x)v ϕ (u − w )dx 12 1 1    N  R

⎛ ≤⎝



⎞ 12 ⎛ |β12 (x)|v 2 (u − w1 )2 dx⎠ ⎝

RN

⎛

1 ≤|∇I(U ), U − AU | 2 ⎝



RN



RN

|β12 (x)|ϕ41 dx⎠ ⎝ ⎞

1 12

|β12 (x)|3 dx⎠ ⎛

≤C|∇I(U ), U − AU | 2 Φ ⎝ 1

⎞ 14 ⎛

⎛ ⎝



⎞ 14 |β12 (x)|v 4 dx⎠

RN



⎞ 16 ⎛ |ϕ1 |6 dx⎠ ⎝

RN





⎞ 14

(3.4)

|β12 (x)|v 4 dx⎠

RN

⎞ 14

|β12 (x)|v 4 dx⎠ .

RN

Similarly, we have   ⎛ ⎞ 14      1  β12 (x)u2 ϕ2 (v − w2 )dx ≤ C|∇I(U ), U − AU | 2 Φ ⎝ |β12 (x)|u4 dx⎠ .   N  N R

(3.5)

R

In view of condition (B + ), it follows (3.2)-(3.5) that

|∇I(U ), Φ| 1

≤U − AU  Φ + C|∇I(U ), U − AU | 2 Φ ⎛⎛ ⎞ 14 ⎛ ⎞ 14 ⎞   ⎟ ⎜ |β12 (x)|u4 dx⎠ + ⎝ |β12 (x)|v 4 dx⎠ ⎠ · ⎝⎝ RN

RN

⎛ ≤U − AU  Φ + C|∇I(U ), U − AU | 2 Φ ⎝ 1



⎞ 14 |β12 (x)|(u4 + v 4 )dx⎠

(3.6)

RN 1 2

≤U − AU  Φ + C|∇I(U ), U − AU | Φ ⎞ 14 ⎛    β11 (x)u4 + 2β12 (x)u2 v 2 + β22 (x)v 4 dx⎠ . ·⎝ RN

By (3.6), it implies that

1

∇I(U ) ≤U − AU  + C|∇I(U ), U − AU | 2 ⎛ ⎞ 14    ·⎝ β11 (x)u4 + 2β12 (x)u2 v 2 + β22 (x)v 4 dx⎠ . RN

(3.7)

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

19

On the other hand, by taking 2 < r < 4, a straightforward calculation gives 1 I(U ) − U, U − AU  r      1 1 1 1 2 = − U  + − β11 (x)u4 + 2β12 (x)u2 v 2 + β22 (x)v 4 dx 2 r r 4 −



1 r

RN

(3.8)

  β12 (x) u2 (v − w2 )v + v 2 (u − w1 )u dx.

RN

In view of condition (B + ), it follows (3.3) and Hölder’s inequality that       2  β12 (x)u v(v − w2 )dx   N  R

⎛ ≤⎝



⎞ 12 ⎛ |β12 (x)|u2 (v − w2 )2 dx⎠ ⎝

RN

⎛

1 ≤C|∇I(U ), U − AU | 2 ⎝



⎞ 12 |β12 (x)|u2 v 2 dx⎠

RN



|β12 (x)|(v 4 + u4 )dx⎠

RN

⎛ 1 ≤C|∇I(U ), U − AU | 2 ⎝



(3.9)

⎞ 12

⎞ 12 (β11 (x)u4 + 2β12 (x)u2 v 2 + β22 (x)v 4 )dx⎠ .

RN

Similarly, we can derive that   ⎞ 12 ⎛        1  β12 (x)v 2 u(u − w1 )dx ≤ C|∇I(U ), U − AU | 2 ⎝ β11 (x)u4 + 2β12 (x)u2 v 2 + β22 (x)v 4 dx⎠ .   N  N R

R

(3.10) Using (3.8)-(3.10) and Young’s inequality leads to  U 2 +

(β11 (x)u4 + 2β12 (x)u2 v 2 + β22 (x)v 4 )dx

RN

 ⎞     ≤C ⎝|I(U )| + |U, U − AU | +  β12 (x)(u2 (v − w2 )v + v 2 (u − w1 )u)dx⎠  N  ⎛

R

  12 1 ≤C |I(U )| + |U, U − AU | + |∇I(U ), U − AU | 2 (β11 (x)u4 + 2β12 (x)u2 v 2 + β22 (x)v 4 )dx RN

 12  1 4 2 2 4 2 (β11 (x)u + 2β12 (x)u v + β22 (x)v )dx ≤C |I(U )| + U U − AU  + |∇I(U ), U − AU | RN

 ≤C |I(U )| + εU 2 + Cε U − AU 2 + Cε |∇I(U ), U − AU |

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

20



 +ε

4

2 2

4

(β11 (x)u + 2β12 (x)u v + β22 (x)v )dx ,

RN

where ε is arbitrary and Cε is constant. Thus we further deduce that ⎛ ⎝





β11 (x)u4 + 2β12 (x)u2 v 2 + β22 (x)v

 4

⎞ 14 dx⎠

RN

(3.11)

 1 ≤C |I(U )| + U − AU 2 + |∇I(U ), U − AU | 4

 1 1 1 ≤C |I(U )| 4 + U − AU  2 + |∇I(U ), U − AU | 4 . Substituting (3.11) into (3.7) yields

 1 1 1 1 ∇I(U ) ≤U − AU  + C|∇I(U ), U − AU | 2 |I(U )| 4 + U − AU  2 + |∇I(U ), U − AU | 4 1

≤U − AU  + ε∇I(U ) + Cε U − AU  |I(U )| 2 + Cε U − AU 2 + Cε U − AU 3 . Consequently, we obtain

 1 ∇I(U ) ≤ CU − AU  1 + |I(U )| 2 + U − AU 2 .

2

Lemma 3.4. Let Kc = {U | U ∈ H, ∇I(U ) = 0, I(U ) = c}, Kc∗ = Kc \ W. Assume that N is a symmetric closed neighborhood of Kc∗ . Then there exists ε0 > 0 such that for 0 < ε < ε0 there exists a continuous map σ : [0, 1] × H → H satisfying: (1) (2) (3) (4) (5)

σ(0, U ) = U , ∀ U ∈ H. σ(t, U ) = U , ∀ t ∈ [0, 1] and |I(U ) − c| ≥ 2ε. σ(t, −U ) = −σ(t, U ), ∀ (t, U ) ∈ [0, 1] × H. σ(1, I c+ε \(N ∪ W )) ⊂ I c−ε . σ(t, P1± ) ⊂ P1± , σ(t, P2± ) ⊂ P2± , ∀t ∈ [0, 1].

In particular, let η = σ(1, ·), then we have (6) (7) (8) (9)

η(−U ) = −η(U ), ∀U ∈ H. η|I c−2ε = Id. η(I c+ε \(N ∪ W )) ⊂ I c−ε . η(P1± ) ⊂ P1± , η(P2± ) ⊂ P2± .

Lemma 3.5. For sufficiently small δ > 0, there holds I(U ) ≥ We are now in a position to prove Theorem 1.2.

δ2 2

for U ∈ Σ = (∂P1+ ∩ ∂P1− ) ∩ (∂P2+ ∩ ∂P2− ).

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

21

Proof of Theorem 1.2. Define a continuous function ϕ(n) : B 2n → H by ϕ(n) (t) = ϕ(n) (t1 , t2 ) = Rn (t1 u, t2 v), where B 2n is a unit ball in R2n and Rn > 0 is sufficiently large. Here all notations are the same as given in the preceding section. We thus have (1) If t1 = 0, then ϕ(n) (t) = Rn (0, tv) ∈ P1+ ∩ P1− ; and if t2 = 0, then ϕ(n) (t) = Rn (tu, 0) ∈ P2+ ∩ P2− . (2) For t ∈ B 2n , ϕ(n) (−t) = Rn (−t1 u, −t2 v) = −ϕ(n) (t). (3) sup I(ϕ(n) (t)) < 0 < inf I(u, v). (u,v)∈Σ

t∈∂B 2n

For t ∈ ∂B 2n , we get I(ϕ(n) (t)) = I(Rn (t1 u, t2 v))    1 2 |t1 ∇u|2 + λ1 |t1 u|2 + |t2 ∇v|2 + λ2 |t2 v|2 dx = Rn 2 RN

1 − Rn4 4





 β11 (x)|t1 u|4 + 2β12 (x)|t1 u|2 |t2 v|2 + β22 (x)|t2 v|4 dx

RN

=

1 1 2 Rn (t1 u, t2 v)2 − Rn4 2 4



  β11 (x)|t1 u|4 + 2β12 (x)|t1 u|2 |t2 v|2 + β22 (x)|t2 v|4 dx.

RN

In view of condition (B + ), we have (n)

I(ϕ

1 (t) ≤ Rn2 (t1 u, t2 v)2 − CRn4 2

 |β12 (x)|(|t1 u|4 + |t2 v|4 )dx.

RN

From the proof of Theorem 1.1, we see that I(ϕ(n) (t)) < 0. Define cj = inf

sup

B∈Γj (u,v)∈B\W

I(u, v),

where   Γj = B | ∃ n ≥ j, B = ϕ(B 2n \Y ), ϕ ∈ Gn , −Y = Y ⊂ B 2n , γ(Y ) ≤ n − j , and   Gn = ϕ ∈ C(B 2n , H)| ϕ(−t) = −ϕ(t), ϕ(t) ∈ P1+ ∩ P1− , t1 = 0; ϕ(t) ∈ P2+ ∩ P2− , t2 = 0 ϕ|∂B 2n = ϕ(n) . According to Theorem 2.5 of [13], cj (j ≥ 3) are critical values of I, and Kcj \W = ∅. Otherwise, if Kcj \ W = ∅, by Lemma 3.4, there exist ε > 0 and a continuous map η such that • • • •

η(−U ) = −η(U ), ∀ U ∈ H. η|I cj −2ε = Id. η(I cj +ε \W ) ⊂ I cj −ε . η(P1± ) ⊂ P1± , η(P2± ) ⊂ P2± .

22

X. Liu et al. / J. Math. Anal. Appl. 481 (2020) 123478

By the definition of cj , there exists B ∈ Γj such that

sup I(U ) ≤ cj + ε, and then B \ W ⊂ I cj +ε . It is

U ∈B\W

easy to verify that η(B) ∈ Γj . However, there also holds

η(B) \ W ⊂ (η(B \ W ) ∪ η(W )) \ W ⊂ η(B \ W ) ⊂ I cj −ε , which yields a contradiction. Similarly, we can prove cj → ∞ as j → ∞. Consequently, system (1.1) has infinitely many sign-changing solutions. 2 References [1] B.D. Esry, Chris H. Greene, James P. Burke Jr., John L. Bohn, Hartree-Fock theory for double condensates, Phys. Rev. Lett. 78 (1997) 3594–3597. [2] E. Timmermans, Phase seperation of Bose Einstein condensates, Phys. Rev. Lett. 81 (1998) 5718–5721. [3] M. Mitchell, M. Segev, Self-trapping of inconherent white light, Nature 387 (1997) 880–882. [4] P.H. Rabinowitz, Minimax Methods in Critical Point Theory with Applications to Differential Equations, CBMS Reg. Conf. Ser. Math., vol. 65, AMS, Providence, R.I., 1986. [5] S. Chang, C.S. Lin, T.C. Lin, W. Lin, Segregated nodal domains of two-dimensional multispecies Bose-Einstein condensates, Phys. D 196 (2004) 341–361. [6] P. Panayotaros, M. Sepúlveda, O. Vera, Solitary waves for a coupled nonlinear Schrödinger system with dispersion management, Electron. J. Differential Equations 107 (2010) 1–26. [7] T. Bartsch, N. Dancer, Z.-Q. Wang, A Liouville theorem, a-priori bounds, and bifurcating branches of positive solutions for a nonlinear elliptic system, Calc. Var. Partial Differential Equations 37 (2010) 345–361. [8] E.N. Dancer, J. Wei, T. Weth, A priori bounds versus multiple existence of positive solutions for a nonlinear Schrödinger system, Ann. Inst. H. Poincaré Anal. Non Linéaire 27 (2010) 953–969. [9] B. Noris, M. Ramos, Existence and bounds of positive solutions for a nonlinear Schrödinger system, Proc. Amer. Math. Soc. 138 (2010) 1681–1692. [10] S. Terracini, G. Verzini, Multipulse phase in k-mixtures of Bose-Einstein condensates, Arch. Ration. Mech. Anal. 194 (2009) 717–741. [11] R. Tian, Z.-Q. Wang, Multiple solitary wave solutions of nonlinear Schrödinger systems, Topol. Methods Nonlinear Anal. 37 (2011) 203–223. [12] J. Wei, T. Weth, Radial solutions and phase separation in a system of two coupled Schrödinger equations, Arch. Ration. Mech. Anal. 190 (2008) 83–106. [13] J.Q. Liu, X.Q. Liu, Z.-Q. Wang, Multiple mixed states of nodal solutions for nonlinear Schrödinger systems, Calc. Var. Partial Differential Equations 52 (2015) 565–586. [14] J.Q. Liu, X.Q. Liu, Z.-Q. Wang, Infinitely many sign-changing solutions for a nonlinear Schrödinger systems with finite potential well, Sci. China Ser. A 46 (2016) 587–604 (in Chinese). [15] L.R. Huang, E. Rocha, J. Chen, Positive and sign-changing solutions of a Schrödinger-Poisson system involving a critical nonlinearity, J. Math. Anal. Appl. 408 (2013) 55–69. [16] W. Shuai, Q. Wang, Existence and asymptotic behavior of sign-changing solutions for the nonlinear Schrödinger-Poisson system in R3 , Z. Angew. Math. Phys. 66 (2015) 3267–3282. [17] N. Hirano, Multiple existence of sign changing solutions for coupled nonlinear Schrödinger equations, Nonlinear Anal. 73 (2010) 2580–2593. [18] T. D’Aprile, A. Pistoia, Existence, multiplicity and profile of sign-changing clustered solutions of a semiclassical nonlinear Schrödinger equation, Ann. Inst. H. Poincaré Anal. Non Linéaire 26 (2009) 1423–1451.