Positive solutions for Schrödinger system with asymptotically periodic potentials

Nonlinear Analysis 134 (2016) 215–235

Contents lists available at ScienceDirect

Nonlinear Analysis www.elsevier.com/locate/na

Positive solutions for Schr¨odinger system with asymptotically periodic potentials Jun Wang a,∗ , Qing He a , Lu Xiao b , Fubao Zhang c a

Faculty of Science, Jiangsu University, Zhenjiang, Jiangsu, 212013, PR China School of management, Jiangsu University, Zhenjiang, Jiangsu, 212013, PR China c Department of Mathematics, Southeast University, Nanjing 210096, PR China b

article

info

Article history: Received 19 June 2015 Accepted 4 January 2016 Communicated by Enzo Mitidieri Keywords: Weakly coupled Schr¨ odinger system Positive solutions Variational methods

abstract This paper is concerned with the following class of elliptic equations



−∆u + λ(x)u = µf (u) + βv 2 u in RN , −∆v + κ(x)v = νg(v) + βu2 v in RN ,

where u, v ∈ H 1 (RN ), N ≤ 3, µ, ν, β > 0 are coupling constants, λ(x) and κ(x) are asymptotically periodic functions, f and g are continuous functions with subcritical growth. This type of system arises, in particular, in models in Bose–Einstein condensates theory. We prove the existence of positive solution for this weakly coupled system with β > 0 sufficiently large. Furthermore, we obtain some sufficient conditions for the nonexistence of positive solutions. © 2016 Elsevier Ltd. All rights reserved.

1. Introduction and main results In present paper, we study the following two-component coupled nonlinear Schr¨odinger equations  ∂Φ  −i~ 1 − ∆x Φ1 + a(x)Φ1 − (µ|Φ1 |2 + β|Φ2 |2 )Φ1 = 0 in (0, ∞) × RN , ∂t (1.1)  −i~ ∂Φ2 − ∆x Φ2 + b(x)Φ2 − (ν|Φ2 |2 + β|Φ1 |2 )Φ2 = 0 in (0, ∞) × RN , ∂t where N ≤ 3, i is the imaginary unit, ~ is the Planck constant, µ, ν > 0 and β ̸= 0 are coupling constants. These systems of equations, also known as Gross–Pitaevskii equations, have applications in many physical ∗ Corresponding author. E-mail addresses: [email protected] (J. Wang), [email protected] (Q. He), [email protected] (L. Xiao), [email protected] (F. Zhang).

http://dx.doi.org/10.1016/j.na.2016.01.011 0362-546X/© 2016 Elsevier Ltd. All rights reserved.

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

216

problems such as in nonlinear optics and in Bose–Einstein condensates theory for multispecies Bose–Einstein condensates. For instance, when N = 2, problem (1.1) arises in the Hartree–Fock theory for a double condensate, i.e., a binary mixture of Bose–Einstein condensates in two different hyperfine states (see [13, 33]). From physical point of view, the solution Φj denotes the jth component of the beam in Kerr-like photorefractive media. The positive constants µ and ν are for self-focusing in the jth component of the beam. The coupling constant β is the interaction between the two components of the beam. The sign of the scattering length β determines whether the interactions of states are repulsive or attractive. In the attractive case the components of a vector solution tend to go along with each other, leading to synchronization. In the repulsive case, the components tend to segregate from each other, leading to phase separations. These phenomena have been documented in experiments as well as in numeric simulations (see [8,27] and references therein). To get the standing waves, we substitute Φ1 (t, x) = e−iEt/~ u(x)

and Φ2 (t, x) = e−iEt/~ v(x),

u, v ∈ H 1 (RN )

into (1.1). Then we know that u, v solve  −∆u + λ(x)u = µu3 + βv 2 u in RN , −∆v + κ(x)v = νv 3 + βu2 v in RN ,

(1.2)

(1.3)

where u, v ∈ H 1 (RN ), λ(x) = a(x) − E and κ(x) = b(x) − E, the constants µ, ν > 0 and β ∈ R. Note that (1.3) possesses semi-trivial solutions, i.e., (u, 0) and (0, v). So, here and in the sequel, we say z = (u, v) is a nontrivial positive solution of (1.3) if z solves (1.3) and both u, v are positive in RN . For the case of λ and κ are independent of x, many papers considered the existence and multiplicity of positive solutions of (1.3), for example, see [4–7,9,10,20,23,31,25,26,12,24] and the reference therein. Particularly, the interesting papers [4,31]proved that there exists 0 < β¯1 < β¯2 such that (1.3) has nontrivial positive solution for 0 < β < β¯1 or β¯2 < β. For the non-constant potentials case, most of the literature is devoted to the study of the so-called semiclassical states, which is a different topic from the one treated here. For instance, see [28,29,34,17,11,22] and the reference therein. Recently, the paper [16] considered the system (1.3). Assume λ, κ ∈ C 1 (RN ) such that lim λ(x) = λ∞ ,

|x|→∞

lim κ(x) = κ∞ ,

|x|→∞

inf λ(x) ≤ sup λ(x) = λ∞

x∈RN

x∈RN

and

inf κ(x) ≤ sup κ(x) = κ∞ .

x∈RN

x∈RN

The author proved that there exists 0 < β¯3 < β¯4 such that (1.3) has nontrivial positive solution for 0 < β < β¯3 or β¯4 < β. Motivated by the above results, we shall continue to study the existence of nontrivial positive solutions of (1.3). Here we consider the case when λ(x) and κ(x) are asymptotically periodic functions, i.e., the infinity limit of λ(x) and κ(x) is periodic functions. In fact, we shall consider a more general system than (1.3). That is,  −∆u + λ(x)u = µf (u) + βv 2 u in RN , (A ) −∆v + κ(x)v = νg(v) + βu2 v in RN , where u, v ∈ H 1 (RN ), µ, ν, β > 0 are constant, λ(x) and κ(x) are asymptotically periodic functions, f and g are continuous functions with subcritical growth. To state the mainly results for our case, we first give some assumptions. Throughout the paper, we assume that λ(x) and κ(x) are continue functions verifying the following assumptions (R1 ) There is δ0 > 0 such that λ(x), κ(x) ≥ δ0 for all x ∈ RN .

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

217

(R2 ) There are positive continuous periodic functions λT (x), κT (x) : RN → R, that is, λT (x + 1) = λT (x),

κT (x + 1) = κT (x)

such that λ(x) ≤ λT (x) ∀x ∈ RN ,

|λ(x) − λT (x)| → 0

as |x| → ∞

κ(x) ≤ κT (x) ∀x ∈ RN ,

|κ(x) − κT (x)| → 0

as |x| → ∞.

Concerning the functions f and g, we assume the following hypotheses (F1 ) f, g ∈ C(R) such that f (t), g(t) ≤ c(t + tp−1 ), where c > 0, 2 < p < 2∗ , 2∗ = ∞ if N = 1, 2, and 2∗ = 6 if N = 3. f (t) = g(t) = 0 for t ≤ 0. g(s) (F2 ) f (s) s and s are strictly increasing on intervals (0, ∞). s f (s) F (s) G(s) (F3 ) lims→0 s = lims→0 g(s) s = 0. Moreover, lims→∞ s2 = lims→∞ s2 = ∞, where F (s) = 0 f (t)dt s and G(s) = 0 g(t)dt. (F4 ) There exist a3 > 0 and δ > 0 such that for all (u, v) and h ∈ R with |h| ≤ δ |f (u + h) − f (u)| ≤ a3 |h|(1 + |u|p−1 ),

|g(v + h) − g(v)| ≤ a3 |h|(1 + |v|p−1 ).

The energy functional corresponding to system (A ) is defined by   1 1 (|∇u|2 + λ(x)|u|2 + |∇v|2 + κ(x)|v|2 ) − (µF (u) + 2βu2 v 2 + νG(v)), J (u, v) = 2 RN 4 RN

(1.4)

for (u, v) ∈ E ≡ H 1 (RN ) × H 1 (RN ). We should point out that the system (A ) also possesses semi-trivial solutions of type (u, 0) or (0, v). A solution (u, v) of (A ) is nontrivial if u ̸= 0 and v ̸= 0. A solution (u, v) with u > 0 and v > 0 is called a positive solution. A solution is called a nontrivial ground state solution (or positive ground state solution) if its energy is minimal among all the nontrivial solutions (or all the nontrivial positive solutions) of (A ). Then, we have the following main results. Theorem 1.1. Assume that (R1 )–(R2 ) and (F1 )–(F4 ) hold. Then there exists β ∗ > 0 such that for β > β ∗ the following results hold: (i) (A ) has a positive ground state solution z = (u, v) ∈ E. 1,σ (ii) lim|x|→∞ u(x) = lim|x|→∞ v(x) = 0, lim|x|→∞ |∇u(x)| = lim|x|→∞ ∇v(x) = 0 and u, v ∈ Cloc (RN ) with σ ∈ (0, 1). Furthermore, there exists θ0 > 0 such that for every θ < θ0 we have |u(x)| + |v(x)| ≤ ce−θ|x| , where c = cβ > 0. (iii) L is compact in E, where L denotes the set of all positive ground state solutions of ( A ). Remark 1.2. The condition (F4 ) was introduced in [19]. It has been also used to study the existence of solutions for Schr¨ odinger equation, see [35]. It is easy to see that (R4 ) is equivalent to f, g being locally Lipschitzian in s and satisfying |fs′ (s)| ≤ C(1 + |s|p−1 ),

|gs′ (s)| ≤ C(1 + |s|p−1 )

for some C > 0 and all x ∈ RN , u ∈ R for which the derivatives fs′ (s) and fs′ (s) exist.

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

218

Remark 1.3. We shall use Nehari manifolds constraint methods to prove our main results. Usually, the authors always need to assume that f, g ∈ C 1 , for instance, see [3,9,15,20,31] and the references therein. If f and g are C 1 functions, it follows that the energy functional J ∈ C 2 , and the corresponding Nehari manifolds N ∈ C 1 , where N is given by   N = (u, v) ∈ H 1 (RN ) × H 1 (RN ) \ {0, 0} : J ′ (u, v)(u, v) = 0 . From these properties of J and N , one can easily deduce that critical points of J on N are critical points of J on H 1 (RN ) × H 1 (RN ). However, in present paper we cannot obtain these properties, since f and g are just continuous functions, and N is only a continuous submanifolds of H 1 (RN ) × H 1 (RN ). To overcome this difficulty, we should carefully study the elementary properties for N as in [32]. By doing this we can reduce variational problem for indefinite functional to minmax problem on a manifold and find positive solutions for (A ). Remark 1.4. (1) Obviously, if 2 ≤ q1 , q2 < 2∗ − 1, then f (s) = sq1 ln(1 + s) and g(s) = sq2 ln(1 + s) satisfy the conditions (F1 )–(F4 ). (2) If f (s) = g(s) = sp , then f and g satisfy the conditions (F1 )–(F4 ). Moreover, if p = 3, then (A ) reduces to the well known system (1.3). Next we concern with the nonexistence of nontrivial ground state solution of (A ). We assume that (R3 ) λT (x), κT (x) such that lim|x|→∞ λ(x) − λT (x) = 0, lim|x|→∞ κ(x) − κT (x) = 0, 0 < δ0 ≤ λT (x) ≤ λ(x) and 0 < δ0 ≤ κT (x) ≤ κ(x), where δ0 > 0 is a positive constant. Moreover, λT (x) ̸≡ λ(x) or κT (x) ̸≡ κ(x). Then the following results hold. Theorem 1.5. Assume that (R1 ), (R3 ) and (F1 )–(F3 ) hold. There exists βˆ∗ > 0 such that (A ) has no positive ground state solution for β > βˆ∗ . Remark 1.6. If F (tu) = t4 F (u) and G(tu) = t4 G(u) for each t, u ̸= 0, we can give an accurate definition of βˆ∗ and β ∗ (see (2.10) and (4.10)). However, we do not give it here in order to avoid introducing heavy notations at this stage. Remark 1.7. The condition (F2 ) can be replaced by the following weaker condition (F2′ )

1 2 f (s)s

− F (s) and 12 g(s)s − G(s) are strictly increasing on intervals (0, ∞).

Indeed, by a direct calculation one can show that the condition (F2′ ) is weaker than (F2 ) (see (4.7)). Moreover, one easily uses the condition (F2′ ) to prove w(t) has a unique global maximum point for t > 0, where w(t) is given in Lemma 2.1. 2. The periodic system Throughout the paper, we use the following notation: • Let | · |q denote the usual Lq -norm and (·, ·)2 be the usual L2 := L2 (RN )-inner product.  • ∥ · ∥ is the norm of H 1 (RN ) defined by ∥u∥2 = RN (|∇u|2 + u2 ).

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

219

 • ∥ · ∥M is a norm defined by ∥u∥2M = R3 (|∇u|2 + M |u|2 ), for a positive function or constant M . 1 • HM = HM (RN ) = {u ∈ H 1 (RN ) : ∥u∥2M < ∞} for a positive function or constant M . 1 1 • For z = (u, v) ∈ HM × HQ and ∥z∥2HM ×HQ = ∥u∥2M + ∥v∥2Q , where M and Q are positive functions or constants. Since the periodic system is playing an important role in proving the existence results for asymptotically periodic system, we shall study the periodic system here. That is, we concern with the following periodic system  −∆u + λT (x)u = µf (u) + βv 2 u in RN , (AT ) −∆v + κT (x)v = νg(v) + βu2 v in RN , where u, v ∈ H 1 (RN ), λT (x) and κT (x) satisfy the condition (R2 ). Before going further, we shall give some notations. For the Hilbert space H 1 (RN ), the inner product is denoted by  (u1 , u2 ) = (∇u1 ∇u2 + u1 u2 ), RN 1

and correspondingly the norm denoted by ∥u∥ = (u, u) 2 . Set E = H 1 (RN ) × H 1 (RN ). 1 is a Hilbert space Next we shall establish the variational setting for the system (AT ). Obviously, HQ endowed with inner product  1 (u, v)Q = ∇u∇v + Q(x)uv, for u, u ∈ HQ , RN

∥u∥2Q

and the induced norm denoted by = (u, u)Q , where Q is a positive function. Clearly, ∥ · ∥Q and ∥ · ∥ are equivalent norms if Q is a positive bounded function. Let ET = Hλ1T × Hκ1T . For z = (u, v) ∈ ET , ∥z∥2T = ∥u∥2λT + ∥v∥2κT . Then it is clear that ET = E. Now on ET we define the functional   1 β JT (z) = JT (u, v) = (∥u∥2λT + ∥v∥2κT ) − u2 v 2 . (µF (u) + νG(v)) − 2 2 RN RN u v for z = (u, v) ∈ ET , where F (u) = 0 f (s)ds and G(v) = 0 g(s)ds. Obviously, JT ∈ C 1 (ET , R) and a standard argument shows that critical points of JT are solutions of (AT ) (see [31,21]). We first consider the case β > 0 sufficient large. In order to find critical points for JT , we will use the Nehari methods as in [31,21]. The Nehari manifold corresponding to JT is defined by NT = {(u, v) ∈ ET \ {(0, 0)} : JT′ (u, v)(u, v) = 0}. Thus for z = (u, v) ∈ NT , one sees that   (|∇u|2 + λT (x)|u|2 + |∇v|2 + κT (x)|v|2 ) = RN

 (µf (u)u + νg(v)v) + 2β

RN

u2 v 2 .

From this one deduces that for z = (u, v) ∈ NT ,        1 β 1 f (u)u − F (u) + ν g(v)v − G(v) + u2 v 2 JT |NT (u, v) = µ 2 2 2 RN RN  1 = (|∇u|2 + |∇v|2 + λT (x)|u|2 + κT (x)|v|2 ) 4 RN       1 1 f (u)u − F (u) + ν g(v)v − G(v) . + µ 4 4 RN In the following we shall prove some elementary properties for Nε .

(2.1)

RN

(2.2)

220

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

Lemma 2.1. Assume that (R1 ) and (F1 )–(F3 ) hold. Moreover, λT and κT are continuous 1-periodic functions bounded from below by a positive constant. Then for µ, ν, β > 0 we have that for all z = (u, v) ∈ ST = {z = (u, v) ∈ ET : ∥z∥2T = ∥u∥2λT + ∥v∥2κT = 1}, there exists a unique tz > 0 such that mT (z) = tz z ∈ NT . z Conversely, for each z = (u, v) ∈ NT , we define m ˇ T (z) = m−1 T (z) = ∥z∥T ∈ ST . That is, the mapping mT is a homeomorphism between ST ⊂ ET and NT , and NT is a Nehari manifold. Proof. We follow the idea of [36, Lemma 2.2]. For each z = (u, v) ∈ ST and t > 0, we define w(t) = JT (tu, tv). It is easy to verify that w(0) = 0, w(t) < 0 for t > 0 large. Moreover, we claim that w(t) > 0

for t > 0 small.

(2.3)

In fact, we deduce from (F1 ) and (F3 ) that for each ϵ > 0 there exists Cϵ > 0 such that |f (t)|, |g(t)| ≤ ϵ|t| + Cϵ |t|p−1

and |F (t)|, |G(t)| ≤ ϵ|t|2 + Cϵ |t|p

(2.4)

where p is given in (F1 ). It follows that  t2 (|∇u|2 + |∇v|2 + λT (x)|u|2 + κT (x)|v|2 ) w(t) = JT (tu, tv) = 2 RN   β − (µF (tu) + νG(tv)) − t4 u2 v 2 . 2 RN RN t2 β ≥ (∥u∥2λT + ∥v∥2κT ) − ϵt2 (|u|22 + |v|22 ) − Cϵ (tp |v|pp + tq |u|pp ) − t4 |u|24 |v|24 2 2 t2 p 2 2 p q q 4 ≥ (1 − cϵ)(∥u∥λT + ∥v∥κT ) − cCϵ (t ∥u∥λT + t ∥v∥κT ) − ct (∥u∥4λT + ∥v∥4κT ). 2 Since p > 2, if ϵ small enough such that 1 − cϵ > 0, we obtain that w(t) > 0 for t > 0 small. Therefore, maxt>0 w(t) is achieved at t = tz > 0 so that w′ (tz ) = 0. Standard arguments show that tz z ∈ NT (see [37, Sect 4]). Suppose that there exists tz,1 > tz,2 > 0 such that tz,1 z, tz,2 z ∈ Nε . Then one has that   t2z,1 (∥u∥2λT + ∥v∥2κT ) = (µf (tz,1 u)tz,1 u + νg(tz,1 v)tz,1 v) + 2βt4z,1 u2 v 2 and RN RN   (2.5) 2 2 2 4 2 2 tz,2 (∥u∥λT + ∥v∥κT ) = (µf (tz,2 u)tz,2 u + νg(tz,2 v)tz,2 v) + 2βtz,2 u v . RN

RN

Then we see that        g(tz,1 v) g(tz,2 v) f (tz,1 u) f (tz,2 u) 2 2 2 2 − u + ν − v + 2β(tz,1 − tz,2 ) u2 v 2 , 0= µ tz,1 u tz,2 u tz,1 v tz,2 v RN RN RN which makes no sense in view of (F2 ) and tz,1 > tz,2 > 0. Conversely, the inverse of mT is given by m ˇ T (z) = m−1 T (z) =

z . ∥z∥T

That is, the mapping mT is a homeomorphism between ST ⊂ ET and NT .

(2.6) 

Lemma 2.2. Let the assumptions of Lemma 2.1 be satisfied. Then for µ, ν, β > 0 one can deduce NT has the following properties: (i) The set NT is bounded away from 0. Furthermore, NT is closed in ET . (ii) There is α > 0 such that tz ≥ α for each z ∈ ST and tz z ∈ NT . Moreover, for each compact subset W ⊂ ST , there exists CW > 0 such that tz ≤ CW , for all z ∈ W. (iii) cT = inf NT JT ≥ ρ > 0 and JT is bounded below on NT .

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

221

Proof. The conclusions (i) and (ii) can be accomplished by using the same arguments as in [36, Lemma 2.2]. Here we only prove the conclusion (iii). Indeed, for µ, ν, β, s > 0 and z = (u, v) ∈ ET \ {0}, similar to w(t) in Lemma 2.1 we obtain that for each ϵ > 0, there exists Cϵ > 0 such that    s2 β 4 2 2 2 2 JT (sz) = (|∇u| + |∇v| + λT (x)|u| + κT (x)|v| ) − (µF (su) + νG(sv)) − s u2 v 2 2 RN 2 RN RN s2 (1 − cϵ)(∥u∥2λT + ∥v∥2κT ) − sp cCϵ (∥v∥pλT + ∥u∥pκT ) − s4 c(∥v∥4λT + ∥u∥4κT ). ≥ 2 So, there is ρ > 0 such that JT (sz) ≥ ρ > 0 for s > 0 small. On the other hand, similar to [36, Remark 3.3], we deduce from Lemma 2.1 that cT = inf JT (u) = NT

inf

max JT (sz) = inf max JT (sz).

z∈ET \{(0,0)} s>0

So we get that cT ≥ ρ > 0 and JT |NT ≥ ρ > 0.

z∈ST s>0

(2.7)



Now we shall consider the functionals UˆT : ET \ {0} → R and UT : ST → R defined by UˆT (z) = JT (m ˆ T (z))

and UT = JT |ST ,

where m ˆ T (z) = tz z = tz (u, v) is the unique maximum of JT on ET . If {un } ⊂ ET such that JT (un ) → c andJT′ (un ) → 0, we say {un } a Palais–Smale sequence for JT , or shortly called (PS)-sequence for JT . As in [32], we have the following lemma. Lemma 2.3 (See Corollary 3.3 of [32]). Under the assumptions of Lemma 2.1, for µ, ν, β > 0 we have that (i) UT ∈ C 1 (ST , R), and UT′ (w)z = ∥mε (w)∥T JT′ (mε (w))z

for z ∈ Tw ST .

(ii) {wn } is a Palais–Smale sequence for UT if and only if {mT (wn )} is a Palais–Smale sequence for JT . If {un } ⊂ NT is a bounded Palais–Smale sequence for JT , then m ˇ T (zn ) is a Palais–Smale sequence −1 z for UT , where m ˇ T (z) = mT (z) = ∥z∥T . (iii) inf UT = inf JT = cT . ST

NT

Moreover, z ∈ ST is a critical point of UT if and only if mT (z) is a critical point of JT , and the corresponding critical values coincide. Concerning with the following single equation − ∆u + λT (x)u = µf (u),

u ∈ H 1 (RN )

(2.8)

222

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

and − ∆v + κT (x)v = νg(v),

v ∈ H 1 (RN ).

(2.9)

Following the idea of the proof of [36, Theorem 3.8], one can easily prove the following results. Lemma 2.4. Assume that λT and f satisfy the conditions (R1 )–(R2 ) and (F1 )–(F3 ) respectively. The following results hold. (i) (2.8) has at least one positive ground state solution u1 in H 1 (RN ). 1,σ (RN ) with σ ∈ (0, 1). Furthermore, there (ii) lim|x|→∞ u1 (x) = 0, lim|x|→∞ |∇u1 (x)| = 0 and u1 ∈ Cloc − εc |x−x1 | exist constants C, c > 0 such that |u1 (x)| ≤ Ce for all x ∈ RN , where u1 (x1 ) = maxx∈RN u(x). (iii) If κT and g satisfy the conditions (R1 )–(R2 ) and (F1 )–(F3 ) respectively, one has that the conclusions of (i)–(ii) remain true for (2.9). It is well-known that the main difficulty to prove the existence of nontrivial positive solutions(z = (u, v), u, v ̸≡ 0) for (AT ) is that we should exclude the semi-trivial solutions (u, 0) and (0, v). Here we shall borrow an idea of [31] to overcome this difficulty. Let U and V denote the positive ground state solution of (2.8) and (2.9) respectively. We define     F (U ) G(V ) ∗ RN RN ˆ   ,µ . (2.10) β1 = max ν U 2V 2 U 2V 2 RN RN Then we have the following main results. Lemma 2.5. There exists βˆ∗ ≥ βˆ1∗ such that any positive least energy solutions of (AT ) are nontrivial for β > βˆ∗ . That is, if z = (u, v) is a positive least energy solution of (AT ), then u, v ̸≡ 0 for β > βˆ∗ . Proof. It is sufficient to prove that cT < min{JT (U, 0), JT (0, V )}.

(2.11)

We distinguish the following two cases. (i) F (tu) = t4 F (u) and G(tu) = t4 G(u) for each t, u ̸= 0. So, F (u) = 14 f (u)u and G(v) = 14 g(v)v in this case. Set LT (z) = LT (u, v) =

(∥u∥2λT + ∥v∥2κT )2 . 4 RN (µF (u) + νG(v) + 2βu2 v 2 ) 

(2.12)

First, by using the same arguments as in [31, Lemma 3.3], we know that for β ≥ 0 cT =

inf

z=(u,v)∈ET \{(0,0)}

LT (z).

(2.13)

For s, t > 0, we define the function √ √ k(s, t) = LT ( sU, tV ) = 

(s∥U ∥2λT + t∥V ∥2κT )2 √ √ (µF ( sU ) + νG( tU ) + 2stβU 2 V 2 ) RN

(s∥U ∥2λT + t∥V ∥2κT )2 , (µs2 F (U ) + νt2 G(U ) + 2stβU 2 V 2 ) RN

= 

(2.14)

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

223

on the set Ω = {(s, t) : s ≥ 0, t ≥ 0, (s, t) ̸= (0, 0)}. Since F (U ) = 14 f (U )U , G(V ) = 41 g(V )V , U and V are solutions of (2.8) and (2.9) respectively, it follows that

k(s, 0) =

∥U ∥4λT  = JT (U, 0), µ RN F (U )

k(0, t) =

∥V ∥4κT  = JT (0, V ). ν RN G(V )

(2.15)

Thus, to guarantee (2.11) it is sufficient to prove that k does not attain its minimum over Ω on the lines s = 0 or t = 0. The function k is a fraction of two quadratic forms in (s, t), and elementary analysis shows that the quantity (As + Bt)2 , + 2Dst + Et2

Cs2

A, B, C, D, E > 0,

(2.16)

does not attain its minimum in Ω on the axes if and only if AD − BC > 0,

BD − AE > 0,

(2.17)

and then the minimum is attained for s = AD − BC and t = BD − AE. Clearly, (2.17) is equal to  µ∥V ∥2κT RN F (U )  β> ∥U ∥2λT RN U 2 V 2

 ν∥U ∥2λT RN G(V )  and β > . ∥V ∥2κT RN U 2 V 2

(2.18)

Moreover, since U and V are solutions of (2.8) and (2.9) respectively, it follows that   ν RN G(V ) µ∥V ∥2κT RN F (U )   = U 2V 2 ∥U ∥2λT RN U 2 V 2 RN

and

  ν∥U ∥2λT RN G(V ) µ RN F (U )   . = U 2V 2 ∥V ∥2κT RN U 2 V 2 RN

(2.19)

Thus, if     F (U ) N G(V ) ∗ R ˆ  RN β > β1 = max ν  , µ , U 2V 2 U 2V 2 RN RN

(2.20)

then (2.11) holds. (ii) F (tu) ̸= t4 F (u) or G(tv) ̸= t4 G(v) for t, u, v ̸= 0. As in Lemma 2.1, one can check that there exists t0 > 0 such that t0 (U, V ) ∈ NT . One infers from U, V > 0 that t20 (∥U ∥2λT

+ ∥V

∥2κT )

 (µf (t0 U )t0 U + νg(t0 V )t0 V ) +

= RN

≥

2βt40



U 2V 2.

2βt40



U 2V 2

RN

(2.21)

RN

Hence we arrive at  t0 ≤

∥U ∥2λT + ∥V ∥2κT  2β RN U 2 V 2

 12 1

:= β − 2 A0 .

(2.22)

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

224

On the other hand, one deduces from t0 (U, V ) ∈ NT and (F1 )–(F3 ) that for each ϵ > 0 there exists Cϵ > 0 such that cT ≤ JT (t0 U, t0 V )   t2 t4 β = 0 (∥U ∥2λT + ∥V ∥2κT ) − (µF (t0 U ) + νG(t0 V )) − 0 U 2V 2 2 2 RN RN     1 1 t4 β = f (t0 U )t0 U − F (t0 U ) + g(t0 V )t0 V − G(t0 V ) + 0 U 2V 2 2 2 RN RN 2  t40 β p 2 2 2 p p ≤ ϵt0 (|U |2 + |V |2 ) + Cϵ t0 (|U |p + |V |p ) + U 2V 2. 2 RN

(2.23)

Moreover, substituting (2.22) into (2.23) we obtain that cT

3 3 p A4 ≤ ϵβ −1 A20 (|U |22 + |V |22 ) + β − 2 Ap0 (|U |pp + |V |pp ) + 0 β −2 2 2 2



U 2V 2

RN

:= h(β).

(2.24)

So, to guarantee (2.11), it is sufficient to show that h(β) < min{JT (U, 0), JT (0, V )}.

(2.25)

lim h(β) = 0,

(2.26)

Since

β→∞

one infers that there exists βˆ2∗ > 0 such that if β > βˆ2∗

(2.27)

then (2.11) holds. Set βˆ∗ = max{βˆ1∗ , βˆ2∗ }. We know that if β > βˆ∗ the conclusions of this lemma hold.



Next, we shall study the mountain pass geometry for the functional JT . Lemma 2.6 (Mountain Pass Geometry, See [37]). For β > 0, the functional JT satisfies the following conditions: (i) There exist positive constants ϑ, α such that JT (z) ≥ ϑ for ∥z∥ε = α. (ii) There exists e ∈ ET with ∥e∥T > α such that JT (e) < 0. From Lemma 2.6, by using the Ambrosetti–Rabinowitz Mountain Pass Theorem without (PS)-condition (see [37]), it follows that there exists a (PS)-sequence {zn } ⊂ Eε such that JT (zn ) → CT′ = inf max JT (γ(t)) γ∈Γ 0≤t≤1

and JT′ (zn ) → 0,

(2.28)

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

225

where Γ = {γ ∈ C(ET , R) : JT (γ(0)) = 0, JT (γ(1)) < 0}. As in Proposition 3.11 of [30] (also see [1]), we shall use the following equivalent characterization of CT′ , which is more adequate to our purpose, given by CT′ =

inf

max JT (tz) = cT .

z∈ET \{0} t>0

(2.29)

Here in the last equality we used (2.7). Lemma 2.7. Let {zn : zn = (un , vn )} ⊂ NT be a minimizing sequence for JT . Then {zn } is bounded in ET . Moreover, there exist r, δ > 0 and a sequence {yn } ⊂ RN such that  lim inf |zn |2 ≥ δ > 0, n→∞

where Br (yn ) = {y ∈ R

N

Br (yn )

: |y − yn | ≤ r}.

Proof. We first prove the boundedness of {zn }. Arguing by contradiction, suppose that ∥zn ∥T → ∞ and JT (zn ) → CT . Let wn = ∥zznn∥T = (wn1 , wn2 ). Then up to a subsequence we assume that wn ⇀ w = (w1 , w2 ) and wn (x) → w(x) a.e. in RN . Moreover, we have either {wn } is vanishing, i.e.,  lim sup |wn |2 = 0 (2.30) n→∞ y∈RN

Br (y)

or non-vanishing, i.e., there exist r, δ > 0 and a sequence {yn } ⊂ RN such that  lim inf |wn |2 ≥ δ > 0. n→∞

(2.31)

Br (yn )

As in [18], we will show neither (2.30) nor (2.31) takes place and this will provide the desired contradiction. If {wn } is vanishing, Lions’s concentration compactness lemma (see [37]) implies wn → 0 in Lp (RN ) ×   L (RN )(∀p ∈ (2, 2∗ )). Therefore from (2.4) we deduce that RN F (ξwn1 ), RN G(ξwn2 ) → 0 as n → ∞ for each ξ ∈ R. So, we infer from Lemma 2.1 that p

cT + o(1) ≥ JT (zn ) ≥ JT (ξwn )    βξ 4  ξ2 ν µ 2 2 1 2 1 2 = (∥wn ∥λT + ∥wn ∥κT ) − F (ξwn ) + G(ξwn ) − |w1 |2 |wn2 |2 2 4 4 2 RN n RN ξ2 βξ 4 1 2 2 2 ξ2 ≥ − |wn |4 |wn |4 + o(1) = + o(1), 2 2 2 as n → ∞. Now we arrive a contradiction if ξ is large enough. Hence non-vanishing must hold. From (2.31), one infers that there exists {yn′ } ⊆ ZN such that  lim inf |wn |2 ≥ δ > 0. (2.32) n→∞

′ ) Br (yn

yn′ ).

Set w ˜n (x) = wn (x + Since ∥w ˜n ∥T = ∥wn ∥T = 1, there exist subsequences we still denoted by {w ˜n } N t N ∗ t such that w ˜n (x) ⇀ w0 (x) in ET and w ˜n (x) → w0 (x) in Lloc (R ) × Lloc (R )(∀t ∈ [2, 2 )). Moreover, w ˜n (x) → w0 (x) a.e. in RN . So, we infer from (2.32) that there exists nonempty set Ω ⊆ Br (0) such that meas(Ω ) > 0,

w ˜n (x) → w0 (x) ̸= 0

for x ∈ O.

Since |zn (x + yn′ )| = |wn (x + yn′ )| ∥zn ∥T , it follows that |zn (x + yn′ )| → ∞ in Ω , as n → ∞. Then it follows from (F3 ) and Fatou’s lemma that   µ ν    F (u ) + G(v ) 1 JT (zn ) n n 2 ≤ − min , |w | 0≤ n ∥zn ∥2T 2 4 4 |zn |2 RN

226

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

  µ ν    F (u (x + y )) + G(v (x + y )) 1 n n n n 2 = − min , |wn (x + yn )| 2 4 4 u2n (x + yn ) + vn2 (x + yn ) RN    µ ν  1 F (un (x + yn )) + G(vn (x + yn )) 2 ≤ − min , |wn (x + yn )| → −∞, 2 4 4 u2n (x + yn ) + vn2 (x + yn ) Ω as n → ∞. This is impossible. Thus, {zn } is bounded in ET . Next we prove the latter conclusion of this lemma. Since {zn } is bounded, if  lim sup |zn |2 = 0, n→∞ y∈RN

B(y,r)

we infer from Lions’s concentration compactness lemma that zn → 0 in Lp (RN ) × Lp (RN )(∀p ∈ (2, 2∗ ))   as n → ∞. We infer from (2.4) that RN f (vn )vn , RN g(un )un → 0, as n → ∞. Moreover, it follows from JT′ (zn )zn = 0 that   ∥un ∥2λT + ∥vn ∥2κT = (µf (un )un + νg(vn )vn ) + 2β u2n vn2 → 0, RN

RN

as n → ∞. Thus, we obtain zn → 0 in ET . This contradicts with NT is bounded away from 0.



Let us now state the main results for the periodic problem (AT ). Theorem 2.8. Let the assumptions of (F1 )–(F3 ) be satisfied. Then for β > β ∗ , the following results hold. (i) (AT ) has a positive ground state solution zT = (u, v). 1,σ (ii) lim|x|→∞ u(x) = lim|x|→∞ v(x) = 0, lim|x|→∞ |∇u(x)| = lim|x|→∞ |∇v(x)| = 0 and u, v ∈ Cloc (RN ) −θ0 |x| with σ ∈ (0, 1). Furthermore, there exists θ0 > 0 such that |u(x)| + |v(x)| ≤ ce . 1 N (iii) L is compact in H (R ), where L denotes the set of all positive ground state solutions of ( AT ). Proof. (i) From the conclusion (iii) of Lemma 2.2 we know that cT > 0. Moreover, if z0 ∈ NT satisfies JT (z0 ) = cT , then m ˇ T (z0 ) is a minimizer of UT and therefore a critical point of UT , so that z0 is a critical point of UT by Lemma 2.3. It remains to show that there exists a minimizer z of JT |NT . From Lemma 2.6 we know that JT satisfies the mountain pass geometry. Thus, using a version of the mountain pass theorem without (P S)c condition (see [37]), there exists {zn : zn = (un , vn )} ⊂ ET satisfying JT (zn ) → cT ,

and JT′ (zn ) → 0.

(2.33)

By Lemma 2.7, we know that {zn } is bounded there exist r′ , δ > 0 and a sequence {yn′ } ⊂ RN such that  lim inf |zn |2 ≥ δ > 0. n→∞

′ ,r ′ ) B(yn

So we can choose r > r′ > 0 and a sequence {yn } ⊂ ZN such that  δ lim inf |zn |2 ≥ > 0. n→∞ 2 B(yn ,r)

(2.34)

Set z˜n = (˜ un (x), v˜n (x)) = zn (x + yn ) = (un (x + yn ), vn (x + yn )). Since JT and NT are invariant under translations, we know that {˜ zn } is bounded in ET . Hence, one sees that z˜n ⇀ z = (u, v) ̸= 0 and JT′ (z) = 0. Thus, z ∈ NT . It remains to show that JT (z) = cT . From (F1 )–(F2 ) we can deduce that 1 f (s)s > F (s), 2

1 g(s)s > G(s) 2

for s ̸= 0.

(2.35)

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

227

So, we infer from (2.35) and Fatou’s lemma we get that     1 1 cT = lim inf JT (zn ) − JT′ (zn )zn = lim inf JT (˜ zn ) − JT′ (˜ zn )˜ zn n→∞ n→∞ 2 2        1 1 β 2 2 f (˜ un )˜ un − F (˜ un ) + ν g(˜ vn )˜ vn − G(˜ vn ) + u ˜n v˜n = lim inf µ n→∞ 2 2 2 R3        1 β 2 2 1 ≥ f (u)u − F (u) + ν g(v)v − G(v) + u v µ 2 2 2 R3 1 ′ = JT (z) − JT (z)z = JT (z). 2 Hence JT (z) ≤ cT . The reverse inequality follows from the definition of cT since z ∈ NT . Furthermore, since β > β ∗ , it follows that u, v ̸= 0. Finally, we need find a positive ground state solution for (AT ). In fact, for each z ∈ NT , there exists t > 0 such that t(|u|, |v|) ∈ NT . From the condition (F1 ) and the form of JT , we deduce that JT (t|u|, t|v|) ≤ JT (tu, tv). Furthermore, it follows from z ∈ NT that JT (tz) ≤ JT (z). So, we prove that JT (t|u|, t|v|) ≤ JT (z) and t(|u|, |v|) is a nonnegative ground state solution. It follows from maximum principle (see [14]) that zT := t(|u|, |v|) > 0 for all x ∈ RN . This finishes the proof of the conclusion (i). By using the same arguments as in [34,36], one can check the conclusions (ii) and (iii). Here we omitted the details.  Remark 2.9. From the assumptions (R1 )–(R2 ), we know that the spectrum is contained in (δ0 , +∞). By using the standard arguments (see [19]), one can prove that the periodic problem for the equation has infinitely many solutions. However, since our system (1.3) has semitrivial solutions, i.e., (u, 0) or (0, v), and the energy of these solutions may be very large, we cannot find the conditions to guarantee the solution is nontrivial, i.e., both components are nonzero. We shall consider this problem in the future. 3. Representation of Palais–Smale sequences Before going to discuss the behavior of Palais–Smale sequences, we first need to establish the variational setting for the system (A ). First, we use λ(x) and κ(x) to replace λT (x) and κT (x) in the beginning of Section 2. Then one can obtain the spaces Hλ1 and Hκ1 . Set Eλκ = Hλ1 × Hκ1 . Thus, we infer from the conditions (R1 ) and (R2 ) that Eλκ = E and the norms ∥ · ∥, ∥ · ∥λ and ∥ · ∥κ are equivalent. The Nehari manifold corresponding to J is defined by N = {(u, v) ∈ Eλκ \ {(0, 0)} : J ′ (u, v)(u, v) = 0} , where J is given in (1.4). Thus, for z = (u, v) ∈ N , one sees that    2 2 2 2 (|∇u| + λ(x)|u| + |∇v| + κ(x)|v| ) = (µf (v)v + νg(u)u) + 2β RN

RN

u2 v 2 .

(3.1)

RN

From this one deduces that for z = (u, v) ∈ N ,        1 1 β J |N (u, v) = µ f (v)v − F (u) + ν g(u)u − G(u) + u2 v 2 2 2 2 RN RN  1 = (|∇u|2 + |∇v|2 + λ(x)|u|2 + κ(x)|v|2 ) 4 RN       1 1 + µ f (v)v − F (u) + ν g(u)u − G(u) . 4 4 RN

(3.2)

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

228

Set Sλκ = {z = (u, v) ∈ Eλκ : ∥u∥2λ + ∥v∥2κ = 1}. Thus, using J , N , Eλκ , Sλκ , m and m ˇ to replace JT , NT , ET , ST , mT and m ˇ T , respectively, in Section 2, we infer that the conclusions of Lemmas 2.1–2.3 remain true. Moreover, we also have that c = inf J (u) = N

max J (sz) = inf max J (sz).

inf

z∈Eλκ \{(0,0)} s>0

z∈Sλκ s>0

(3.3)

Now we are ready to discuss the behavior of the Palais–Smale sequences of J . Lemma 3.1. Let {zn |zn = (un , vn )} ⊂ N ⊂ Eλκ be a sequence of J such that (p1 ) J (zn ) → c > 0; (p2 ) J ′ (zn ) → 0 strongly in Eλκ . Then replacing {zn }, if necessary, with a subsequence, there exist a solution z0 of (A ), a number ℓ ∈ N∪{0}, ℓ functions z˜1 , · · ·, z˜ℓ of E and ℓ sequences of points {ynj }, ynj ∈ ZN , 0 ≤ j ≤ ℓ, such that (i) (ii) (iii) (iv)

|ynj | → ∞(1 ≤ j ≤ ℓ), |yni − ynj | → ∞ if i ̸= j; ℓ zn − j=1 z˜j (· − ynj ) → z0 in Eλκ ; ℓ J (zn ) → J (z0 ) + j=1 JT (˜ zj ); z˜j are critical points of JT .

Proof. We follow the idea of [3,35]. As in the proof of Lemma 2.7, one can check that the boundedness of {zn }. Without loss of generality we assume that (un , vn ) ⇀ z0 = (u0 , v0 ) in E and (un , vn ) → (u0 , v0 ) a.e. in RN . Moreover, one has that J ′ (z0 ) = 0. If zn → z0 in Eλκ , we are done. So we can assume that zn does not converge strongly to z0 in E. Set wn,1 (x) = un (x) − u0 (x),

zn,1 (x) = vn (x) − v0 (x).

Obviously, zn,1 ⇀ 0 in E, but not strongly. We claim that J (un , vn ) = J (wn,1 + u0 , zn,1 + v0 ) = J (u0 , v0 ) + JT (wn,1 , zn,1 ) + o(1)

(3.4)

and J ′ (un , vn )(ϕ, ψ) = JT′ (wn,1 , zn,1 )(ϕ, ψ) + o(1) We deduce from (R2 ) and zn → z0 in

Lploc (RN )

×

Lploc (RN )(∀p

for all (ϕ, ψ) ∈ E.

(3.5)

∈ [1, 2∗ )) that

∥un ∥2λ = ∥wn,1 + u0 ∥2λ = ∥wn,1 ∥2λT + ∥u0 ∥2λ + o(1) ∥vn ∥2λ = ∥zn,1 + v0 ∥2λ = ∥zn,1 ∥2λT + ∥v0 ∥2λ + o(1),

(3.6)

and (un , ϕ)λ = (wn,1 + u0 , ϕ)λ = (wn,1 , ϕ)λT + (u0 , ϕ)λ + o(1) (vn , ψ)λ = (zn,1 + v0 , ψ)λ = (zn,1 , ψ)λT + (v0 , ϕ)λ + o(1).

(3.7)

So, we have that 

J (un , vn ) = J (u0 , v0 ) + JT (wn,1 , zn,1 ) + µ (F (v0 ) + F (zn,1 ) − F (vn )) RN  +ν (G(u0 ) + G(wn,1 ) − G(un )) + o(1) RN

J (un , vn )(ϕ, ψ) = J ′ (u0 , v0 )(ϕ, ψ) + JT′ (wn,1 , zn,1 )(ϕ, ψ) + o(1)   +µ (f (v0 ) + f (zn,1 ) − f (vn ))ψ + ν (g(u0 ) + g(wn,1 ) − g(un ))ϕ. ′

RN

RN

(3.8)

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

229

By (F4 ), a standard argument shows that             (G(u ) + G(w ) − G(u )) + (F (v ) + F (z ) − F (v )) 0 n,1 n  → 0, 0 n,1 n   N  N R R          → 0, +  (g(u ) + g(w ) − g(u ))ϕ (f (v ) + f (z ) − f (v ))ψ 0 n,1 n 0 n,1 n    

(3.9)

RN

RN

uniformly for ∥(ϕ, ψ)∥Eλκ ≤ 1 (see Proposition 4.2 of [19]). Therefore, (3.3) and (3.4) hold. Set 

|wn,1 |p ,

δw = lim sup sup n→∞ y∈RN



|zn,1 |p .

δz = lim sup sup n→∞ y∈RN

B1 (y)

(3.10)

B1 (y)

Let us show that δw + δz > 0. Otherwise, both of them are equal to zero, and then Lions’s concentration compactness lemma (see [37]) implies that wn,1 → 0, zn,1 → 0 strongly in Lt (RN )(∀t ∈ (2, 2∗ )). We deduce from (3.5) that JT′ (wn,1 , zn,1 )(ϕ, ψ) = o(1). So, one has   2 2 ∥wn,1 ∥2λT + ∥zn,1 ∥2λT = (µf (zn,1 )zn,1 + νg(wn,1 )wn,1 ) + 2β wn,1 zn,1 → 0, (3.11) RN

RN

as n → ∞. This contradicts with (wn,1 , zn,1 ) does not converge strongly to (0, 0). Thus, we can assume that δw > 0. Similar to (2.34) we can choose yn1 ∈ ZN such that  δw |wn,1 |p ≥ . (3.12) 2 1) B1 (yn Let us consider the sequence (wn,1 (x + yn1 ), zn,1 (x + yn1 )). We may assume that (wn,1 (x + yn1 ), zn,1 (x + yn1 )) ⇀ u1 , v˜1 ) a.e. on RN . It follows from (3.12) that (˜ u1 , v˜1 ) = z˜1 in Eλκ and (wn,1 (x + yn1 ), zn,1 (x + yn1 )) ⇀ (˜ u ˜1 ̸= 0. However, (wn,1 , zn,1 ) ⇀ (0, 0). So, {yn1 } is an unbounded sequence. Up to a subsequence we may assume that |yn1 | → ∞. Moreover, it follows from (3.5) that z˜1 is a solution of (AT ). We set (wn,2 (x), zn,2 (x)) = (wn,1 (x) − u ˜1 (x − yn1 ), zn,1 (x) − v˜1 (x − yn1 )).

(3.13)

Similar to (3.4)–(3.5), one has JT (wn,1 , zn,1 ) = JT (˜ u1 , v˜1 ) + JT (wn,2 , zn,2 ) + o(1)

(3.14)

and JT′ (wn,2 , zn,2 )(ϕ, ψ) = o(1)

for all (ϕ, ψ) ∈ E.

(3.15)

Now, if (wn,2 , zn,2 ) → 0 in Eλκ , we are done. Otherwise, (wn,2 , zn,2 ) ⇀ 0 and not strongly and we repeat the argument. By iterating this procedure we obtain sequences of points ynj ∈ ZN such that |ynj | → ∞, |yni − ynj | → ∞ if j ̸= i, as n → ∞, and a sequence of functions (wn,j (x), zn,j (x)) = (wn,j−1 (x) − u ˜j−1 (x − ynj−1 ), zn,j (x) − j−1 v˜j−1 (x − yn )) such that (wn,j (x + ynj ), zn,j (x + ynj )) ⇀ (˜ uj (x), v˜j (x))

in Eλκ ,

JT′ (˜ uj , v˜j ) = 0,

(3.16)

and J (zn ) = J (z0 ) +

k 

JT (˜ zj ) + JT (wn,k+1 , zn,k+1 ) + o(1).

(3.17)

j=1

Since JT (˜ zj ) ≥ CT > 0 for all j, then by (3.17) the iteration must stop at some finite index.



J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

230

4. Proof of the main results We first recall some information for semitrivial solutions, i.e., the solution of a form (u, 0) or (0, v). Precisely, we concern with the following single equation − ∆u + λ(x)u = µf (u),

u ∈ H 1 (RN )

(4.1)

− ∆v + κ(x)v = νg(v),

v ∈ H 1 (RN ).

(4.2)

and

From the results of Lemma 2.4, one can use the idea of [2,36] to prove the following results. Lemma 4.1. Assume that λ and f satisfy the conditions (R1 )–(R2 ) and (F1 )–(F3 ), respectively. The following results hold. (i) (4.1) has at least one positive ground state solution U1 in H 1 (RN ). 1,σ (ii) lim|x|→∞ U1 (x) = 0, lim|x|→∞ |∇U1 (x)| = 0 and U1 ∈ Cloc (RN ) with σ ∈ (0, 1). Furthermore, there c − ε |x−x1 | for all x ∈ RN , where U1 (x1 ) = maxx∈RN U1 (x). exist constants C, c > 0 such that |U1 (x)| ≤ Ce (iii) If κ and g satisfy the conditions (R1 )–(R2 ) and (F1 )–(F3 ) respectively, one has that the conclusions of (i)–(ii) remain true for (4.2). Let d1 =

inf (u,0)∈N

dT,1 =

J (u, 0),

inf (u,0)∈NT

JT (u, 0),

d2 =

inf (0,v)∈N

dT,2 =

J (0, v), inf

(0,v)∈NT

JT (0, v).

(4.3)

According to Lemmas 2.4 and 4.1, we know that d1 = J (U1 , 0), d2 = J (0, V1 ), dT,1 = JT (U, 0) and dT,2 = JT (0, V ), for some positive U1 , V1 , U, V ∈ H 1 (RN ). Moreover, one sees that c ≤ min{d1 , d2 }.

(4.4)

Lemma 4.2. Under the assumptions of Theorem 1.1, for each β > 0, we have that cT and c are attained by z ∈ NT and z ∈ N , respectively. Proof. One uses the same arguments as in Theorem 2.8 to prove cT is attained. We assume that cT = JT (zT ) = JT (uT , vT ). We deduce from (R2 ) that   ∥zT ∥2λκ ≤ ∥zT ∥2λT κT = (µf (vT )vT + νg(uT )uT ) + 2β u2T vT2 . (4.5) RN

RN

So, there is tT ∈ (0, 1) such that tT zT ∈ N . It follows from (3.2) that c ≤ J (tT zT )        1 1 t4T β = µ f (tT vT )tT vT − F (tT vT ) + ν g(tT uT )tT uT − G(tT uT ) + u2 v 2 2 2 2 RN T T RN        1 1 β ≤ µ f (vT )vT − F (vT ) + ν g(uT )uT − G(uT ) + u2 v 2 2 2 2 RN T T RN = JT (zT ) = cT .

(4.6)

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

231

Here we used the facts that H1 (s) = 12 f (s)s − F (s) and H2 (s) = 12 g(s)s − G(s) are increasing functions for s > 0. This can be accomplished by the condition (F2 ). We only prove H1 (s) is increasing function. Indeed, one deduces from (F2 ) that for s1 ≥ s2 > 0  s1  f (s) 1 f (s1 ) s1 f (s1 ) 2 F (s1 ) − F (s2 ) = s sds = f (s1 )s1 − ds ≤ s s s1 2 2s1 2 s2 s2 ≤

1 1 f (s1 )s1 − f (s2 )s2 . 2 2

(4.7)

That is, H1 (s1 ) ≥ H1 (s2 ). So, we infer from (4.6) that c ≤ cT . Next we consider two cases: c = cT and c < cT . (1) If c = cT takes place, then by (4.6), we have tT = 1. This implies that zT ∈ N and c = J (zT ). This is our conclusion. (2) If c < cT takes place, then as in the proof of Theorem 2.8, there exists a (P S)c sequence {zn }. By Lemma 3.1, there exist subsequence ℓ ∈ N, z˜1 , · · ·, z˜ℓ of E and ℓ sequences of points {ynj }, ynj ∈ Z N , 0 ≤ j ≤ ℓ, such that (i) |ynj | → ∞(1 ≤ j ≤ ℓ), |yni − ynj | → ∞ if i ̸= j; k (ii) zn − j=1 z˜j (· − ynj ) → z0 in Eλκ ; k (iii) J (zn ) → J (z0 ) + j=1 JT (˜ zj ); (iv) z˜j are critical points of JT . From the proof of Lemma 3.1, we know that z˜j ̸= (0, 0). So, cT ≤ JT (˜ zj ). We infer from c < cT that ℓ = 0. This implies that zn → z0 This shows that z0 ∈ N and J (z0 ) = c.

in Eλκ .

(4.8)



From Lemma 4.2 we know that c may be attained by semitrivial solutions of the form (u, 0) or (0, v). In the following we shall prove that c is attained by nontrivial solution for β > 0 sufficiently large. To do this, it suffices to show that c < min{d1 , d2 } = min{J (U1 , 0), J (0, V1 )}.

(4.9)

Next we shall give two conditions to guarantee (4.9) holds. On the one hand, replacing U, V by U1 , V1 in the proof of Lemma 2.5, we infer that there exist β1∗ > 0 and β2∗ > 0 such that β > β ∗ , c is attained by nontrivial solution (u, v) and u, v ̸= 0, where     G(V1 ) N F (U1 ) ∗ ∗ ∗ ∗ R RN  β = max{β1 , β2 } and β1 = max µ  , ν , (4.10) U 2V 2 U 2V 2 RN 1 1 RN 1 1 β2∗ is defined the same as in Lemma 2.5. On the other hand, if we additionally assume that f, g ∈ C 1 (R), we can follow the idea of [4,16] to find other conditions to guarantee (4.9) holds. Set βˆ = min{βˆ1 , βˆ2 },

(4.11)

where βˆ1 =

inf (u,0)∈D1

inf

ϕ∈H 1 (RN )\{0}



∥ϕ∥2λ , u2 ϕ2 RN

βˆ2 =

inf (0,v)∈D2

inf

φ∈H 1 (RN )\{0}



∥φ∥2κ v 2 φ2 RN

(4.12)

232

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

and D1 = {(u, 0) ∈ N : J (u, 0) = d1 },

D2 = {(0, v) ∈ N : J (0, v) = d2 }.

(4.13)

ˆ (4.9) holds. Indeed, it follows from β > βˆ that there exist (u, 0) ∈ D1 , (0, v) ∈ D2 , We claim that for β > β, 1 N ϕ, φ ∈ H (R ) such that ∥ϕ∥2λ < β, u2 ϕ2 RN





∥φ∥2κ < β. v 2 φ2 RN

(4.14)

Since for each ψ1 , ψ2 ∈ H 1 (RN ), one has that J ′ (u, 0)(0, ψ2 ) = 0,

J ′ (0, v)(ψ1 , 0) = 0.

(4.15)

So, it is easily to see that {0} × H 1 (RN ) ⊂ T(u,0) N = {z ∈ E : J (u, 0)z = 0} and H 1 (RN ) × {0} ⊂ T(0,v) rN = {z ∈ E : J (0, v)z = 0}. Let η1 , η2 ∈ C 2 ((−ϵ, ϵ), N ) satisfy η1′ (0) = (0, φ),

η1 (0) = (u, 0),

η2 (0) = (0, v),

η2′ (0) = (ϕ, 0).

(4.16)

We infer from the Taylor expansion of J (ηi (t))(i = 1, 2) and J ′ (u, 0) = J ′ (0, v) = 0 that 1 J (ηi (t)) = J (ηi (0)) + J ′′ (ηi (0))[ηi′ (0), ηi′ (0)]t2 + o(t2 ). 2

(4.17)

Since J (u, 0)[(0, φ), (0, φ)] = ′′

∥φ∥2κ

 −β

u2 φ2 < 0,

(4.18)

RN

it follows that J (η1 (t)) − J (η1 (0)) < 0. Similarly, one can check that J (η2 (t)) − J (η1 (0)) < 0. Thus, (4.9) holds. Proof of Theorem 1.1. From the above arguments we know that for β > β ∗ , (A ) has a nontrivial ground state solution z = (u, v) ∈ N such that c = J (u, v). By same arguments as in Theorem 2.8 we can find a positive ground state zˆ for (A ). This proves the conclusion (i) of Theorem 1.1. The conclusions (ii) and (iii) follow the same arguments as in Theorem 2.8.  Proof of Theorem 1.5. We use the idea of [35]. First we claim that c = cT .

(4.19)

Indeed, it follows from λT (x) ≤ λ(x) and λT (x) ≤ λ(x) that JT (u, v) ≤ J (u, v) for each u, v ∈ H 1 (RN ). Moreover, one deduces from (2.7) and (3.3) that c ≥ cT . Next we shall prove that c ≤ cT . Since β > βˆ∗ , it follows from Theorem 2.8 that (AT ) has a nontrivial positive ground state solution (u0 , v0 ). Moreover, (u0 , v0 ) is the unique global maximum of JT (tu0 , tv0 ). Set wn = (u0 (· − yn ), v0 (· − yn )), where yn ∈ RN and |yn | → ∞ as n → ∞. Then by Lemma 2.1, it follows that there exists tn > 0 such that tn wn ∈ N is the unique global maximum of J (tn wn ) for each n. One can check that {tn } is bounded. In fact, if tn → ∞, one sees that J (tn wn ) → −∞ as n → ∞. However, J (tn wn ) ≥ 0. A contradiction. Thus, {tn } is bounded. Therefore, we get c ≤ J (tn wn )   t2 t2 = n (|∇u0 (x − yn )|2 + λ(x)|u0 (x − yn )|2 ) + n (|∇v0 (x − yn )|2 2 RN 2 RN  βt4 |u0 (x − yn )|2 |v0 (x − yn )|2 + κ(x)|v0 (x − yn )|2 ) − n 2 RN

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

233

 −

(µF (u0 (x − yn ))) + νG(v0 (x − yn ))   t2 t2 (λ(x + yn ) − λT )u20 + n (κ(x + yn ) − κT )v02 = JT (tn u0 , tn v0 ) + n 2 RN 2 RN   t2n t2n 2 ≤ cT + (λ(x + yn ) − λT )u0 + (κ(x + yn ) − κT )v02 . 2 RN 2 RN RN

It is clear that for each ϵ > 0, there exists R > 0 such that  (λ(x + yn ) − λT (x))u20 ≤ cϵ.

(4.20)

(4.21)

|x|≥R

Moreover, we conclude from Lebesgue’s dominated convergence theorem that   2 (λ(x + yn ) − λT (x))u0 = ( lim λ(x + yn ) − λT (x))u20 = 0. lim n→∞

|x|
|x|
(4.22)

Thus, we proved that  RN

(λ(x + yn ) − λT (x))u20 = o(1)

as n → ∞.

(4.23)

(λ(x + yn ) − λT (x))v02 = o(1)

as n → ∞.

(4.24)

Similarly, one can check that  RN

So it follows from the boundedness of {tn } and (4.20)–(4.24) that c = cT for each β > βˆ∗ . In order to prove the main result we use the contradiction argument. Assume, for some β0 > βˆ∗ that there exists positive zˆ such that zˆ = (ˆ u, u ˆ) ∈ N (ˆ u, vˆ ̸= 0) and c = JT (ˆ z ). Clearly, zˆ is the unique global maximum of Lε0 (tˆ z ). First, one sees that CT ≤ JT (tT zˆ) = maxt>0 JT (tˆ z ). In addition, by (R3 ), N it follows that λ(x) ≥ λT (x) and κ(x) ≥ κT (x) for all x ∈ R and JT (z) ≤ J (z) for each z ∈ E. Thus, CT ≤ JT (tT zˆ) ≤ J (tT zˆ) ≤ J (ˆ z ) = c = CT . This implies cT = JT (tT zˆ) = J (tT zˆ). However, one has   1 1 (λT (x) − λ(x))u21 + (κT (x) − κ(x))v12 . (4.25) JT (u1 , v1 ) = J (u1 , v1 ) + 2 RN 2 RN Furthermore, we deduce from (R3 ) that   1 1 2 (λT (x) − λ(x))u1 + (κT (x) − κ(x))v12 < 0 2 RN 2 RN where (u1 , v1 ) = tT zˆ. Thus, JT (u1 , v1 ) < J (u1 , v1 ). This is a contradiction.

(4.26) 

Acknowledgments The authors are grateful the referee’s thoughtful reading of details of the paper and nice suggestions to improve the results. This work was supported by Natural Science Foundation of China (Nos. 11201186, 11571140), Natural Science Foundation of Jiangsu Province (Nos. BK2012282, BK20140525, BK20150478), Jiangsu University foundation grant (Nos. 11JDG117, 13JDG121, 14JDG070), China Postdoctoral Science Foundation funded project (Nos. 2013T60499, 2015M571710). Social Science Foundation of China (No: 15CRK010), Jiangsu Postdoctoral Science Foundation funded project (1401055C).

234

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

References [1] C.O. Alves, M.A.S. Souto, On existence and concentration behavior of ground state solutions for a class of problems with critical growth, Commun. Pure Appl. Anal. 1 (3) (2002) 417–431. [2] C.O. Alves, M.A.S. Souto, M. Montenegro, Existence of solution for two classes of elliptic problems in RN with zero mass, J. Differential Equations 252 (10) (2012) 5735–5750. [3] A. Ambrosetti, G. Cerami, D. Ruiz, Solitons of linearly coupled systems of semilinear non-autonomous equations on Rn , J. Funct. Anal. 254 (11) (2008) 2816–2845. [4] A. Ambrosetti, E. Colorado, Standing waves of some coupled nonlinear Schr¨ odinger equations, J. Lond. Math. Soc. (2) 75 (1) (2007) 67–82. [5] T. Bartsch, N. Dancer, Z.-Q. Wang, A Liouville theorem, a-priori bounds, and bifurcating branches of positive solutions for a nonlinear elliptic system, Calc. Var. Partial Differential Equations 37 (3–4) (2010) 345–361. [6] T. Bartsch, Z.-Q. Wang, Note on ground states of nonlinear Schr¨ odinger systems, J. Partial Differential Equations 19 (3) (2006) 200–207. [7] T. Bartsch, Z.-Q. Wang, J.-C. Wei, Bound states for a coupled Schr¨ odinger system, J. Fixed Point Theory Appl. 2 (2) (2007) 353–367. [8] S.-M. Chang, C.-S. Lin, T.-C. Lin, W.-W. Lin, Segregated nodal domains of two-dimensional multispecies Bose–Einstein condensates, Physica D 196 (3–4) (2004) 341–361. [9] Z.-J. Chen, W.-M. Zou, Positive least energy solutions and phase separation for coupled Schr¨ odinger equations with critical exponent, Arch. Ration. Mech. Anal. 205 (2) (2012) 515–551. [10] Z.-J. Chen, W.-M. Zou, An optimal constant for the existence of least energy solutions of a coupled Schr¨ odinger system, Calc. Var. Partial Differential Equations 48 (3–4) (2013) 695–711. [11] Z.-J. Chen, W.-M. Zou, Standing waves for coupled nonlinear Schr¨ odinger equations with decaying potentials, J. Math. Phys. 54 (11) (2013) 1–21. [12] D.-G. de Figueiredo, O. Lopes, Solitary waves for some nonlinear Schr¨ odinger systems, Ann. Inst. H. Poincar´ e Anal. Non Lin´ eaire 25 (1) (2008) 149–161. [13] B. Esry, C. Greene, J. Burke, J. Bohn, Hartree–Fock theory for double condensates, Phys. Rev. Lett. 78 (1997) 3594–3597. [14] D. Gilbarg, Neil S. Trudinger, Elliptic Partial Differential Equations of Second Order, second ed., in: Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 224, Springer-Verlag, Berlin, 1983. [15] X.-M. He, W.-M. Zou, Existence and concentration behavior of positive solutions for a Kirchhoff equation in R3 , J. Differential Equations 252 (2) (2012) 1813–1834. [16] N. Ikoma, Existence of standing waves for coupled nonlinear Schr¨ odinger equations, Tokyo J. Math. 33 (1) (2010) 89–116. [17] N. Ikoma, K. Tanaka, A local mountain pass type result for a system of nonlinear Schr¨ odinger equations, Calc. Var. Partial Differential Equations 40 (3–4) (2011) 449–480. [18] L. Jeanjean, K. Tanaka, Singularly perturbed elliptic problems with superlinear or asymptotically linear nonlinearities, Calc. Var. Partial Differential Equations 21 (3) (2004) 287–318. [19] W. Kryszewski, A. Szulkin, Generalized linking theorem with an application to a semilinear Schr¨ odinger equation, Adv. Differential Equations 3 (3) (1998) 441–472. [20] T.C. Lin, J.C. Wei, Ground state of N coupled nonlinear Schr¨ odinger equations in Rn , n ≤ 3, Comm. Math. Phys. 255 (3) (2005) 629–653. [21] T.C. Lin, J.C. Wei, Spikes in two coupled nonlinear Schr¨ odinger equations, Ann. Inst. H. Poincar´ e Anal. Non Lin´ eaire 22 (4) (2005) 403–439. [22] T.-C. Lin, J.-C. Wei, Spikes in two-component systems of nonlinear Schr¨ odinger equations with trapping potentials, J. Differential Equations 229 (2) (2006) 538–569. [23] Z.-L. Liu, Z.-Q. Wang, Multiple bound states of nonlinear Schr¨ odinger systems, Comm. Math. Phys. 282 (3) (2008) 721–731. [24] L. Maia, E. Montefusco, B. Pellacci, Positive solutions for a weakly coupled nonlinear Schr¨ odinger system, J. Differential Equations 229 (2) (2006) 743–767. [25] Rainer Mandel, Minimal energy solutions for repulsive nonlinear Schr¨ odinger systems, J. Differential Equations 257 (2) (2014) 450–468. [26] Rainer Mandel, Minimal energy solutions for cooperative nonlinear Schr¨ odinger systems, NoDEA Nonlinear Differential Equations Appl. 22 (2) (2015) 239–262. [27] M. Mitchell, M. Segev, Self-trapping of incoherent white light, Nature 387 (6636) (1997) 880–883. [28] E. Montefusco, B. Pellacci, M. Squassina, Semiclassical states for weakly coupled nonlinear Schr¨ odinger systems, J. Eur. Math. Soc. (JEMS) 10 (1) (2008) 47–71. [29] A. Pomponio, Coupled nonlinear Schr¨ odinger systems with potentials, J. Differential Equations 227 (1) (2006) 258–281. [30] P.H. Rabinowitz, On a class of nonlinear Schr¨ odinger equations, Z. Angew. Math. Phys. 43 (2) (1992) 270–291. [31] B. Sirakov, Least energy solitary waves for a system of nonlinear Schr¨ odinger equations in Rn , Comm. Math. Phys. 271 (1) (2007) 199–221. [32] A. Szulkin, T. Weth, The method of Nehari manifold, in: Handbook of Nonconvex Analysis and Applications, Int. Press, Somerville, MA, 2010, pp. 597–632. [33] E. Timmermans, Phase separation of Bose–Einstein condensates, Phys. Rev. Lett. 81 (1998) 5718–5721. [34] J. Wang, J.-P. Shi, Standing waves of a weakly coupled Schr¨ odinger system with distinct potential functions, J. Differential Equations 260 (2) (2016) 1830–1864.

J. Wang et al. / Nonlinear Analysis 134 (2016) 215–235

235

[35] J. Wang, L.-X. Tian, J.-X. Xu, F.-B. Zhang, Existence and nonexistence of the ground state solutions for nonlinear Schr¨ odinger equations with nonperiodic nonlinearities, Math. Nachr. 285 (11–12) (2012) 1543–1562. [36] J. Wang, L.-X. Tian, J.-X. Xu, F.-B. Zhang, Multiplicity and concentration of positive solutions for a Kirchhoff type problem with critical growth, J. Differential Equations 253 (7) (2012) 2314–2351. [37] M. Willem, Minimax Theorems, in: Progress in Nonlinear Differential Equations and their Applications, vol. 24, Birkh¨ auser Boston Inc., Boston, MA, 1996.