Least energy solutions for a weakly coupled fractional Schrödinger system

Linear Algebra Applications Nonlinear Analysis and 132 its (2016) 141–159 466 (2015) 102–116

Contents lists at ScienceDirect Contents lists available at available ScienceDirect

Linear Algebra and its Applications Nonlinear Analysis www.elsevier.com/locate/laa www.elsevier.com/locate/na

Inverse eigenvalue problem of fractional Jacobi matrix Least energy solutions for a weakly coupled Schr¨odinger with mixed data system YingHe Wei 1 Qing Guo ∗ , Xiaoming College of Science, Minzu University China, Beijing 100081, China of Aeronautics and Astronautics, Department of of Mathematics, Nanjing University Nanjing 210016, PR China

article

info a r t i c l e

abstract i n f o

a b s t r a c t

Article history: Received 7 July 2015 Article history: In this paper, we consider following system of two weakly fractional In thisthe paper, the inverse eigenvalue problemcoupled of reconstructing Received 16 January 2014 Accepted 2 November 2015 a Jacobi matrix from its eigenvalues, its leading principal nonlinear odinger equations Accepted 20 September 2014 Schr¨ Communicated by S. Carl submatrix and part of the eigenvalues of its submatrix    Available online 22 October 2014 is considered. The necessary and for 2p p−1 p+1sufficient conditions N Submitted by Y. Wei  (−∆)s u + u = | u | +b(x) | u| | v| u x ∈ Rare MSC: the existence and uniqueness of the solution derived.   35J20 p+1 Furthermore,2pa numerical p−1  (−∆)s v + ω 2s MSC: v = | v | +b(x) | v | algorithm | u | andv some x ∈ numerical RN . 35J65 examples are given. 15A18 Keywords: © 2014 Published by Elsevier Inc. 15A57 By use of the s-harmonic extension technique, we establish the existence of a nonFractional Schr¨ odinger system trivial least energy solution of the system via variational methods. Especially, in the s-harmonic extension Keywords: autonomous case i.e. b(x) ≡ b, a positive least energy solution with both nontrivial Least energy solution Jacobi matrix components is obtained. Variational methods Eigenvalue © 2015 Elsevier Ltd. All rights reserved. Inverse problem Submatrix

1. Introduction and main results In this paper, we are concerned with the problem of the following fractional nonlinear Schr¨odinger system,    (−∆)s u + u = |u|2p + b(x)|u|p−1 |v|p+1 u x ∈ RN   (1.1) (−∆)s v + ω 2s v = |v|2p + b(x)|v|p−1 |u|p+1 v x ∈ RN , where s ∈ (0, 1), (−∆)s stands for the fractional Laplacian and  +∞ if N ≤ 2s ∗ E-mail address: [email protected]. 2 < 2p + 2 < 2 := 2N s 1  Tel.: +86 13914485239. if N > 2s. N − 2s Here the fractional http://dx.doi.org/10.1016/j.laa.2014.09.031 Laplacian (−∆)s of a function φ ∈ S is defined by 0024-3795/© 2014 Published by Elsevier Inc. s 2s

F((−∆) φ)(ξ) = |ξ| F(φ)(ξ),

∀ s ∈ (0, 1),

∗ Corresponding author. E-mail addresses: [email protected] (Q. Guo), [email protected] (X. He).

http://dx.doi.org/10.1016/j.na.2015.11.005 0362-546X/© 2015 Elsevier Ltd. All rights reserved.

(1.2)

142

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

where S denotes the Schwartz space of rapidly decreasing C ∞ functions in RN , F is the Fourier transform,  i.e. for any φ ∈ S, Fφ(ξ) = 1 N RN e−2πξ·x φ(x)dx. If φ is smooth enough, (−∆)s can also be computed (2π)

2

by the singular integral: s



(−∆) φ(x) = cN,s P.V . RN

φ(x) − φ(y) dy, |x − y|N +2s

where P.V . is the principal value and cN,s is a normalization constant. s The so-called fractional Schr¨ odinger equation i ∂ψ ∂t = (−∆) ψ + V (x)ψ was introduced by Laskin [19,18] through expanding the Feynman path integral from the Brownian-like to the L´evy-like quantum mechanical paths, where (x, t) ∈ RN × (0, ∞), 0 < s < 1, and V : RN → R is an external potential function. Similar to the case s = 1, standing wave solutions to this equation are solutions of the form ψ(x, t) = e−iωt u(x), where u solves the elliptic equation (−∆)s u + (V (x) − ω)u = 0. Recently, the following fractional Schr¨ odinger equation (1.3) with a nonlinear source term is of much interest, and attracts much attention in nonlinear analysis. (−∆)s u + V (x)u = f (x, u).

(1.3)

Particularly, in R1 , Frank and Lenzmann [15] studied the uniqueness and nondegeneracy of the ground state of (−∆)s u + u = uα+1 .

(1.4)

Very recently, Fall and Valdinoci [12] have extended this result in any dimension N when s is sufficiently close to 1. Later, the general case for s ∈ (0, 1) was done by [16] according to which, there is a unique ground state solution u ∈ H 2s+1 ∩ C ∞ , which is positive, radially symmetric, radially decreasing and satisfies the following asymptotic decay properties C1 C2 ≤ u(x) ≤ , 1 + |x|N +2s 1 + |x|N +2s

for all x ∈ RN ,

(1.5)

where 0 < C1 < C2 are constants. Concerning the regularity of the solutions, one can refer to [6,13], etc. Felmer Quaas and Tan [13] studied the existence and regularity of positive solutions of (1.4) with a general nonlinearity f (x, u) with subcritical growth satisfying the Ambrosetti–Rabinowitz condition. In [27], Secchi obtained the existence of the ground state solutions of (1.3). X. Chang and Z-Q. Wang [8] established a Pohozaev identity for (1.3) with power-type nonlinearities. For the results of fractional elliptic equations in bounded domain, we refer to [7,28,29,24,25] and the references therein. In this paper we focus our attention on the weakly coupled fractional elliptic system (1.1). It is easy to see that when u = 0 or v = 0, or u = v, system (1.1) reduces to the single fractional Schr¨odinger equation (1.3), or (1.4). If b(x) ≡ b > 0 we get the following autonomous system    s 2p p−1 p+1  (−∆) u + u = |u| + b|u| |v| u x ∈ RN     s 2s 2p p−1 p+1 (1.6) (−∆) v + ω v = |v| + b|v| |u| v x ∈ RN ,    u(x) → 0, v(x) → 0 as |x| → ∞. It is well-known that system (1.6) with s = 1, i.e.    2p p−1 p+1  −∆u + u = |u| + b|u| |v| u x ∈ RN     2 2p p−1 p+1 −∆v + ω v = |v| + b|v| |u| v x ∈ RN ,    u(x) → 0, v(x) → 0 as |x| → ∞

(1.7)

appears in several branches of physics. For example, this problem arises as a model for propagation of polarized laser beams in birefringent Kerr medium in nonlinear optics (see, for example [3,14,17,33]). Physically,

143

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

the two functions u and v denote the components of the slowly varying envelope of the electrical field, t is the distance in the direction of propagation, x are the orthogonal variables and ∆ is the diffraction operator. The focusing nonlinear terms with (b > 0) in (1.7) describe the dependence of the refraction index of the material on the electric field intensity and the birefringence effects. The parameter b > 0 has to be interpreted as the birefringence intensity and describes the coupling between the two components of the electric field envelope. We notice that there have been some existence results for (1.6) with the classical Laplacian operator, i.e. (1.7). In [21] Maia, Montefusco and Pellacci considered problem (1.7) and proved the existence of ground states for all b ≥ 0, ω > 0 and all subcritical exponents 1 < p + 1 < N/(N − 2)+ . In the case of a cubic nonlinearity with p = 1, the sufficient conditions for the existence of a vector ground state have been found by Ambrosetti, Colorado [2] and de Figueiredo, Lopes [10], whereas the general case with 1 < p+1 < N/(N −2) was investigated in [21]. Recently much attention has been given to proving the existence of positive solutions (u, v) with both u and v nontrivial of the cubic system (1.7) (p = 1), and different families of vector and scalar solutions for (1.7) are obtained by using different method. For example, the existence of soliton solutions has been detected by numerical or perturbation arguments in [1,23,31]. In [2,20,22,30], variational methods have been pursued to prove the existence of ground and bound states with both nontrivial components. In the nonautonomous system where b(x) is a positive function of the variable x, we refer to [21] and the references therein. Our purpose in this paper is to study the similar situation that occurs for the fractional elliptic system (1.1). Using the idea of [20,21], we could prove that problem (1.1) has a least energy solution. However, since the problem here is nonlocal, the proof of the positivity of the solutions cannot be as easy as that in [21], which dealt with the local case with s = 1. In fact, the important property for the local operator ∇ that s ∥∇|u|∥L2 (RN ) = ∥∇u∥L2 (RN ) never holds for the nonlocal operator (−∆) 2 unless u has the definite sign. s s Generally, we only have the inequality ∥(−∆) 2 |u|∥L2 (RN ) ≤ ∥(−∆) 2 u∥L2 (RN ) for u ∈ H s (RN ). To overcome this difficulty, we are going to use the s-harmonic extension technique, which will be introduced in the next section. To the authors’ knowledge, there are very few results for the fractional systems (1.1). We consider the problem in the following Hilbert space E = H s (RN ) × Eω , where Eω = H s (RN ) is endowed with the norm  12  s 2 ∥v∥ω := (−∆) 2 v L2 (RN ) + ω 2s ∥v∥2L2 (RN ) , which is equivalent to the classical form ∥v∥H s (RN ) = endowed with the norm



∥(−∆) 2 v∥2L2 (RN ) + ∥v∥2L2 (RN ) s

 12

. Then E can be

∥(u, v)∥2E = ∥u∥2H s (RN ) + ∥v∥2ω for any (u, v) ∈ E. We define the energy functional I : E → R by   1 1 1 2 2p+2 2p+2 I(u, v) = ∥(u, v)∥E − (|u| + |v| )dx − b(x)|u|p+1 |v|p+1 dx. 2 2p + 2 RN p + 1 RN

(1.8)

We will be in particular interested in the existence of nonstandard least energy solutions of (1.1) or (1.6), that is, solutions with minimal energy on the set of solutions (u, v) to the system such that u ̸≡ 0 and v ̸≡ 0. Conversely, a solution (u, v) with a zero component u ≡ 0 or v ≡ 0 will be called a standard solution. The vector (0, 0) will be referred to as the trivial solution. We will search for nontrivial, or even nonstandard if possible, solutions, or equivalently, for nontrivial or nonstandard critical points of the functional I on the energy space E. Meanwhile, we will use the s-harmonic extension technique to deal with the nonlocal problem, which will be introduced in the next section. The main results are the following.

144

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

Theorem 1.1. Assume (1.2). Then for every b > 0 there exists a least energy solution (u, v) ̸= (0, 0) to problem (1.6), with u ≥ 0, v ≥ 0 and both u and v are radial. Theorem 1.2. Assume (1.2) and suppose that  1    f (ω) − 1 ω ≥ 1 2 b ≥ 1 1   − 1 ω ≤ 1,  f 2 ω

(1.9)

where     p+1  N 1 1 1 N ω 2s(p+1)−N p . 1− + 2s 1 − 1− f (ω) = 1 + 2s p+1 ω 2s p+1

(1.10)

Then there exists a nonstandard least energy solution W = (u, v) of problem (1.6), with u > 0 and v > 0. Remark 1.3. (i) Throughout this paper, we fix ω to consider b as a parameter depending on ω. (ii) Note that if (u, v) is a solution of system (1.6), then the pair ( 1ps v( ωx ), 1ps u( ωx )) is a solution of (1.6) with ω replaced by

1 ω.

ω

ω

As a consequence of this scaling-invariant property, the condition (1.9) is symmetric.

Theorem 1.2 gives a sufficient condition to find a nonstandard least energy solution to (1.6) with both nontrivial components. Furthermore, arguing as in [21] which corresponds to the local case where s = 1, we are able to prove the following necessary condition under the hypothesis p ≥ 1. Theorem 1.4. Assume (1.2) and p ≥ 1. If there exists a nonstandard least energy solution W = (u, v) to problem (1.6) with both nontrivial components, then it holds that b ≥ 2p − 1.

(1.11)

Remark 1.5. Since the proof of Theorem 1.4 can just follow that in [21] only to alter some calculation details, so we omit the details in this paper. Theorem 1.2 combined with Theorem 1.4 immediately implies the following. Corollary 1.6. Assume (1.2), ω = 1 and p ≥ 1. Then there exists a least energy solution of problem (1.6) with both nontrivial components if and only if b satisfies (1.11). Remark 1.7. (i) By a simple computation, we have that f (ω) ≥ f (1) = 2p+1 for ω ≥ 1 and f ( ω1 ) ≥ 2p+1 for ω ≤ 1. So Theorems 1.2 and 1.4 are not contradictory. (ii) Theorems 1.2 and 1.4 imply that for b large, the least energy solution has both nontrivial components, while for b small, the least energy solution must be of the form (u0 , 0)(ω ≥ 1) or (0, v0 )(ω ≤ 1) with u0 and v0 being the solutions of the single equations. The problem is completely solved only in the case when ω = 1, p ≥ 1 (Corollary 1.6), similarly as stated in [21]. Finally for the non-autonomous system (1.1) (with b(x) ̸≡ b), we have the following result. Theorem 1.8. Suppose N > 2s and (1.2). We assume that b(x) ∈ L∞ (RN ) satisfies the following conditions lim b(x) =: b∞ > 0

|x|→∞

b(x) ≥ b∞ ,

b(x) ̸≡ b∞ .

(1.12) (1.13)

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

145

Then the following holds. (a) The problem (1.1) possesses a nontrivial least energy solution. (b) If we further assume that b∞ satisfies (1.9), then there exists a nonstandard least energy solution with both nontrivial components. This paper is organized as follows. The next section contains some preliminary facts on the fractional Laplacian and gives an introduction of the s-harmonic extension technique. In Section 3, we study the Nehari manifold N and give some qualitative properties concerning the Mountain Pass level. The proofs of the main theorems can be found in Section 4. 2. Preliminaries and the local realization 2.1. Some facts on the fractional Laplacian In this section, we first introduce some definitions and notations. We consider the fractional Sobolev space    s N 2 N 2s 2 2 H (R ) = u ∈ L (R ) : (|ξ| |ˆ u| + |ˆ u| )dξ < ∞ , RN

with the norm defined by  ∥u∥ := ∥u∥H s (RN ) =

(|ξ|2s |ˆ u|2 + |ˆ u|2 )dξ

 12 .

RN

When there is no confusion, we will often write H s , Lq instead of H s (RN ), Lq (RN ), otherwise we will illustrate. Also to save notation, for any function φ ∈ Lq (RN ) with 1 ≤ q < ∞, we denote   q1   q1 q q = |φ| dx . ∥φ∥q = |φ| dx RN

In this paper, we also need to define the product Banach spaces Lq ≡ Lq (RN ) × Lq (RN ) equipped with the 1 norm ∥Φ∥q ≡ ∥Φ∥Lq = (∥φ∥qq + ∥ψ∥qq ) q for Φ = (φ, ψ) ∈ Lq . For the reader’s convenience, we review the embedding results for fractional Sobolev spaces. Proposition 2.1 ([27]). (i) Let s ∈ (0, 1) and 2s < N . Then there exists a constant C > 0 depending on N, s, such that  s  ∥u∥2∗s ≤ C (−∆) 2 u2 for every u ∈ H s (RN ), where 2∗s = N2N −2s is the fractional critical exponent. Hence, the embedding H s (RN ) ⊂ Lr is continuous for any r ∈ [2, 2∗s ], and compact for any r ∈ [2, 2∗s ). (ii) Let s ∈ (0, 1) and 2s = N . Then there exists a constant C > 0 depending on N, s, such that ∥u∥r ≤ C∥u∥H s for every u ∈ H s (RN ) and every r ∈ [2, +∞). (iii) Let s ∈ (0, 1) and 2s > N . Then there exists a constant C > 0 depending on N, s, such that ∥u∥C 0,α ≤ C∥u∥H s loc

s

N

for every u ∈ H (R ), and α =

2s−N 2 .

0,α Cloc

denotes the H¨ older spaces.

The standard Sobolev–Gagliardo–Nirenberg inequality can also be proved in the fractional spaces.

146

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

Proposition 2.2 ([27]). Let q ∈ (1, +∞). Then there exists a constant C depending on N, s and q, such that (q−1)N (q−1)N  s  q+1− 2s 2s  2  ∥u∥q+1 ∥u∥2 q+1 ≤ C (−∆) u 2 for every u ∈ H s (RN ). One major tool in variational methods is the following vanishing lemma, originally proved by P.L. Lions. Lemma 2.3. Suppose {uk } is a bounded sequence in H s (RN ) satisfying  lim sup |uk (x)|2 dx = 0, k→+∞ y∈RN

B(y,R)

for some R > 0. Then, uk → 0 in L (R ) for every 2 < q < 2∗s . q

N

Proof. The proof can be obtained by standard interpolation and Proposition 2.2, following the same method as used in [13,9].  2.2. The local realization Note that the operator (−∆)s is nonlocal, we can apply the s-harmonic extension technique introduced by [7] to transform (1.1) into a local problem. Definition 2.4. For a given regular function u in RN , we define that w = Es (u) is its s-harmonic extension +1 to the upper half-space RN , if w is a solution of the problem +  +1 −div(y 1−2s ∇w) = 0 in RN + N w = u on R × {y = 0}. In [7], it is proved that (−∆)s u(x) = −κs limy→0+ y 1−2s ∂w ∂y (x, y), where κs > 0 is a normalization +1 +1 constant. Moreover, we define the space X s (RN ) as the completion of C0∞ (RN ) under the norm + +    12 ∥w∥X s (RN +1 ) = +

κs

+1 RN +

y 1−2s |∇w|2 dxdy

.

The extension operator is well defined for smooth functions through a Poisson kernel whose explicit expression is given in [7]. It can also be defined in the space H˙ s (RN ), which is defined as the completion of s C0∞ (RN ) under the norm ∥u∥H˙ s (RN ) = ∥(−∆) 2 u∥2 . In fact, it can be proved that  s  ∥Es (u)∥X s (RN +1 ) = (−∆) 2 u2 . (2.1) +

s

+1 (RN ), +

On the other hand, for a function w ∈ X we will denote its trace on RN × {y = 0} as Tr(w) or simply as w(x, 0). This trace operator is also well defined and it satisfies that ∥Tr(w)∥H˙ s (RN ) ≤ ∥w∥X s (RN +1 ) . +

One may also refer to [4] for these properties of the extension. Now, we can transform the nonlocal problem (1.1) into the following local one  +1 −div(y 1−2s ∇w1 ) = 0 in RN +     +1 1−2s  ∇w2 ) = 0 in RN −div(y +   s 2p p−1 p+1 ∂ w + w = |w | + b(x)|w | |w | w1 on RN × {0}  1 1 1 1 2 ν        s ∂ν w2 + w2 = |w2 |2p + b(x)|w2 |p−1 |w1 |p+1 w2 on RN × {0},

(2.2)

(2.3)

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

147

1−2s ∂w2 s 1 + where ∂νs w1 (x, 0) := −κs limy→0+ y 1−2s ∂w ∂y (x, y) and ∂ν w2 (x, 0) := −κs limy→0 y ∂y (x, y). We then may work on the following Hilbert space    N +1 N +1 s s 2 2s 2 ˜ E := (w1 , w2 ) ∈ X (R+ ) × X (R+ ) : |w1 (x, 0)| + ω |w2 (x, 0)| dx < ∞ , (2.4) RN

which is endowed with the norm   12 ∥(w1 , w2 )∥E˜ := ∥w1 ∥2X s (RN +1 ) + ∥w2 ∥2X s (RN +1 ) + ∥w1 (x, 0)∥2L2 (RN ) + ω 2s ∥w2 (x, 0)∥2L2 (RN ) . +

+

˜ we consider now the functional For any (w1 , w2 ) ∈ E,   κs 1 1−2s 2 2 J(w1 , w2 ) = y |w1 (x, 0)|2 + ω 2s |w2 (x, 0)|2 dx (|∇w1 | + |∇w2 | )dxdy + +1 2 RN 2 RN +    1 |w1 (x, 0)|2p+2 + |w2 (x, 0)|2p+2 + 2b(x)|w1 (x, 0)|p+1 |w2 (x, 0)|p+1 dx. (2.5) − 2p + 2 RN We note that the assumptions (1.2) and (3.1) (below) imply that both the energy functional I, defined by (1.8), and J, by (2.5), are both of class C 1 . In fact, this properties for J can be obtained by the trace inequality (2.2). +1 ) and H˙ s = H˙ s (RN ) for short. In the following, we denote X s = X s (RN + Definition 2.5. We say that (w1 , w2 ) ∈ X s × X s is an energy solution to problem (2.3), if for every function couple (φ, ψ) ∈ X s × X s it holds that  κs y 1−2s (⟨∇w1 (x, y), ∇φ(x, y)⟩ + ⟨∇w2 (x, y), ∇ψ(x, y)⟩)dxdy +1 RN +

 +  =

w1 (x, 0)φ(x, 0) + ω 2s w2 (x, 0)φ(x, 0)dx

RN



   |w1 |2p + b(x)|w1 |p−1 |w2 |p+1 w1 φ(x, 0) + |w2 |2p + b(x)|w2 |p−1 |w1 |p+1 w2 ψ(x, 0)dx.

RN

Definition 2.6. We say that (u, v) ∈ H s (RN ) × H s (RN ) is an energy solution to problem (1.1), if its components u and v are the trace on RN × {y = 0} of the functions w1 and w2 respectively, and the couple (w1 , w2 ) ∈ X s × X s is an energy solution to problem (2.3). ˜ is an energy solution to problem (2.3), or a critical point of J, By the arguments as above, if (w1 , w2 ) ∈ E then its trace couple (u, v) ∈ E with u = w1 (x, 0) and v = w2 (x, 0) is a critical point of I and is an energy or weak solution of problem (1.1). The converse is also right. By the equivalence of these two formulations, we will use both formulations in the sequel to their best advantage. By s-harmonic extension and similar arguments as in [27,8], we introduce a variational identity (known as the Pohozaev identity) for any solution u ∈ H s of the problem (−∆)s u + u = |u|2p u

x ∈ RN

(2.6)

and for any solution v ∈ H s of the problem (−∆)s v + ω 2s v = |v|2p v respectively.

x ∈ RN

(2.7)

148

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

Proposition 2.7. (i) Assume that u ∈ H s is a weak solution of (2.6) and v ∈ H s is a weak solution of (2.7). Then there hold that  N − 2s  (−∆) 2s u2 + N ∥u∥22 = N ∥u∥2p+2 , (2.8) 2p+2 2 2 2 2p + 2 and  N − 2s  (−∆) 2s v 2 + N ω 2s ∥v∥22 = N ∥v∥2p+2 . (2.9) 2p+2 2 2 2 2p + 2 (ii) Any weak solution (u, v) to system (1.6) satisfies that     N − 2s  (−∆) 2s u2 + (−∆) 2s v 2 + N [∥u∥22 + ω 2s ∥v∥22 ] 2 2 2 2 N 2p+2 p+1 [∥u∥2p+2 = 2p+2 + ∥v∥2p+2 + 2b∥uv∥p+1 ]. 2p + 2

(2.10)

Remark 2.8. When s = 1, the standard strategy to prove the Pohozaev identity is to multiply the equation by ⟨x, ∇u⟩ and integrate by parts. For the fractional Laplacian, the author in [26] shows the identity using the same technique, before doing which, they only have to prove the regularity and decay estimate. We omit the proof here and leave one to refer to [26] for more details. One can refer to [8] for another proof by means of the s-harmonic extension technique. 3. Nehari manifold and qualitative properties of the Mountain Pass level In this section, we may consider a more general case, where the measurable function b : RN → R satisfies the following hypotheses  ∞ N m N b(x) = b1 (x) + b2 (x), b1 ∈ L (R ), b2 ∈ L (R ), (3.1) N m = , if N > 2s; m ≥ 1, if N < 2s; m > 1, if N = 2s. N − (p + 1)(N − 2s) We also assume that b(x) ≥ 0,

b(x) ̸≡ 0.

(3.2)

Remark 3.1. The b(x) ∈ L∞ with the conditions (1.12) and (1.13) obviously meets (3.1) and (3.2). Further special case of (3.1) is when b(x) ≡ b > 0. As discussed before, the hypotheses (1.2), (3.1) and the trace inequalities (2.2) imply that the energy functional J, defined by (2.5), is of class C 1 , and so we can define the Nehari manifold ˜ \ {(0, 0)} : ⟨J ′ (W ), W ⟩ = 0}, N = {W = (w1 , w2 ) ∈ E (3.3) ˜ is defined by (2.4). We show that the Nehari manifold possesses the following where the Hilbert space E properties. ˜ Lemma 3.2. (i) For every W = (w1 , w2 ) ∈ E\{(0, 0)}, there exists a unique t¯(W ) > 0 such that t¯(W )W ∈ N and the maximum of J(tW ) for t ≥ 0 is achieved at t = t¯(W ). The function ˜ \ {(0, 0)} → (0, ∞) : W → t¯(W ) E ˜ with N . is continuous and the map W → t¯(W )W defines a homeomorphism of the unit sphere of E (ii) J is bounded from below on N by a positive constant.

149

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

Proof. (i) Define F (w1 , w2 ) =

1 1 (|w1 (x, 0)|2p+2 + |w2 (x, 0)|2p+2 ) + b(x)|w1 (x, 0)|p+1 |w2 (x, 0)|p+1 . 2p + 2 p+1

(3.4)

In view of (3.1) and (1.2), we can use H¨ older’s inequalities, the fractional Sobolev embedding (see 1 Proposition 2.1) and the trace inequalities (2.2) to obtain that for N > 2s, there exist constants Cω,p >0 and Cω,p > 0 such that  ∥b1 ∥∞ p+1 1 ∥w1 (·, 0)∥p+1 |F (w1 , w2 )|dx ≤ Cω,p ∥(w1 , w2 )∥2p+2 + 2p+2 ∥w2 (·, 0)∥2p+2 ˜ E p+1 RN ∥b2 ∥m p+1 + ∥w1 (·, 0)∥p+1 2N ∥w2 (·, 0)∥ 2N N −2s N −2s p+1 1 ≤ Cω,p ∥(w1 , w2 )∥2p+2 ˜ E p+1  p+1 ∥b1 ∥∞  + C ∥w1 (·, 0)∥2 + ∥w1 (·, 0)∥2∗s ∥w2 (·, 0)∥2 + ∥w2 (·, 0)∥2∗s p+1 ∥b2 ∥m p+1 ∥w1 (·, 0)∥p+1 + 2N ∥w2 (·, 0)∥ 2N N −2s N −2s p+1 1 ≤ Cω,p ∥(w1 , w2 )∥2p+2 ˜ E

∥b1 ∥∞ p+1 p+1 C (∥w1 (·, 0)∥2 + ∥w1 ∥X s ) (∥w2 (·, 0)∥2 + ∥w2 ∥X s ) p+1 ∥b2 ∥m p+1 + ∥w1 (·, 0)∥p+1 2N ∥w2 (·, 0)∥ 2N N −2s N −2s p+1 ∥b1 ∥∞ ∥b2 ∥m p+1 1 ≤ Cω,p ∥(w1 , w2 )∥2p+2 + C∥(w1 , w2 )∥2p+2 + ∥w1 (·, 0)∥p+1 2N ∥w2 (·, 0)∥ 2N ˜ ˜ E E N −2s N −2s p+1 p+1 +

≤ Cω,p ∥(w1 , w2 )∥2p+2 , ˜ E where the constant C may be different from line to line. The cases when N ≤ s can be obtained similarly in view of the conditions (3.1) and Principle 2.1. Thus, F satisfies that  1 F (w1 , w2 )dx = 0, (3.5) lim 2 ∥(w1 ,w2 )∥E N ˜ →0 ∥(w1 , w2 )∥ ˜ E R which implies immediately that (0, 0) is a strict local minimum of J. Moreover, since F (w1 , w2 ) ≥

1 (|w1 (x, 0)|2p+2 + |w2 (x, 0)|2p+2 ), 2p + 2

then for every t > 0, we deduce the following inequality   ∥(w1 , w2 )∥2E˜ 1 2p+2 2p+2 2p+2 J(tw1 , tw2 ) ≤ t − (∥w1 (·, 0)∥2p+2 + ∥w2 (·, 0)∥2p+2 ) , 2t2p 2p + 2 ˜ J(T W ) < 0 for T > 0 sufficiently large. which means that for every W = (w1 , w2 ) ∈ E, ˜ \ {(0, 0)} and t > 0, we set Now, for any W = (w1 , w2 ) ∈ E g(t) := J(tW ) = J(tw1 , tw2 ).

(3.6)

The above arguments imply that there exists t¯ = t¯(W ) > 0 such that g(t¯) = maxt>0 g(t). More precisely, for t > 0, we compute that d J(tW ) = J ′ (tW )W dt    = t ∥W ∥2E˜ − t2p ∥W (·, 0)∥2p+2 + 2 2p+2

RN

b(x)|w1 (x, 0)|p+1 |w2 (x, 0)|p+1 dx

 ,

150

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

and thus, every positive critical point t of g satisfies    p+1 p+1 ∥W ∥2E˜ − t2p ∥W (·, 0)∥2p+2 + 2 b(x)|w (x, 0)| |w (x, 0)| dx = 0. 1 2 2p+2 RN

Since p > 0, the point t¯ = t¯(W ) is the unique value of t > 0 at which t¯(W )W ∈ N , and t¯(W ) =

∥W (·, 0)∥2p+2 2p+2 + 2

1  2p

∥W ∥2E˜

  RN

b(x)|w1 (x, 0)|p+1 |w2 (x, 0)|p+1 dx

.

(3.7)

So, we obtain that 1 max g(t) = g(t¯) = t>0 2



1 1− p+1

2(p+1)

 ∥W (·, 0)∥2p+2 2p+2 + 2

∥W ∥E˜  RN

b(x)|w1 (x, 0)|p+1 |w2 (x, 0)|p+1 dx

 p1 . (3.8)

˜ Furthermore, the assumptions (1.2), (3.1) and Thus, N is radially homeomorphic to the unit sphere in E. ¯ (2.2) imply that the application W → t(W ) is continuous. We have proved conclusion (i). (ii) Now let W = (w1 , w2 ) ∈ N . Again by the Sobolev embedding (Proposition 2.1), H¨older’s inequalities and the trace inequalities (2.2), there exists a constant C0 > 0 such that  2p+2 2 0 = ∥W ∥E˜ − ∥W (·, 0)∥2p+2 − b(x)|w1 (x, 0)|p+1 |w2 (x, 0)|p+1 ≥ ∥W ∥2E˜ − C0 ∥W ∥2p+2 , ˜ E RN

which implies that there exists a constant C1 > 0 such that for all W = (w1 , w2 ) ∈ N ∥(w1 , w2 )∥2E˜ ≥ C1 > 0. As a result, there exists a constant C2 > 0 such that for all W = (w1 , w2 ) ∈ N ,  1 1 1 2p+2 2 J(w1 , w2 ) = ∥(w1 , w2 )∥E˜ − ∥W (·, 0)∥2p+2 − b(x)|w1 (x, 0)|p+1 |w2 (x, 0)|p+1 dx 2 2p + 2 p + 1 RN 1 1  = − ∥(w1 , w2 )∥2E˜ > C2 > 0, 2 2p + 2 and (ii) follows.



We define cN := inf J(W ), W ∈N

ct :=

inf

max J(tW ),

˜ t≥0 W ∈E\{0}

(3.9) (3.10)

and c := inf max J(γ(t)),

(3.11)

˜ γ is continuous and γ(0) = 0, J(γ(1)) < 0}. Γ = {γ : [0, 1] → E,

(3.12)

γ∈Γ t∈[0,1]

where

Using the standard method associated to a single equation as proved in [32], we have the following qualitative result. Lemma 3.3. One has cN = ct = c. Proof. First from Lemma 3.2, it follows that ct = inf J(t¯(W )W ) = inf J(W ) = cN . ˜ W ∈E

W ∈N

151

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

˜ \ {(0, 0)} and for t sufficiently large, which implies that c ≤ ct . On the one hand, J(tW ) < 0 for every w ∈ E On the other hand, cN ≤ c because every γ ∈ Γ intersects N . Then the conclusion follows.  We need the following lemma to find a non-negative solution. Lemma 3.4. If W ∈ N and J(W ) = c with c defined in (3.11), then W is a critical point of J. Proof. One can refer to Theorem 4.3 of [32] for the details of the proof.



4. Proof of the main results In this section, we show Theorems 1.1 and 1.8 by means of considering the equivalent local problem (2.3). When proving Theorem 1.2, we will directly come to the original problem (1.6). Proof of Theorem 1.1. We will use the notations introduced in Section 3, where b(x) is replaced by the constant b. We consider the Mountain-pass level c defined by (3.11). Lemma 3.2(ii) and Lemma 3.3 imply that c > 0. In view of Lemmas 3.2 and 3.3 and by the standard method, we can use the Ekeland variational principle (see [11,32]) to find a sequence Wn = (w1,n , w2,n ) such that as n → +∞, ′

J (Wn ) → 0,

J(Wn ) → c, ˜′. strongly in E

(4.1) (4.2)

We compute (2p + 2)c = lim [(2p + 2)J(Wn ) − ⟨J ′ (Wn ), Wn ⟩] = lim p∥Wn ∥2E˜ , n→∞

n → +∞

n→∞

˜ Thus, there exists some W = (w1 , w2 ) ∈ E ˜ such that as n → +∞, up to obtain that Wn is bounded in E. to a subsequence, (w1,n , w2,n ) ⇀ (w1 , w2 )

˜ weakly in E,

(w1,n (·, 0), w2,n (·, 0)) → (w1 (·, 0), w2 (·, 0)) (w1,n (·, 0), w2,n (·, 0)) → (w1 (·, 0), w2 (·, 0))

(4.3) strongly in

Lqloc (RN ),

∀q ∈

[1, 2∗s ),

N

almost everywhere in R .

Firstly, we show that there exist xk ∈ RN , β > 0 and R > 0 such that up to a further subsequence,  (|w1,k (x, 0)|2 + |w2,k (x, 0)|2 )dx ≥ β > 0, ∀k ∈ N,

(4.4) (4.5)

(4.6)

B(xk ,R)

where B(xk , R) = {x ∈ RN : |x − xk | < R} ⊂ RN . In fact, if not, then on the one hand,  sup (|w1,n (x, 0)|2 + |w2,n (x, 0)|2 )dx → 0, n → +∞, z∈RN

B(z,R)

which, combined with Lemma 2.3 and the condition (1.2), implies that (w1,n (x, 0), w2,n (x, 0)) → (0, 0), On the other hand, we get from (4.1) and (4.2) that

strongly in L2p+2 (RN ).

(4.7)

152

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

1 0 < c = J(Wn ) − ⟨J ′ (Wn ), Wn ⟩ + o(1) 2   1 1 = 1− (|w1,n (x, 0)|2p+2 + |w2,n (x, 0)|2p+2 )dx 2 p+1 N  R 1 + b 1− (|w1,n (x, 0)|p+1 |w2,n (x, 0)|p+1 )dx + o(1) p+1 N   R 1 1 2p+2 ≤ 1− (1 + b)(∥w1,n (·, 0)∥2p+2 2p+2 + ∥w2,n (·, 0)∥2p+2 ) + o(1), 2 p+1 which contradicts (4.7). Thus, (4.6) holds. Next, we set ¯ k (x) = (w W ¯1,k (x, y), w ¯2,k (x, y)) = (w1,k (x + xk , y), w2,k (x + xk , y)).

(4.8)

The translation invariance respect to the first N variable x of the functional J (with b(x) ≡ b) implies that ¯ k is also a Palais–Smale sequence at level c. Then arguing as we did for Wk , up to a subsequence, there W ¯ = (w ¯ k satisfying (4.3)–(4.5). Thus W ¯ is a critical point of J. Moreover, exists a weak limit W ¯1 , w ¯2 ) of W (4.6) implies that  (|w ¯1,k (x, 0)|2 + |w ¯2,k (x, 0)|2 )dx ≥ β > 0, ∀k ∈ N. (4.9) B(0,R)

¯ k (·, 0) → W ¯ (·, 0) strongly in L2 (RN ) by (4.4), we obtain that Since W loc  (|w ¯1 (x, 0)|2 + |w ¯2 (x, 0)|2 )dx ≥ β > 0, B(0,R)

˜∋W ¯ = (w which implies that E ¯1 , w ¯2 ) ̸= (0, 0) is a nontrivial weak solution of system (2.3) with b(x) ≡ b. ¯ ) = c. In fact, on the one hand, since (w Thirdly, we show that J(W ¯1 , w ¯2 ) ̸= (0, 0) is a critical point of ¯ ) ≥ c. On the other hand, from Fatou’s J, then we deduce that (w ¯1 , w ¯2 ) ∈ N , and so, by Lemma 3.3, J(W lemma we get that   ¯ k ) − 1 ⟨J ′ (W ¯ k ), W ¯ k⟩ c = lim J(W k→∞ 2    1 1 = lim 1− (|w ¯1,k (x, 0)|2p+2 + |w ¯2,k (x, 0)|2p+2 )dx k→∞ 2 p+1 N R    1 (|w ¯1,k (x, 0)|p+1 |w ¯2,k (x, 0)|p+1 )dx +b 1 − p+1 N   R 1 1 1− (|w ¯1 (x, 0)|2p+2 + |w ¯2 (x, 0)|2p+2 )dx ≥ 2 p+1 N R  1 (|w ¯1 (x, 0)|p+1 |w ¯2 (x, 0)|p+1 )dx + b 1− p+1 N R 1 ′ ¯ ¯ ¯ ¯ ). = J(W ) − ⟨J (W ), W ⟩ = J(W 2 ¯ ) = c. By Lemma 3.3, c = cN and so W ¯ = (w In conclusion, it must hold that J(W ¯1 , w ¯2 ) is a least energy solution to system (2.3) with b(x) ≡ b. Notice that J(|w ¯1 |, |w ¯2 |) = J(w ¯1 , w ¯2 ) = c and ⟨J ′ (|w ¯1 |, |w ¯2 |), (|w ¯1 |, |w ¯2 |)⟩ = ⟨J ′ (w ¯1 , w ¯2 ), (w ¯1 , w ¯2 )⟩ = 0. Then, Lemma 3.4 implies that (|w ¯1 |, |w ¯2 |) is a critical point of J at the same level of (w ¯1 , w ¯2 ). Hence, we find a least energy solution to system (2.3) with both nonnegative components, which is denoted by W = (w1 , w2 ) for short.

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

153

If w1 and w2 are both nontrivial functions, the strong maximum principle, which we refer to [16,6], implies that w1 > 0 and w2 > 0 are both positive functions. Invoking the regularity theory [16,15,6], we +1 infer that w1 , w2 ∈ C 0,γ (RN ), for some γ ∈ (0, 1). Thus, if we denote the trace of W = (w1 , w2 ) as + (u, v) := (w1 (·, 0), w2 (·, 0)), then there must hold that u > 0 and v > 0, and (u, v) is a nonstandard weak solution to problem (1.6) whose components are both positive. Similar arguments as used for the classical local case with s = 1 [5,16] can be applied to (1.6) to deduce that u and v are both radial functions up to a translation, and decay to zero as expressed by (1.5). Next, we should show that (u, v) is a least energy solution in the following sense. If we take any weak or energy solution (¯ u, v¯) to problem (1.6), then the equivalence of the energy solution to system (2.3) with the weak solution to (1.1) or (1.6) (Definitions 2.5 and 2.6) implies ¯ = (w that there exists a energy solution W ¯1 , w ¯2 ) = (Es (¯ u), Es (¯ v )) to problem (2.3) (where b(x) ≡ b), which is the s-harmonic extension of (¯ u, v¯) satisfying that u ¯=w ¯1 (·, 0), v¯ = w ¯2 (·, 0) and I(¯ u, v¯) = J(w ¯1 , w ¯2 ) by ¯ ∈ N , then also by (2.1), it follows that I(¯ (2.1). Since W u, v¯) = J(w ¯1 , w ¯2 ) ≥ c = J(w1 , w2 ) = I(u, v), which means that the solution (u, v) must possess the minimal energy on the set of solutions of system (1.6). Otherwise, if one of the components w1 and w2 is identically zero, then similar argument as above gives that the trace of the other is positive, radial and decay to zero as |x| → ∞. Then in this case we have found a standard least energy solution with just one positive component. Consequently, we have proved the theorem in both cases.  Next, we will search for a nonstandard solution to system (1.6). Proof of Theorem 1.2. Observe that there exist two standard solutions of system (1.6), one of which is of the form (u, 0), where u a solution of (2.6), while the other is (0, v) with v solving the problem (2.7). From the facts on solutions of the singular fractional Schr¨odinger equation (one may refer to [16] for example), the problem (2.6) has a unique (up to a translation) positive radial solution denoted by u0 . Then by scaling, α the unique positive solution of (2.7) turns out to be v0 (x) := ω p u0 (ωx). In view of the above observations, it remains to show that the least energy solution (u, v) we have found in Theorem 1.1 satisfies the following c = I(u, v) < min{I(u0 , 0), I(0, v0 )}.

(4.10)

Since u0 and v0 are solutions of (2.6) and (2.7) respectively, using the notations defined in Section 2, we have that ∥v0 ∥2ω = ∥v0 ∥2p+2 2p+2 .

∥u0 ∥2 = ∥u0 ∥2p+2 2p+2 ,

(4.11)

Thus, (4.11) combined with the Pohozaev identities (2.8) and (2.9) gives that     N N N N 2s 2 ∥u0 ∥22 = s − , ω ∥v ∥ = s − + ∥u0 ∥2p+2 + ∥v0 ∥2p+2 0 2 2p+2 2p+2 . 2 2p + 2 2 2p + 2

(4.12)

α

From v0 (x) := ω p u0 (ωx), (4.11) and (4.12), we get the following relations   1 1 C := I(u0 , 0) = 1− ∥u0 ∥2p+2 2p+2 , 2 p+1

(4.13)

and 1 I(0, v0 ) = 2

 1−

1 p+1



∥v0 ∥2p+2 2p+2 = ω

2(p+1)s −N p

C.

Then it remains to find a pair (φ, ψ) with both nontrivial components φ ̸≡ 0 and ψ ̸≡ 0 such that   2(p+1)s c ≤ max I(tφ, tψ) ≤ min C, ω p −N C . t≥0

(4.14)

154

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

Since N associated with the functional I possesses the same properties (stated in Lemma 3.2) as that with J, then for any (u, v) ∈ E we have the equivalent form of the expression (3.8), that is  p1   1 1 (∥u∥2 + ∥v∥2ω )p+1 G(u, v) := max g(t) = 1− . p+1 2p+2 t>0 2 p+1 ∥u∥2p+2 2p+2 + ∥v∥2p+2 + 2b∥uv∥p+1 Thus, we are reduced to find a pair (φ, ψ) with both nontrivial components such that   2(p+1)s c ≤ G(φ, ψ) ≤ min C, ω p −N C . Firstly, we consider the case where ω ≥ 1. The hypothesis (1.2) implies that this case, min{C, ω

2(p+1)s −N p

2(p+1)s p

(4.15) − N > 0. Thus, in

C} = C and (4.15) turns out to be c ≤ G(φ, ψ) ≤ C = I(u0 , 0).

(4.16)

To show (4.16), we choose (φ, ψ) = (v0 , v0 ). From (4.11), (4.14) and (2.5), we get that   1 1 ∥v0 ∥2p+2 (1 + c(ω))p+1 p 1 ω 1− , G(v0 , v0 ) = 2 p+1 2(1 + b)∥v0 ∥2ω     N N 1 1 where c(ω) = 2s (1 − p+1 ) . By (4.11) and (4.14), we have that 1 − p+1 + ω12s 1 − 2s G(v0 , v0 ) = ω

2(p+1)s −N p

 C

(1 + c(ω))p+1 2(1 + b)

 p1 .

Thus, (4.16) is equivalent to ω

2(p+1)s −N p



(1 + c(ω))p+1 2(1 + b)

 p1 ≤ 1,

which is exactly the condition (1.9) in the case ω ≥ 1. Next, when ω ≤ 1, then min{C, ω and proceed as above to find

2(p+1)s −N p

G(u0 , u0 ) =

1 2

 1−

C} = ω 1 p+1



2(p+1)s −N p

∥u0 ∥2



C. In this case, we choose (φ, ψ) = (u0 , u0 )

(1 + c(ω))p+1 2(1 + b)

 p1 .

By (4.11) and (4.13), we then get that  G(u0 , u0 ) = C

(1 + c(ω))p+1 2(1 + b)

 p1 .

Thus in the case of ω ≤ 1, the condition (1.9) implies that   2(p+1)s 2(p+1)s c ≤ G(u0 , u0 ) ≤ min C, ω p −N C = ω p −N C. In conclusion, the theorem has been proved in both cases.



Finally, we show Theorem 1.8 by considering the problem (2.3), which is the local realization of and equivalent to problem (1.1). Proof of Theorem 1.8. Part (a) We define the functional at infinity J∞ (w1 , w2 ) :=

1 1 ∥(w1 , w2 )∥2E˜ − ∥(w1 (·, 0), w2 (·, 0))∥2p+2 2p+2 2 2p + 2  1 − b∞ |w1 (x, 0)|p+1 |w2 (x, 0)|p+1 dx. p+1 N R

(4.17)

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

155

In the following, we use the notations Γ and c defined by (3.12) and (3.11). Step 1. We show that there exists a nontrivial solution W ̸= (0, 0) to system (2.3) with energy J(W ) ≤ c. Arguing as in the beginning of the proof of Theorem 1.1, we use Ekeland variational principle to find ˜ such that as n → +∞, J(Wn ) → c, J ′ (Wn ) → 0 strongly in E ˜ ′ . By a sequence Wn = (w1,n , w2,n ) ∈ E computing (2p + 2)c = lim ((2p + 2)J(Wn ) − ⟨J ′ (Wn ), Wn ⟩) = lim p∥Wn ∥2E˜ , n→∞

n→∞

˜ Then there exists some W = (w1 , w2 ) ∈ E ˜ such that as n → +∞, up we get that Wn is bounded in E. to a subsequence, (4.3)–(4.5) are satisfied. Thus, (w1 , w2 ) is a critical point of J, which, however, could be trivial. Case 1. We suppose that (w1 , w2 ) = (0, 0) is trivial. In this case, we claim first that the sequence Wn is also a Palais–Smale sequence for the limit functional J∞ defined by (4.17) at the same level c. In fact, since Wn is bounded and satisfies (w1,n (·, 0), w2,n (·, 0)) → (0, 0) strongly in Lqloc (RN ) for any q ∈ [1, 2∗s ), then as n → ∞,  J∞ (Wn ) − J(Wn ) = (b∞ − b(x))|w1,n (x, 0)|p+1 |w2,n (x, 0)|p+1 dx → 0, (4.18) RN

where we use H¨ older inequalities, the trace inequality (2.2) and the condition (1.12). Similarly for any ˜ it holds that (φ, ψ) ∈ E, sup ∥(φ,ψ)∥E ˜ ≤1

=

′ |⟨J∞ (Wn ) − J ′ (Wn ), (φ, ψ)⟩|

sup ∥(φ,ψ)∥E ˜ ≤1

   

 (b∞ − b(x)) |w1,n (x, 0)|p−1 |w2,n (x, 0)|p+1 w1,n (x, 0)φ(x, 0)

RN

 + |w2,n (x, 0)|p−1 |w1,n (x, 0)|p+1 w2,n (x, 0)ψ(x, 0)  dx → 0, which combined with (4.18) implies the claim. Now since J∞ is translation invariant, then we can follow the ¯ k for J∞ by translating Wk . Moreover, arguments in the proof of Theorem 1.1 to construct a new sequence W ¯ ¯ Wk weakly converges to a nontrivial pair of functions W = (w ¯1 , w ¯2 ) ̸= (0, 0), which is a critical point of J∞ . Now Case 1 would be further divided into two more cases. ¯ = (w Case 1.1. Suppose W ¯1 , w ¯2 ) has one trivial component, for example w ¯2 ≡ 0, then immediately, ′ J(w ¯1 , 0) = J∞ (w ¯1 , 0) and J ′ (w ¯1 , 0) = J∞ (w ¯1 , 0). Hence, we have that (w ¯1 , 0) is also a nontrivial critical point of J, which is exactly the nontrivial solution of (2.3). Moreover, Fatou’s lemma implies that ′ ¯ ) =J∞ (W ¯ ) = J ∞ (W ¯ ) − 1 ⟨J∞ ¯ ), W ¯⟩ J(W (W 2   1 1 = 1− |w ¯1 (x, 0)|2p+2 dx 2 p+1 RN   1 1 ≤ lim 1− |w ¯1,k (x, 0)|2p+2 dx k→∞ 2 p+1 RN ′ ¯ k ) − 1 ⟨J∞ ¯ k ), W ¯ k ⟩) = c, = lim (J∞ (W (W k→∞ 2

which gives the desirable result in this case.

156

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

¯ = (w ˜ of J∞ with both nontrivial Case 1.2. Otherwise, we have found a critical point W ¯1 , w ¯2 ) ∈ E components w ¯1 ̸≡ 0 and w ¯2 ̸≡ 0. We define a function  x y ¯ , t > 0, , W t t γ(t) := 0, t = 0. By scaling, we calculate that ¯2 ∥2X s ), ∥γ(t)∥2E˜ = tN (∥w ¯1 (·, 0)∥22 + ω 2s ∥w ¯2 (·, 0)∥22 ) + tN −2s (∥w ¯1 ∥2X s + ∥w ˜ Furthermore, it holds that which implies that the pass γ ∈ C([0, ∞), E). tN −2s J∞ (γ(t)) = ¯2 ∥2X s ) (∥w ¯1 ∥2X s + ∥w 2  1 1 1 1 − tN − ∥w ¯1 (·, 0)∥22 − ω 2s ∥w ¯2 (·, 0)∥22 + ∥w ¯1 (·, 0)∥2p+2 ∥w ¯2 (·, 0)∥2p+2 2p+2 + 2p+2 2 2 2p + 2 2p + 2  b∞ + ∥w ¯1 w ¯2 (·, 0)∥p+1 p+1 . p+1 Note that (u, v) := (w1 (·, 0), w2 (·, 0)) is a weak solution to the fractional system (1.1). Then, the Pohozaev identity (2.10) and the extension identity (2.1) imply that 1 1 ¯1 (·, 0)∥22 − ω 2s ∥w ¯2 (·, 0)∥22 − ∥w 2 2 1 1 b∞ + ∥w ¯1 (·, 0)∥2p+2 ∥w ¯2 (·, 0)∥2p+2 ∥w ¯1 w ¯2 (·, 0)∥p+1 2p+2 + 2p+2 + p+1 > 0. 2p + 2 2p + 2 p+1 Thus, since N > 2s (i.e. N ≥ 2s + 1), it follows from (2.10) d (N − 2s)tN −2s−1 J∞ (γ(t)) = (∥w ¯1 ∥2X s + ∥w ¯2 ∥2X s ) dt 2  N N N − tN −1 − ∥w ¯1 (·, 0)∥22 − ω 2s ∥w ¯2 (·, 0)∥22 + ∥w ¯1 (·, 0)∥2p+2 2p+2 2 2 2p + 2  N N ∥w ¯2 (·, 0)∥2p+2 ∥w ¯1 w ¯2 (·, 0)∥p+1 + 2p+2 + b∞ p+1 2p + 2 p+1  > 0, t ∈ (0, 1), < 0, t > 1. ˜ So if we take a constant L > 1 sufficiently large, the path γL (t) := γ(Lt) will satisfy that γL ∈ C([0, 1], E), ¯ (x) ∈ γL ([0, 1]), J∞ (γL (1)) < 0 and maxt∈[0,1] J∞ (γL (t)) = J∞ (W ¯ ). Since the components w W ¯1 ̸≡ 0 and w ¯2 ̸≡ 0, then from the assumption (1.12) and (1.13), we get the following strict inequality J(γL (t)) < J∞ (γL (t)) for any t ∈ [0, 1]. Using Fatou’s lemma, we compute that ′ ¯ ) = J ∞ (W ¯ ) − 1 ⟨J∞ ¯ ), W ¯⟩ J∞ (W (W 2 ¯ k ) − 1 ⟨J ′ (W ¯ k ), W ¯ k ⟩) = c. ≤ lim (J∞ (W k→∞ 2 ∞

As a result, ¯ ) ≤ c, c ≤ max J(γL (t)) < max J∞ (γL (t)) = J∞ (W t∈[0,1]

t∈[0,1]

which is a contradiction. As a consequence, Case 1.2 can never happen.

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

157

Case 2. It remains to deal with the case when the critical point W = (w1 , w2 ) of J is nontrivial, that is, W = (w1 , w2 ) ̸= (0, 0). In fact, in this case, we get again from the Fatou lemma that 1 J(W ) = J(W ) − ⟨J ′ (W ), W ⟩ 2   1 ′ ≤ lim J(Wn ) − ⟨J (Wn ), Wn ⟩ = lim J(Wn ) = c, n→∞ n→∞ 2 which gives the desirable result, showing that there exists a nontrivial solution W = (w1 , w2 ) ̸= (0, 0) of (1.1) with J(W ) ≤ c. Step 2. We show that c is attained. In fact, since any nontrivial critical point W of J satisfies that W ∈ N , then by Lemma 3.3, it follows immediately that J(W ) ≥ cN = c. Thus, the nontrivial critical point W = (w1 , w2 ) ̸= (0, 0) of J with J(W ) ≤ c obtained in step 1 must satisfy J(W ) = c, which is then the least energy solution of system (2.3). Furthermore, since J(|w1 |, |w2 |) = J(w1 , w2 ) = c and ⟨J ′ (|w1 |, |w2 |), (|w1 |, |w2 |)⟩ = ⟨J ′ (w1 , w2 ), (w1 , w2 )⟩ = 0, then, Lemma 3.4 implies that (|w1 |, |w2 |) is a critical point of J at the same level of (w1 , w2 ). Hence, we find a nontrivial least energy solution with both nonnegative components, which is denoted by W = (w1 , w2 ) by itself. Following the argument in the proof of Theorem 1.1, if we denote the trace of W = (w1 , w2 ) as (u, v) := (w1 (·, 0), w2 (·, 0)), then by (2.1), I(u, v) = J(w1 , w2 ) and (0, 0) ̸≡ (u, v) ∈ E is a nontrivial weak solution to problem (1.1), which possess the minimal energy on the set of solutions of system (1.1). Moreover, if the coefficient function b(x) is sufficiently smooth, then w1 , w2 must be C 0,γ with some γ ∈ (0, 1) and thus each component of (u, v) = (w1 (·, 0), w2 (·, 0)) would also be nonnegative. In conclusion, we have proved Part (a) of this theorem. Part (b) We further suppose that b∞ satisfies the condition (1.9) in Theorem 1.2. From the assumption ˜ and then (1.13), we get that J(W ) < J∞ (W ) for any W ∈ E, c ≤ c∞ ,

(4.19)

where c, c∞ are defined by (3.11) for J and J∞ respectively. Using Theorem 1.2, we can find a nonstandard least energy solution (u∞ , v∞ ) of system (1.6) (where b is replaced by b∞ ) with both nontrivial components. Moreover, if we denote W∞ = (Es (u∞ ), Es (v∞ )), then by (2.1) and the proof of Theorem 1.1, it holds that c∞ = J∞ (W∞ ) = I∞ (u∞ , v∞ ) < min{I∞ (u0 , 0), I∞ (0, v0 )} = min{I(u0 , 0), I(0, v0 )} = min{J(w1,0 , 0), J(0, w2,0 )}, where w1,0 = Es (u0 ) and w2,0 = Es (v0 ). Thus by (4.19), c ≤ c∞ < min{J(w1,0 , 0), J(0, w2,0 )}. From Part (a), we get a nontrivial critical point (w1 , w2 ) ̸= (0, 0) of J at level c. Consequently, we obtain that J(w1 , w2 ) = c < min{J(w1,0 , 0), J(0, w2,0 )}.

(4.20)

By similar argument as in the proof of Theorem 1.2, the strict inequality in (4.20) implies that the components w1 ̸≡ 0 and w2 ̸≡ 0 are both nontrivial. Then as the proof of Part (a), we obtain that (u, v) := (w1 (·, 0), w2 (·, 0)) ̸= (0, 0) is the nonstandard least energy solution with both nontrivial components, concluding the proof of Part (b) of Theorem 1.8. 

158

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

Remark 4.1. Theorem 1.8 still holds when b(x) satisfies (3.1) with b2 ≥ 0 and b1 satisfying (1.12) and b(x) ≥ b∞ , b(x) ̸≡ b∞ . Indeed, the integral with b2 (x) defines a compact operator and b2 ≥ 0 is a sufficient condition to compare J with J∞ . Acknowledgments The first author Q. Guo was supported by the Project of NSFC for Young Scholars of China (Grant No. 11301564). The second author was supported by National Natural Science Foundation of China (Grant Numbers 11371212, 10601063, 11271386). References [1] N. Akhmediev, A. Ankiewicz, Solitons, Nonlinear Pulses and Beams, Chapman and Hall, London, 1997. [2] A. Ambrosetti, E. Colorado, Standing waves of some coupled nonlinear Schr¨ oinger equations, J. Lond. Math. Soc. 75 (2007) 67–82. [3] L. Berg´ e, Wave collapse in physics: principles and applications to light and plasma waves, Phys. Rep. 303 (5–6) (1998) 259–370. [4] C. Br¨ andle, E. Colorado, A. Pablo, A concave–convex elliptic problem involving the fractional Laplacian, Proc. Roy. Soc. Edinburgh Sect. A 143 (1) (2010) http://dx.doi.org/10.1017/S0308210511000175. [5] J. Busca, B. Sirakov, Symmetry results for semilinear elliptic systems in the whole space, J. Differential Equations 163 (2000) 41–56. [6] X. Cabr´ e, Y. Sire, Nonlnear equations for fractional Laplacians I: regularity, maximum principles, and Hamiltonian estimates, arXiv:1012.0867v2. [7] L. Caffarelli, L. Silvesre, An extension problem related to the fractional Laplacian, Comm. Partial Differential Equations 32 (2007) 1245–1260. [8] X. Chang, Z.-Q. Wang, Ground state of scalar field equation involving fractional Laplacian with general nonlinearity, Nonlinearity 26 (2013) 479–494. [9] V. Coti Zelati, P. Rabinowitz, Homoclinic type solutions for a semilinear elliptic PDE on RN , Comm. Pure Appl. Math. 5 (1992) 1217–1269. [10] D.G. de Figueiredo, O. Lopes, Solitary waves for some nonlinear Schr¨ odinger systems, Ann. Inst. H. Poincar´ e Anal. Non Lin´ eaire 25 (2008) 149–161. [11] I. Ekeland, On the variational principle, J. Math. Anal. Appl. 47 (1974) 324–353. [12] M.M. Fall, E. Valdinci, Uniqueness and nondegeneracy of positive solutions of (−∆)s u + u = up in RN when s is close to 1, Comm. Math. Phys. 329 (2014) 383–404. [13] P. Felmer, A. Quaas, J.G. Tan, Positive solutions of nonlinear Schr¨ odinger equation with the fractional Laplacian, Proc. Roy. Soc. Edinburgh Sect. A 142 (2012) 1237–1262. [14] G. Fibich, G. Papanicolaou, Self-focusing in the perturbed and unperturbed nonlinear Schr¨ odinger equation in critical dimension, SIAM J. Appl. Math. 60 (2000) 183–240. [15] R.L. Frank, E. Lenzmann, Uniqueness oand nondegeneracy of ground states for (−∆)s Q + Q − Qα+1 = 0 in R, arXiv:1009.4042. [16] R.L. Frank, E. Lenzmann, L. Silvestre, Uniqueness of radial solutions for the fractional Laplacian, arXiv:1302.2652. [17] M. Lakshmanan, T. Kanna, R. Radhakrishnan, Shape-changing collisions of coupled bright solitons in birifringent optical fibers, Rep. Math. Phys. 46 (2000) 143–156. [18] N. Laskin, Fractional quantum mechanics and L´ evy path integrals, Phys. Lett. A 268 (2000) 298–305. [19] N. Laskin, Fractional Schr¨ odinger equation, Phys. Rev. E (3) 66 (2002) 056108. [20] T.C. Lin, J. Wei, Ground state of N coupled nonlinear Schr¨ oinger equations in R3 , n ≤ 3, Comm. Math. Phys. 255 (2005) 629–653. [21] L.A. Maia, E. Montefusco, B. Pellacci, Positive solutions for a weakly coupled nonlinear Schr¨ odinger system, J. Differential Equations 229 (2006) 743–767. [22] L.A. Maia, E. Montefusco, B. Pellacci, Orbital stability property for coupled nonlinear Schr¨ odinger equations, Adv. Nonlinear Stud. 10 (3) (2010) 681–705. [23] E.A. Ostrovskaya, Y.S. Kivshar, D.V. Skryabin, W.J. Firth, Stability of multihump optical solitons, J. Opt. B 1 (1999) 77–83. [24] X. Ros-Oton, J. Serra, The Pohozaev identity for the fractional Laplacian, Arch. Ration. Mech. Anal. 213 (2014) 587–628. [25] X. Ros-Oton, J. Serra, Nonexistence results for nonlocal equations with critical and supercritical nonlinearities, Comm. Partial Differential Equations 40 (2015) 115–133. [26] S. Secchi, On fractional Schr¨ odinger equations in RN without the Ambrosetti–Rabinowitz condition. Preprint, 2012. Available online at: http://arxiv.org/abs/1210.0755. [27] S. Secchi, Ground state solutions for nonlinear fractional Schr¨ odinger equations in RN , J. Math. Phys. 54 (2013) 031501. [28] R. Servadei, E. Valdinoci, Moutain pass solutions for nonlinear elliptic operators, J. Math. Anal. Appl. 389 (2012) 887–898. [29] R. Servadei, E. Valdinoci, Variational methods for non-local operator of elliptic type, Discrete Contin. Dyn. Syst. 5 (2013) 2105–2137.

Q. Guo, X. He / Nonlinear Analysis 132 (2016) 141–159

159

[30] B. Sirakov, Least energy solitary waves for a system of nonlinear Schr¨ odinger equations in Rn , Comm. Math. Phys. 271 (2007) 199–221. [31] M.V. Tratnik, J.E. Sipe, Bound solitary waves in optical fibers, Phys. Rev. A 38 (1998) 2011–2017. [32] M. Willem, Minimax Theorems, in: Progress in Nonlinear Differential Equations and their Applications, vol. 24, Birkh¨ auser, Boston, MA, 1996. [33] V.E. Zakharov, Collapse of Langmuir waves, Sov. Phys. JETP 35 (1972) 908–914.