Ground state solutions for quasilinear Schrödinger systems

Ground state solutions for quasilinear Schrödinger systems

J. Math. Anal. Appl. 389 (2012) 322–339 Contents lists available at SciVerse ScienceDirect Journal of Mathematical Analysis and Applications www.els...

254KB Sizes 3 Downloads 64 Views

J. Math. Anal. Appl. 389 (2012) 322–339

Contents lists available at SciVerse ScienceDirect

Journal of Mathematical Analysis and Applications www.elsevier.com/locate/jmaa

Ground state solutions for quasilinear Schrödinger systems Yuxia Guo a,1 , Zhongwei Tang b,∗,2 a b

Department of Mathematics, Tsinghua University, Beijing 100084, PR China School of Mathematical Sciences, Beijing Normal University, Laboratory of Mathematics and Complex Systems, Ministry of Education, Beijing 100875, PR China

a r t i c l e

i n f o

a b s t r a c t This paper is concerned with the quasilinear Schrödinger systems in R N :

Article history: Received 31 May 2011 Available online 26 November 2011 Submitted by J. Wei

⎧    1 2α 2 ⎪ ⎪ |u |α −2 | v |β u , ⎪ −u + λa(x) + 1 u − |u | u = ⎪ 2 α+β ⎨    1 2β ⎪ − v + λb(x) + 1 v − | v |2 v = |u |α | v |β−2 v , ⎪ ⎪ 2 α +β ⎪ ⎩ u (x) → 0, v (x) → 0 as |x| → ∞,

Keywords: Quasilinear Schrödinger systems Orlicz space Ground state solution

∗ where λ > 0 is a parameter, α > 2, β > 2, α + β < 2 · 2∗ and 2∗ = N2N −2 for N  3, 2 = +∞ for N = 1, 2 is the critical Sobolev exponent. By using the Nehari manifold method and concentration compactness principle in the Orlicz space, we prove the existence of ground state solution which localize near the potential well int{a−1 (0)} = int b−1 (0) for λ large enough. © 2011 Elsevier Inc. All rights reserved.

1. Introduction We consider the following quasilinear Schrödinger systems in the entire space R N :

⎧    1 2α ⎪ ⎪ | u |α − 2 | v |β u , −u + λa(x) + 1 u − |u |2 u = ⎪ ⎪ 2 α+β ⎨    1 2β ⎪ − v + λb(x) + 1 v − | v |2 v = |u |α | v |β−2 v , ⎪ ⎪ 2 α + β ⎪ ⎩ u (x) → 0, v (x) → 0 as |x| → ∞,

(1.1)

∗ where λ > 0 is a parameter, α > 2, β > 2, α + β < 2 · 2∗ and 2∗ = N2N −2 for N  3, 2 = +∞ for N = 1, 2 is the critical Sobolev exponent. We are interested in the positive solutions for (1.1) with least energy for λ large. In recent years, much attention has been devoted to the quasilinear equation of the form:

  −u + λ V (x)u − k u 2 u = |u | p −2 u in R N ,

(1.2)

part of the interest is due to the fact that the solution of (1.2) is related to the existence of solitary wave solution for the quasilinear Schrödinger equation of the form:







 



i ∂t w = − w + V (x) w − f | w |2 w − kh | w |2 h | w |2 w

* 1 2

Corresponding author. E-mail address: [email protected] (Z. Tang). Research supported by NSFC (10871110). Research supported by NSFC (10801013).

0022-247X/$ – see front matter doi:10.1016/j.jmaa.2011.11.064

©

2011 Elsevier Inc. All rights reserved.

in R N ,

(1.3)

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

323

where V (x) is a given potential, k is a positive constant, f , h are suitable functions. See for example, in [15], by using a constrained minimization argument, the existence of positive ground state solution was proved by Poppenberg, Schmitt and Wang. In [13], by a change of variables, Liu, Wang and Wang used an Orlicz space to prove the existence of soliton solution of (1.2) via mountain pass theorem. In [14], Colin and Jeanjean also made use of change variables but work in the Sobolev space H 1 (R N ), they proved the existence of positive solution for (1.2) from the classical results given by Berestycki and Lions [6]. In particularly, in [10], by using Nehari manifold method and concentration compactness principle (see [12]), the existence of ground state solution for quasilinear Schrödinger equation of the form:

   1  −u + λa(x) + 1 u − k |u |2 u = |u | p −1 u , 2

in R N ,

(1.4)

was proved by Guo and Tang. Moreover, they showed that the solution was localized near the potential well int{a−1 (0)} for λ large enough. It is worth to be pointed out that the existence of bump bound state solutions for the related semilinear Schrödinger equation (1.4) for k = 0 has been extensively studied. We refer the readers to Bartsch and Wang [5], Byeon and Wang [8], A. Ambrosetti, M. Badiale and S. Cingolani [3], A. Ambrosetti, A. Malchiodi, S. Secchi [4], Byeon and Wang [7] and the references therein. There are also several papers concerned with the following semilinear elliptic systems in an open domain D (possibly unbounded) in R N

⎧ 2α ⎪ ⎪ | u |α − 2 | v |β u , −ε 2 u + W (x)u = ⎪ ⎪ α +β ⎪ ⎪ ⎨ 2β −ε 2  v + V (x) v = |u |α | v |β−2 v , ⎪ α +β ⎪ ⎪ ⎪ ⎪ u > 0, v > 0, ⎪ ⎩ u (x) = 0, v (x) = 0, on ∂ D .

(1.5)

For instance, in [1], for D = R N , W (x) and V (x) are strictly positive and bounded away from zero in R N and there exists an open bounded domain Λ ⊂ R N such that for some point x0 ∈ Λ and positive number ρ0 ,

(i)

W (ξ ), V (ξ )  ρ0

∀ξ ∈ ∂Λ,

and

(ii)

W (x0 ), V (x0 ) < ρ0 ,

C.O. Alves proved that there exists an ε¯ 0 such that for all 0 < ε < ε¯ 0 , system (1.5) admits a positive solution (u ε , v ε ). Moreover the maximum points of u ε (x) and v ε (x) converge to the same point which is certain minimum point in Λ. For D bounded and α + β = 2∗ , W (x) = −λ, V (x) = −μ with 0 < λ∗ < λ, μ < −λ1 , Han [11] proved that problem (1.5) exits cat D ( D ) positive solutions, where −λ1 is the first eigenvalue of − in D. For more results related to problem (1.5) with bounded domain, we refer to C.O. Alves, D.C. de Morais and M.A.S. Souto [2], P. Han and Cao [9], Tang [16] and the references therein. The purpose of this paper is to study the existence of ground state solution for quasilinear Schrödinger system with potential well int{a−1 (0)} = int{b−1 (0)}. Compared with the semilinear case, our difficulties lie in two aspects: On one hand, the associated energy functional is not well defined on the natural functional space due to the growth condition. On the other hand, it causes problems in using the constrained method due to the appearance of the new term (|u |2 )u and (| v |2 ) v. We follow the idea of Liu, Wang and Wang [13] and treat our problem in an Orlicz space. By using Nehari manifold method and concentration compactness principle, we proved that there exists a ground state solution which localize near the potential well int{a−1 (0)} = int{b−1 (0)} for λ large. We assume ( A 1 ) a(x), b(x) ∈ C (R N , R) satisfy a(x)  0, b(x)  0 and Ω := int{a−1 (0)} = int{b−1 (0)} is nonempty with smooth boundary ¯ = a−1 (0) = b−1 (0). and Ω ( A 2 ) there exist M 1 , M 2 > 0 such that μ({x ∈ R N | a(x)  M 1 }) < ∞, μ({x ∈ R N | b(x)  M 2 }) < ∞, where μ denotes the Lebesgue measure in R N . We point out that the assumption ( A 2 ) is very weak even in dealing with the semilinear operator − + (λa(x) + 1) I on R N , which was firstly introduced by T. Bartsch and Z.Q. Wang [5]. For simplicity, in the following of the paper, we denote A λ (x) = λa(x) + 1 and B λ (x) = λb(x) + 1. Note that under our assumptions, the corresponding Euler functional for (1.1) is not well defined on the space











X := u , v ∈ H 1 R N RN

A λ (x)u 2 dx < ∞,



B λ (x)u 2 dx < ∞ . RN

We follow the idea √ of [13] and make √ the following change of variable: Let h(t ) = 12 t 1 + t 2 + 12 ln(t + 1 + t 2 ), then h(t ) is strictly monotone and hence has an inverse function denoted by f (s), moreover,

324

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

f (s) ∼

s,

| s | 1, 2 |s| s,

f  (s) = 

| s | 1,

1 1 + f 2 (s)

(1.6)

.

Let G (s) = f 2 (s), then G (s) is convex and there exists C 0 > 0 such that G (2s)  C 0 G (s). Moreover



G (s) = f 2 (s) ∼

s2 , 2|s|,

| s | 1, | s | 1,

(1.7)

and

G  (s) = 

2 f (s) 1+

f 2 (s)

G  (s) =

,

2

(1 + f 2 (s))2

> 0.

(1.8)

Now we introduce the Orlicz space (see [17]):









A

E Gλ = w

A λ G ( w ) < +∞

RN

equipped with the norm:

 

  | w |GA λ := inf ξ 1 + A λ G ξ −1 v dx. ξ >0

RN A

B

Then E G λ is a Banach space. Similarly we can define a Banach space E G λ with A λ replaced by B λ . Let





A

B







|∇ u |2 dx < ∞,

H λG =: (u , v ) u ∈ E G λ , v ∈ E G λ , RN

|∇ v |2 dx < ∞ RN

equipped with the norm:

  (u , v ) = ∇ u  2 + ∇ v  2 + |u | A λ + | v | B λ . L L G G λ We define the functional Φλ on H λG by

Φλ ( u , v ) =

1



2



 |∇ u |2 + A λ f 2 (u ) + |∇ v |2 + B λ f 2 ( v ) dx −

RN

2

α+β



f (u ) α f ( v ) β dx.

(1.9)

RN

Let







N λ := (u , v ) ∈ H λG \ {0, 0} Φλ (u , v ), (u , v ) = 0



be the Nehari manifold and

cλ =

inf

(u , v )∈ N λ

Φλ (u , v ).

(1.10)

We say that ( f (u λ ), f ( v λ )) is a least energy solution of (1.1) if (u λ , v λ ) ∈ N λ is such that c λ is achieved. Note that under our assumptions, for λ large enough, the following Dirichlet problem is a kind of “limit” problem defined in Ω :

⎧  1 2α ⎪ ⎪ u α −1 v β , −u + u − |u |2 u = ⎪ ⎪ 2 α + β ⎪ ⎪ ⎨  1 2β − v + v − | v |2 v = v β−1 u α , ⎪ 2 α +β ⎪ ⎪ ⎪ ⎪ u > 0, v > 0, ⎪ ⎩ u = 0, v = 0, on ∂Ω,

(1.11)

where Ω = int{a−1 (0)} = int{b−1 (0)}. Here are our main results: Theorem 1.1. Assume that ( A 1 ), ( A 2 ) are satisfied, 2 < α , 2 < β , α + β < 2 · 2∗ . Then for λ large, c λ is achieved by a critical point (u λ , v λ ) of Φλ such that ( f (u λ ), f ( v λ )) is a least energy solution of (1.1). Furthermore, for any sequence λn → +∞, (u λn , v λn ) has a subsequence converging to (u , v ) such that ( f (u ), f ( v )) is a least energy solution of (1.11).

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

325

The paper is organized as follows: Section 2 are preliminaries, we will prove some properties for the Orlicz norm and other important propositions that will be used for the proof of the main results. Section 3 is devoted to the “limit problem” and the proof of Theorem 1.1 is given in this section too. The ingredients of the paper is that we treat the problem in an Orlicz space and we give the precise estimates between the Orlicz norm and the related integral in an Orlicz space, which enable us to hand the problem in a uniform way. We believe that these estimates can be applied to other quasilinear problems or elliptic problems through the Orlicz space method. 2. Preliminaries In this section, we first prove a precise estimate between the Orlicz norm and some integrals in an Orlicz space H λG . In the following and throughout of the paper, we will use the same C to denote different constants. Lemma 2.1. There exist two constants C 1 > 0, C 2 > 0 such that for any (u , v ) ∈ H λG , it holds

 



 

2 C 1 min (u , v )λ , (u , v )λ 





 |∇ u |2 + |∇ v |2 dx +

RN







A λ f 2 (u ) + B λ f 2 ( v ) dx

RN

  2    C 2 max (u , v )λ , (u , v )λ . Proof. We first prove

 



 

2 C 1 min (u , v )λ , (u , v )λ 





 |∇ u |2 + |∇ v |2 dx +

RN





(2.1)



A λ f 2 (u ) + B λ f 2 ( v ) dx.

RN

We claim that for any ξ > 1, there exists a constant C > 0 such that

G (ξ t )  C ξ 2 G (t ) for ∀t > 0. Indeed, if 0 < t < 1, we have

G (ξ t )  C |ξ t |2 = C ξ 2 t 2  C ξ 2 G (t ),

if |ξ t | < 1

and

G (ξ t )  C |ξ t | = C ξ 2 t 2  C ξ 2 G (t ),

if |ξ t | > 1.

If t  1, we have

G (ξ t )  C |ξ t |  C ξ 2 t 2  C ξ 2 G (t ).    1 A Case 1. If RN A λ f 2 (u ) dx = RN A λ G (u ) dx < 1. Let ξ = [ RN A λ G (u ) dx] 2 , by the definition of | · |G λ , we have

|u |GA λ

 12





A λ G (u ) dx RN





 12 A λ G (u ) dx

RN



C

+



A λ G (u ) dx RN

+C

Aλ G RN





u

 1 [ RN A λ G (u ) dx] 2

dx

 12 A λ G (u ) dx

RN

 12 A λ G (u ) dx

 12



.

RN

Hence

∇ u L 2 + |u |GA λ



C

|∇ u | dx + RN



 12

2



A λ G (u ) dx

RN

. 

Case 2. If RN A λ f 2 (u ) dx = RN A λ G (u ) dx  1. Let ξ = RN A λ G (u ) dx. Then

|u |GA λ 



RN



A λ G (u ) dx + RN



A λ G (u ) dx RN



Aλ G 

u R N A λ G (u ) dx





dx  2 RN

A λ G (u ) dx.

326

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

Hence

∇ u L 2 + |u |GA λ

 ∇ u L 2 + 2

A λ G (u ) dx

RN

  12 2 

 2 ∇ u L 2 + A λ G (u ) dx RN



RN





|∇ u |2 dx +

4

A λ G (u ) dx .

RN

Thus there exists C > 0 such that



A



A

C min ∇ u  L 2 + |u |G λ , ∇ u  L 2 + |u |G λ

2 



|∇ u |2 dx +

 RN

A λ f 2 (u ) dx.

RN

Similarly we have

C min



 ∇ v L 2 + | v |GB λ , ∇ v L 2

2  + | v |GB λ



2

 RN

Therefore there exists C 1 > 0 such that



 

 2 

C 1 min (u , v )λ , (u , v )λ



B λ f 2 ( v ) dx.

|∇ v | dx + RN

  |∇ u | + |∇ v |2 dx +

RN







A λ f 2 (u ) + B λ f 2 ( v ) dx.

RN

By a similar argument, we can prove the other side of the inequality (2.1).

2

Lemma 2.2. The map: (u , v ) → ( f (u ), f ( v )) from H λG into L q (R N ) × L q (R N ) is continuous for 2  q  2 · 2∗ . Proof. Let (u 1 , v 1 ) = ( f (u ), f ( v )), then ∇ u 1 = √ ∇ u2

1+ f ( u )





, ∇ v1 = √

 |∇ u 1 |2 + |∇ v 1 |2 + A λ f 2 (u ) + B λ f 2 ( v ) dx 

RN

∇v

1+ f 2 ( v )





. By Lemma 2.1, we have

 |∇ u |2 + |∇ v |2 + A λ f 2 (u ) + B λ f 2 ( v ) dx

RN

  2    C 2 max (u , v )λ , (u , v )λ ,

which implies that ( f (u ), f ( v )) ∈ L 2 (R N ) × L 2 (R N ). On the other hand, we have



2 2 ∇ f (u ) dx =

RN





2 f (u )∇ u 2  dx  4 |∇ u |2 dx 1 + f 2 (u )

RN

RN

and





 1∗ 

 14 

 14 2·2 2 2  1 2 ∗ 2 f (u ) dx C C |∇ u | dx  C ∇ u L 2 + |u |λG 2 , ∇ f (u ) dx 2

RN

this implies that f (u ) ∈ L

RN 2·2∗

RN

(R N ). Taking account of the following inequality:

       f (u ) q   f (u )θ 2  f (u )12−θ ∗ L L L ·2 for q ∈ [2, 2 · 2∗ ] and 0 < θ < 1, we obtained that f (u ) ∈ L q (R N ) for q ∈ [2, 2 · 2∗ ]. Similarly we can prove that f ( v ) ∈ L q (R N ) for q ∈ [2, 2 · 2∗ ]. By considering the property of f (t ) (see (1.6)), the continuity of the map can be easily checked. 2 Let Φλ be defined in (1.9), the following facts are due to [13].

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

327

Proposition 2.3. (i) Φλ is well defined on H λG ; (ii) Φλ is continuous in H λG ; (iii) Φλ is G˘ateaux differentiable and for (u , v ) ∈ H λG , the G-derivative Φλ (u , v ) is continuous with respect to (u , v ) in the strong– weak topology, that is, if (un , v n ) → (u , v ) strongly in H λG , then Φλ (un , v n )  Φλ (u , v ) weakly. Moreover the G˘ateaux derivative Φλ (u , v ) has the form:



 Φλ (u , v ), (φ, ψ)

  = ∇ u ∇φ + ∇ v ∇ψ + A λ f (u ) f  (u )φ + B λ f ( v ) f  ( v )ψ dx RN





α+β

RN

f (u ) α −2 f ( v ) β f (u ) f  (u )φ dx − 2β α+β



f (u ) α f ( v ) β−2 f ( v ) f  ( v )ψ dx.

(2.2)

RN

Recall that a sequence {(un , v n )} ⊂ H λG is said to be a Palais–Smalec sequence ((PS)c sequence in short) of Φλ if Φλ (un , v n ) → 0 and Φλ (un , v n ) → c. We say that the functional Φλ satisfies (PS)c condition if any of the (PS)c sequence (up to a subsequence, if necessary) {(un , v n )} converges strongly in H λG . Let K λ = {(u , v ) ∈ H λG | Φλ (u , v ) = 0} be the critical set of Φλ . Lemma 2.4. There exists 0 < σ < 1 which is independent of λ such that

    (u , v )  (u , v )  σ , λ 1





for all (u , v ) ∈ K λ \ (0, 0) and λ  1.

Proof. For any (u , v ) ∈ K λ \ {(0, 0)}, we may assume that (u , v )λ  1 (otherwise the conclusion is true). Then we have



0 = Φλ (u , v ),

= RN



−2 

f (u ) f  (u )

,

f (v )



f (v )    

 f 2 (u ) f 2(v ) 2 2 2 2 1+ u | + A f ( u ) dx + 1 + v | + B f ( v ) dx |∇ |∇ λ λ 1 + f 2 (u ) 1 + f 2(v )







RN

f (u ) α f ( v ) β dx

RN

  |∇ u |2 + A λ f 2 (u ) + |∇ v |2 + B λ f 2 ( v ) dx − 2

RN

  2    C 1 min (u , v )λ , (u , v )λ − 2



f (u ) α f ( v ) β dx

RN





α +β f (u )

α 

 α+β

RN



α +β f (v )

β  α+β

RN

  2    2  α+β    2  C 1 min (u , v )λ , (u , v )λ − C 2 max (u , v )λ , (u , v )λ   2  α+β = C 1 (u , v )λ − C 2 (u , v )λ 2 , note that

α + β > 4, we obtain the desired results. 2

Lemma 2.5. There exists a positive constant c 0 > 0 independent of λ such that if {(un , v n )} is a (PS)c sequence for Φλ . Then it holds

    2(α + β)c 2(α + β)c lim sup(un , v n )λ  max , n→∞ α+β −4 α+β −4 and either c  c 0 or c = 0. f (u )

Proof. Since {(un , v n )} is a (PS)c sequence, take (φ, ψ) = ( f  (un ) , n (φ, ψ)λ  C (un , v n )λ and

f (vn ) ), f  (vn )

then |∇φ|  2|∇ un |, |∇ψ|  2|∇ v n |, and

328

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

 



1

c + o(1) −

o (un , v n )λ

α+β

     f (u n ) f ( v n ) Φλ (un , v n ) , , α+β f  (u n ) f  ( v n )  

 

 1 1 f 2 (u n ) 1 1 2 = − 1+ u | dx + − A λ f 2 (un ) dx |∇ n 2 α+β 2 α+β 1 + f 2 (u n )  = Φλ (un , v n ) −

RN



1





+

1 2



RN

α+β −4 2(α + β)



1

1+

α+β



2

f (vn)



 |∇ v n |2 dx +

1 + f 2(vn)

RN

1 2



1

α+β

 |∇ un |2 + |∇ v n |2 + A λ f 2 (un ) + B λ f 2 ( v n ) dx.



B λ f 2 ( v n ) dx RN

(2.3)

RN

It follows that





RN

    2(α + β) |∇ un |2 + |∇ v n |2 + A λ f 2 (un ) + B λ f 2 ( v n ) dx  c + o (un , v n )λ , α+β −4

by Lemma 2.1, we obtain

 



 

2 C 1 min (un , v n )λ , (un , v n )λ 

hence

2(α + β)

α+β −4



 

c + o (un , v n )λ ,

    2(α + β) 2(α + β)   lim sup (un , v n ) λ  max c, c . n→∞ α+β −4 α+β −4

On the other hand, for (un , v n )λ  1, we have

 









o (un , v n )λ = Φλ (un , v n ) ,



f (u n )

,

f (vn)

(2.4)



f  (u n ) f  ( v n )   2    2  α+β    2  C 1 min (un , v n )λ , (un , v n )λ − C 2 max (un , v n )λ , (un , v n )λ   2  α+β = C 1 (un , v n )λ − C 2 (un , v n )λ 2 ,

since 4 < α + β < 2 · 2∗ , there exists

σ1 > 0 (σ1 < 1) such that     2    f (u n ) f ( v n ) 1  Φλ (un , v n ) , ,  C 1 (un , v n )λ , for (un , v n )λ  σ1 .   f (u n ) f ( v n ) 4

Take c 0 =

(α +β−4)σ12 , then if c 2(α +β)

 max

2(α + β)

(α + β − 4)C 1

(2.5)

 c 0 , we have



c,

2(α + β)

(α + β − 4)C 1

c  σ1 < 1,

it follows from (2.4) and (2.5) that

1  4

2    (un , v n )λ  o (un , v n )λ ,

hence (un , v n )λ → 0 and c = 0. Therefore we proved that for above defined c 0 , either c  c 0 or c = 0.

2

Lemma 2.5 implies that any of (PS)c sequence is bounded. Moreover, it means that the critical value zero is isolated. Proposition 2.6. Let C¯ be fixed. Then for any λ > Λ , c  C¯ , then



lim sup

n→∞

B cR

where

B cR 

 > 0, there exist Λ , R  > 0 such that if {(un , v n )} is a (PS)c sequence of Φλ with

f (un ) α f ( v n ) β dx   for 4 < α + β < 2 · 2∗ ,



= {x ∈ R N | |x|  R  }.

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

329

Proof. For ∀ R > 0, let





Then

f (un ) dx  A(R )



 



1

2

λM1 + 1



 λa(x) + 1 f 2 (un ) dx



 (λa + 1) f 2 (un ) + |∇ un |2 dx

A(R )



1

 max

λM1 + 1 

1

α+β −4 

max

λM1 + 1

2(α + β)c

 ,

2(α + β)C¯

α+β −4

2(α + β)c

RN

B(R)



 θ2 

 ,

RN

   + o (un , v n )



λ

2(α + β)C¯



α+β −4





by (2.4)

 + o(1) (by Lemma 2.5) (2.6)



2 < q < 2∗

 1−θ∗



f 2·2 (un ) dx

f 2 (un ) dx





α+β −4

→ 0 as λ → ∞. 

 1q  2  q  q −1 f 2 (un ) dx  f (un ) dx μ B ( R ) q





B ( R ) := x ∈ R N |x|  R , a(x)  M 1 .

A(R )

1

λM1 + 1









A ( R ) := x ∈ R N |x|  R , a(x)  M 1 ,

2·2





 q−q 1

μ B(R)

(0 < θ < 1)

RN

  2  θ   1−θ     q −1  C max (un , v n )λ , (un , v n )λ 2 (un , v n )λ 2 μ B ( R ) q     q−1  (un , v n ) is bounded  C μ B(R) q λ → 0 as R → ∞.

(2.7)

By using the Gagliardo–Nirenberg inequality, we obtain



f (un ) α +β dx  C

B cR



 (α+β)θ 

 ((α+β)(2 1−θ )) 2 ∇ f (un ) 2 dx f (un ) 2 dx

B cR



f (un ) 2 dx +

A(R )

C

θ=

N (α + β − 2)

B cR

 (α +β)θ  C (un , v n )λ 



f (un ) 2 dx +

A(R )



 ((α+β)(2 1−θ )) f (un ) 2 dx

B(R)

 ((α+β)(2 1−θ ))

f (un ) 2 dx

   (un , v n ) is bounded λ

B(R)

 → 0 as λ, R → ∞ by (2.6) and (2.7) . 

In a similar way, we have



f ( v n ) α +β dx   ,

for λ, R large.

B cR

At last, using the Hölder inequality, we obtain

lim sup

n→∞

B cR



f (un ) α f ( v n ) β dx 



β α 

 α+β  α+β f (un ) α +β dx f ( v n ) α +β dx

B c

 .

2(α + β)

B c

2

Lemma 2.7. c λ := inf N λ Φλ (u , v ) is achieved by some (u , v ) = (0, 0).



330

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

Proof. By the definition of c λ and Ekeland Variational Principle, there exists a (PS)cλ sequence {(un , v n )} of Φλ . By Lemma 2.5, we know that {(un , v n )} is bounded. Hence (up to a subsequence) we have

un  u ,



in L 2 ,

vn  v

∇ v n  ∇ v in L 2 ,

∇ un  ∇ u , un → u ,

a.e. in R N ,

vn → v

for 2  q  2 · 2∗ .

f ( v n )  f ( v ) in L q ,

f (un )  f (u ),

We first prove that (u , v ) = (0, 0) and (u , v ) ∈ N λ . In fact,

Φλ ( u n , v n ) − =−



1

1



2

f 2 (u n ) 1+

2 RN

Φλ (un , v n ), f 2 (u

n)



f (u n ) f  (u n )

|∇ un |2 +

,

f (vn) f (vn)

f 2(vn) 1+

f 2(v

n)

   |∇ v n |2 dx + 1 −

2

α+β

   = c λ + o(1) + o (un , v n )λ , it deduces that



1−

2



α+β



f (un ) α f ( v n ) β dx

RN

(2.8)

   f (un ) α f ( v n ) β dx  c λ + o(1) + o (un , v n ) . λ

RN

0 = 14 c0  14 cλ , since f ( v n ) → f ( v ) strongly in L p ( B R ), for any R > 0 and 2 < p < 2∗ . By Proposition 2.6, there exists Λ0 > 0, R 0 > 0 such that for λ  Λ0 , R  R 0 Let



1−

thus



2



α+β

f (un ) α f ( v n ) β dx < 0 ,

B cR

f (u ) α f ( v ) β dx  1 c λ > 0,

(2.9)

4

BR

which implies that (u , v ) = (0, 0). Now we prove (u , v ) ∈ N λ . Indeed since {(un , v n )} is a (PS)cλ sequence, we have

   Φλ (un , v n ), (u , v ) =





 ∇ un ∇ u + ∇ v n ∇ v + A λ f (un ) f  (un )u + B λ f ( v n ) f  ( v n ) v dx

RN

=

α+β +

Let gn := √ quence,

f (u n ) 1+ f 2 ( u n )

gn  g = 

RN



α+β

f (un ) α −2 f (un ) f  (un ) f ( v n ) β u dx

   f (un ) α f ( v n ) β−2 f ( v n ) f  ( v n ) v dx + o (u , v ) . λ

RN



f 2 (u )



in L q R N .

Similarly we have



f (vn) 1 + f 2(vn)

(2.10)

, then gn is bounded in L q (R N ) for 2  q  2 · 2∗ . By the continuity of f , we have, up to a subse-

f (u ) 1+







f (v )

1 + f 2(v )





in L q R N .

Note that ∇ un  ∇ u, ∇ v n  ∇ v in L 2 , we have

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

lim

n→∞

RN



=

331

  ∇ un ∇ u + ∇ v n ∇ v + A λ f (un ) f  (un )u + B λ f ( v n ) f  ( v n ) v dx   |∇ u |2 + |∇ v |2 + A λ f (u ) f  (u )u + B λ f ( v ) f  ( v ) v dx.

RN

In the following, we will prove that



RN

and





f (un ) α −2 f (un ) f  (un ) f ( v n ) β u dx →

f (u ) α −2 f (u ) f  (u ) f ( v ) β u dx

RN



f (un ) α f ( v n ) f  ( v n ) f ( v n ) β−2 v dx →

RN

f (u ) α f ( v ) f  ( v ) f ( v ) β−2 v dx.

RN

Let hn :=

α −2 | f ( v )|β f (u ) | f (un )|√ n n

1+ f 2 ( u n )

hn  h =:

α +β

, then hn is bounded in L α+β−2 (R N ). Again, by the continuity of f , we have

| f (u )|α −2 | f ( v )|β f (u )  1 + f 2 (u )



α +β



in L α+β−2 R N .

In a similar way,

| f (u )|α | f ( v )|β−2 f ( v ) | f (un )|α | f ( v n )|β−2 f ( v n )    1 + f 2(vn) 1 + f 2(v )

α +β





in L α+β−2 R N .

Passing to the limits in (2.10), we get





 |∇ u |2 + |∇ v |2 + A λ f (u ) f  (u )u + B λ f ( v ) f  ( v ) v dx

RN

=



α+β

RN

f (u ) α −2 f (u ) f  (u ) f ( v ) β u dx + 2β α+β



f (u ) α f ( v ) β−2 f ( v ) f  ( v ) v dx,

RN

which is equivalent to Φλ ((u , v )), (u , v ) = 0, that is (u , v ) ∈ N λ and thus Φλ ((u , v ))  c λ . In the following it is sufficient to prove that Φλ ((u , v ))  c λ . f (u ) f (v ) Since Φλ ((un , v n )) → 0 as n → ∞. Take (φ, ψ) = ( f  (u ) , f  ( v ) ), by using the similar arguments as we did above, one can see that (u , v ) satisfies



1+ RN

1+





f 2 (u )

|∇ u |2 + A λ f 2 (u ) dx +

f 2 (u )

1+

RN



f 2(v ) 1+

f 2(v )

|∇ v |2 + B λ f 2 ( v ) dx = 2

f (u ) α f ( v ) β dx.

RN

(2.11) On the other hand, by the continuity of f we know that f (un ) → f (u ), f ( v n ) → f ( v ) a.e. in R , thus by using the facts that ∇ un  ∇ u , ∇ v n  ∇ v , f (un )  f (u ), f ( v n )  f ( v ) in L 2 (R N ) we have N

  Φλ ( u , v ) =



1



2 RN



+

α+β

1 2

RN



 lim

n→∞

− 1 2

RN n→∞



which implies Φλ ((u , v ))  c λ .

1

2

1 + f 2 (u )

1+

α+β

α+β

2

1 + f 2(v )

1+

1

  1 − |∇ u |2 +

f 2(v )



1



f 2 (u )



α+β

2

RN

1+

1



+ lim → cλ



1

1 + f 2 (u n )

 1+

α+β

  1 − |∇ v |2 +

f 2 (u n )



1

2

1 + f 2(vn)



 |∇ v n | +

1 2

 B λ f 2 ( v ) dx

α+β 2

2



1

  1 − |∇ un |2 +

f 2(vn)



A λ f 2 (u ) dx



1

α+β −

 A λ f 2 (un ) dx

1

α+β



 B λ f ( v n ) dx 2

332

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

3. Proof of the main results Consider the following quasilinear Schrödinger system in an open domain Ω ⊂ R N :

⎧  1 2α ⎪ ⎪ | u |α − 1 v β , −u + u − |u |2 u = ⎪ ⎪ 2 α + β ⎪ ⎪ ⎨  1 2β − v + v − | v |2 v = | v |β−1 u α , ⎪ 2 α + β ⎪ ⎪ ⎪ ⎪ u > 0, v > 0, ⎪ ⎩ u = 0, v = 0 on ∂Ω,

(3.1)

where Ω = int{a−1 (0)} = int{b−1 (0)}. We make the same change of variable and use the same notation as in the previous sections. For reader’s convenience, we sketch the notations below. Define the corresponding Orlicz space E G (Ω) by







E G (Ω) = (u , v )



f 2 (u ) dx < +∞,

Ω

f 2 ( v ) dx < +∞ , Ω

with norm:

(u , v ) := |u |G + | v |G . G (Ω) The space H G (Ω) is defined by







2 2 H G (Ω) := (u , v ) ∈ E G (Ω) |∇ u | dx < +∞, |∇ v | dx < +∞ , Ω

Ω

with norm:

  (u , v ) = ∇ u  2 + ∇ v  2 + (u , v ) . L L Ω G (Ω) Let H 0G (Ω) denote the closure of (C 0∞ (Ω))2 under the norm of  · Ω . We define the functional ΦΩ on H 0G (Ω) by

 1  ΦΩ ( u , v ) =



2



 |∇ u |2 + |∇ v |2 + f 2 (u ) + f 2 ( v ) dx −

Ω

2

α+β



f (u ) α f ( v ) β dx.

(3.2)

Ω

Let





 



 N Ω := (u , v ) ∈ H G (Ω) \ (0, 0) : ΦΩ (u , v ), (u , v ) = 0



be the manifold and c (Ω) = inf N Ω ΦΩ (u , v ). We say ( f (u ), f ( v )) is a least energy solution of (3.1) if (u , v ) ∈ N Ω is such that c (Ω) is achieved. Lemma 3.1. limλ→+∞ c λ = c (Ω), where c λ is defined in (1.10). Proof. It is easy to see that c λ  c (Ω) for λ  1. In the following we will prove that c λ is monotone increasing with respect to λ. Assume (u λ1 , v λ1 ), (u λ2 , v λ2 ) are such that c λ1 , c λ2 are achieved. Then





 |∇ u λ2 |2 + |∇ v λ2 |2 + A λ1 f (u λ2 ) f  (u λ2 )u λ2 + B λ1 f ( v λ2 ) f  ( v λ2 ) v λ2 dx

RN





α+β +

RN



α+β

f (u λ ) α −2 f ( v λ ) β f (u λ ) f  (u λ )u λ dx 2 2 2 2 2

f (u λ ) α f ( v λ ) β−2 f ( v λ ) f  ( v λ ) v λ dx. 2 2 2 2 2

RN

We first prove that there exists 0 < γ  1 such that

γ (u λ2 , v λ2 ) ∈ N λ1 . This is sufficient to prove that

(3.3)

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339





γ2

 |∇ u λ2 |2 + |∇ v λ2 |2 dx +

RN

=





α+β +

RN



α+β





333



A λ1 f (γ u λ2 ) f  (γ u λ2 )γ u λ2 + B λ1 f (γ v λ2 ) f  (γ v λ2 )γ v λ2 dx

RN

f (γ u λ ) α −2 f (γ u λ ) β f (γ u λ ) f  (γ u λ )γ u λ dx 2 2 2 2 2

f (γ v λ ) α f (γ v λ ) β−2 f (γ v λ ) f  (γ v λ )γ v λ dx. 2 2 2 2 2

(3.4)

RN

Let

g 1 (γ ) =

f (γ u )[ 2α | f (γ u )|α −2 | f (γ v )|β − A ]u λ2 α +β λ2 λ2 λ1 λ2  dx, 2 γ 1 + f (γ u λ2 )

RN

g 2 (γ ) =

f (γ v )[ 2β | f (γ v )|β−2 | f (γ u )|α − B ] v λ2 α +β λ2 λ2 λ1 λ2  dx. 2 γ 1 + f (γ v λ2 )

RN



And g (γ ) =: g 1 (γ ) + g 2 (γ ), then limγ →0 g (γ )  0 and 0 < RN |∇ u λ2 |2 + |∇ v λ2 |2  g (1), hence there exists 0 < γ0  1 such  that RN |∇ u λ2 |2 + |∇ v λ2 |2 = g (γ0 ), i.e. γ0 (u λ2 , v λ2 ) ∈ N λ1 . Thus c λ1  Φλ1 (γ0 (u λ2 , v λ2 )). Next we will prove that

    Φλ 1 γ 0 ( u λ 2 , v λ 2 )  Φλ 2 ( u λ 2 , v λ 2 ) = c λ 2 . For this purpose, we consider the function 2α

f (γ u λ2 )[ α +β | f (γ u λ2



ρ1 (γ ) =

ρ1 (γ ) defined by

)|α −2 | f (γ u

λ2 )|

β

− A λ2 ] u λ2

γ 1 + f 2 (γ u λ2 )

,

A direct computation shows that

d

 



f (γ u λ2 )

  0,

1 + f 2 (γ u λ2 )

it follows that

d dγ

 

f (γ u λ2 )

 2α | f (γ u )|α −2 | f (γ v )|β u λ2 λ2 λ2 α +β

1 + f 2 (γ u λ2 )

γ

 0.

Moreover,

d

 2α | f (γ u )|α −2 | f (γ v )|β u  λ2 λ2 λ2 α +β



γ α −3 f  (γ u )| f (γ v )|β u 2 γ λ2 λ2 λ2 α +β (α − 2)| f (γ u λ2 )| = 2 γ 2α 2α α −2 f  (γ v )| f (γ v )|β−1 v u γ α −2 | f (γ v )|β u λ2 λ2 λ2 λ2 λ2 λ2 α +β β| f (γ u λ2 )| α +β β| f (γ u λ2 )| + − γ2 γ2 2α 1 f (γ u λ ) α −3 f  (γ u λ ) f  (γ v λ ) f (γ v λ ) β−1 u λ = 2 2 2 2 2 2 α+β γ   f (γ v λ2 ) f (γ u λ2 ) f (γ u λ2 ) f (γ v λ2 ) × (α − 2)  γ u λ2 + β  γ v λ2 −  f (γ v λ2 ) f (γ u λ2 ) f (γ u λ2 ) f  (γ v λ2 )   2α 1 f (γ u λ ) α −3 f  (γ u λ ) f  (γ v λ ) f (γ v λ ) β−1 u λ f (γ u λ2 ) β γ v λ − f (γ v λ2 )  (α > 2) 2 2 2 2 2 2 α +β γ2 f  (γ u λ2 ) f  (γ v λ2 ) 2α

 0.



In the above last inequality, we have used the facts that f (t ) 1 + f 2 (t )  2t, ∀t  0, β > 2. Now we look at d dγ

(

A λ2 f (γ u λ2 )u λ2

γ 1+ f 2 (γ u λ2 )

). Since t =

1 2

 f (t ) 1 + f (t )2 +

1 2

ln( f (t ) +



1 + f (t )2 ), we obtain that for any t ∈ R

334

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

  t − f (t ) 1 + f (t )2 1 + f (t )2 < 0, which implies

d





A λ2







f (γ u λ2 )u λ2

= A λ2

f 2 (γ u

)

γ u 2λ2 − f (γ u λ2 )u λ2 1 + f 2 (γ u λ2 ) − γ 1+ f 2 (γ λu2 ) u 2λ2 λ2

γ 2 (1 + f 2 (γ u λ2 ))  γ u λ2 − f (γ u λ2 ) 1 + f 2 (γ u λ2 )(1 + f 2 (γ u λ2 )) u λ2 = A λ2 γ 2 (1 + f 2 (γ u λ2 ))2 < 0.

γ 1 + f 2 (γ u λ2 )

Hence we obtain that

d dγ

ρ1 (γ ) > 0. ρ2 (γ ) by

Now we define



f (γ v λ2 )[ α +β | f (γ v λ2 )|β−2 | f (γ u λ2 )|α − B λ2 ] v λ2



ρ2 (γ ) =

γ 1 + f 2 (γ v λ2 )

.

In a similar way, we have

d dγ

ρ2 (γ ) > 0. ρ (γ ) =: ρ1 (γ ) + ρ2 (γ ) is strictly monotone increasing in γ > 0. Now we consider the function h(γ )

It concludes that defined by

h(γ ) =

1





2



γ 2 |∇ u λ2 |2 + γ 2 |∇ v λ2 |2 + A λ2 f 2 (γ u λ2 ) + B λ2 f 2 (γ v λ2 ) dx −

RN

Then for 0 < α < 1, by using the monotonicity of

dh(γ ) dγ





ρ (γ ), we have 

γ |∇ u λ2 |2 + A λ2 f (γ u λ2 ) f  (γ u λ2 )u λ2 dx +

= RN

− −

2α α+β

α+β

α+β

RN

RN



−γ



f (γ u λ ) α f  (γ v λ ) f (γ v λ ) β−1 v λ dx 2 2 2 2 α −2 f ( v λ ) β f (u λ ) f  (u λ )u λ dx 2 2 2 2

f (u λ2 )

f (u λ ) α f ( v λ ) β−2 f ( v λ ) f  ( v λ ) v λ dx 2 2 2 2 2 

A λ2 f (u λ2 ) f  (u λ2 )u λ2 + B λ2 f ( v λ2 ) f  ( v λ2 ) v λ2 dx

RN



RN



− RN



A λ2 f (γ u λ2 ) f  (γ u λ2 )u λ2 + B λ2 f (γ v λ2 ) f  (γ v λ2 ) v λ2 dx

2α f (γ u λ ) α −2 f (γ v λ ) β f (γ u λ ) f  (γ u λ )u λ dx 2 2 2 2 2 α+β

+γ RN

f (γ u λ ) α f (γ v λ ) β−2 f (γ v λ ) f  (γ v λ ) v λ dx 2 2 2 2 2



α+β

f (γ u λ ) α f (γ v λ ) β dx. 2 2

RN



RN

α+β

α+β



γ |∇ v λ2 |2 + B λ2 f (γ v λ2 ) f  (γ v λ2 ) v λ2 dx





+



f (γ u λ ) α −1 f  (γ u λ ) f (γ v λ ) β u λ dx 2 2 2 2

RN







RN











2

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

335

    = γ ρ1 (1) − ρ1 (γ ) + γ ρ2 (1) − ρ2 (γ )   = γ ρ (1) − ρ (γ )   > 0 by the monotone of ρ (γ ) . Therefore h(γ ) is monotone increasing with respect to 0 < γ  1, this deduces that

Φλ 1



 1 γ ( u λ2 , v λ2 ) =





2

2

− 1

2

2

RN





 1 γ |∇ u λ2 | + A λ1 f (γ u λ2 ) dx + 2



2

α+β





2

 1 |∇ u λ2 |2 + A λ2 f 2 (u λ2 ) dx +





2

RN





2

α+β

f (u λ ) α f ( v λ ) β dx 2 2



γ 2 |∇ v λ2 |2 + B λ1 f 2 (γ v λ2 ) dx

RN

f (γ u λ ) α f (γ v λ ) β dx 2 2

RN



 |∇ v λ2 |2 + B λ2 f 2 ( v λ2 ) dx

RN

RN

 = Φλ 2 ( u λ 2 , v λ 2 ) . 

Thus we proved that c λ1  c λ2 for λ1 < λ2 . Now we assume that limλ→+∞ c λ = k  c (Ω). Suppose k < c (Ω), then for any sequence {λn }, λn → +∞ as n → ∞, we λ have c λn → k < c (Ω). We assume that (un , v n ) is such that c λn is achieved, by Lemma 2.5, {(un , v n )} is bounded in H Gn . Since (un , v n ) H 1  (un , v n ) H λn , for λn  1, {(un , v n )} is bounded in H 1G , as a result, we have G

∇ un  ∇ u ,

G

  ∇ un  ∇ u in L 2 R N , q

f (un ) → f (u ),

f ( v n ) → f ( v ) in L loc ,

 q

in L R

un  u ,

vn  v

un → u ,

vn → v ,

 N

for 2  q  2 · 2∗ ,

for 2  q  2 · 2∗ ,

,

a.e. in R N .

We claim that (u , v )|Ω c = (0, 0), where Ω c =: {x | x ∈ R N \ Ω}. Indeed, it is sufficient to prove ( f (u ), f ( v ))|Ω c = (0, 0). If not, we have ( f (u ), f ( v ))|Ω c = (0, 0). Then there exists a compact subset F ⊂ Ω c with dist{ F , ∂Ω} > 0 such that ( f (u ), f ( v ))| F = (0, 0), and







f 2 (un ) dx → F

f 2 (u ) dx > 0,



f 2 ( v n ) dx →

F

F

f 2 ( v ) dx > 0. F

0 > 0 such that a(x)  0 , b(x)  0 for any x ∈ F . By the choice of {(un , v n )}, we have     f (u n ) f ( v n ) , 0 = Φλ n (un , v n ) , f  (u n ) f  ( v n )  

 2 f (u n ) 2 2 = 1+ |∇ un | + A λn f (un ) dx 1 + f 2 (u n )

Moreover, there exists



RN



+

1+

RN

  1 Φλn (un , v n ) =





2



2



 +

1 2

2

RN

 |∇ un |2 + A λn f 2 (un ) + |∇ v n |2 + B λn f 2 ( v n ) dx −

RN

1



α β |∇ v n | + B λn f ( v n ) dx − 2 f (un ) f ( v n ) dx, 2

1 + f 2(vn)

hence

=



f 2(vn)



1

α+β −

1+

1

α+β

f 2 (u n ) 1 + f 2 (u n )

 1+





2

f (vn) 1 + f 2(vn)

|∇ un |2 dx +

1 2

RN





|∇ v n |2 dx + RN

− 1 2



2

α+β

RN



1

α+β −

f (un ) α f ( v n ) β dx

A λn f 2 (un ) dx RN

1

α+β



B λn f 2 ( v n ) dx RN

336

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

 

1 2

 

1 2





1

α+β





A λn f 2 (un ) + B λn f 2 ( v n ) dx

RN



1



p+1



 (λn 0 + 1) f 2 (un ) + (λn 0 + 1) f 2 ( v n ) dx

F

→ +∞ as n → ∞. This contradiction shows that ( f (u ), f ( v ))|Ω c = (0, 0) and so does (u , v ). Now we show that





for 2 < q < 2∗ .

f ( v n ) → f ( v ) in L q R N ,

f (un ) → f (u ),

(3.5)

Suppose (3.5) is not true, by concentration compactness principle of P.L. Lions (see [12]), there exist δ > 0, xn , x˜ n ∈ R N with |xn | → +∞, |˜xn | → ∞ such that



ρ > 0 and

f (un ) − f (u ) 2 dx  δ > 0,

lim sup

n→∞

B ρ (xn )



f ( v n ) − f ( v ) 2 dx  δ > 0.

lim sup

n→∞

(3.6)

B ρ (˜xn )

On the other hand, by the choice of {(un , v n )}, we have



Φλn (un , v n ) 

1



2

 +  

2

1



2

 +  =

1

1 2



1

α+β



α+β 1

α+β





2

 +

1 2

1

B ρ (xn )∩{x|a(x) M 1 }

 

λn M 2

α+β

 

λn M 1

α+β



f ( v n ) − f ( v ) 2 dx −

 f ( v n ) − f ( v ) 2 dx

B ρ (˜xn )∩{x|b(x) M 2 }

 f (un ) − f (u ) 2 dx − o(1)

B ρ (xn )

1



 f (un ) − f (u ) 2 dx



f (un ) − f (u ) 2 dx −

B ρ (xn )

α+β

 λn b(x) + 1 f 2 ( v n ) dx

B ρ (˜xn )∩{x|b(x) M 2 }

B ρ (˜xn )

1

 λn a(x) + 1 f 2 (un ) dx



 

λn M 1

1





B ρ (xn )∩{x|a(x) M 1 }

1





 

λn M 2

 f ( v n ) − f ( v ) 2 dx − o(1)

B ρ (˜xn )

→ +∞,

as n → ∞.

In the above proof, we have used the facts that μ( B ρ (xn ) ∩ {x | a(x)  M 1 }) → 0, μ( B ρ (˜xn ) ∩ {x | b(x)  M 2 }) → 0 as n → ∞ and also the L 2 -norms of f ( v n ) and f (un ) are bounded. Since {(un , v n )} is bounded, by the Fatou lemma, we obtain





|∇ u |2 dx  lim

|∇ un |2 dx,

n→∞

RN

Ω





|∇ v |2 dx  lim

|∇ v n |2 dx,

n→∞

RN

Ω







f (u ) f (u )u dx  lim

n→∞

RN

Ω



f  ( v ) f ( v ) v dx  lim



n→∞

Ω

f  (un ) f (un )un dx,

RN

f  ( v n ) f ( v n ) v n dx.

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

337

On the other hand, by the choice of (un , v n ), we have





 |∇ un |2 + A λn f (un ) f  (un )un + |∇ v n |2 + B λn f ( v n ) f  ( v n ) v n dx

RN

Hence







=

α+β

RN

f (un ) α −2 f ( v n ) β f (un ) f  (un )un dx + 2β α+β



f (un ) α f ( v n ) β−2 f ( v n ) f  ( v n ) v n dx.

RN

  |∇ u |2 + f  (u ) f (u )u + |∇ v |2 + f  ( v ) f ( v ) v dx

Ω



2

 lim

|∇ un | dx + lim

n→∞

n→∞

RN

RN





|∇ v n |2 dx + lim

 lim

n→∞

n→∞

RN

2α α+β

 lim

n→∞

+ lim

f  (un ) f (un )un dx

RN



n→∞

α+β

f  ( v n ) f ( v n ) v n dx

RN

f (un ) α −2 f ( v n ) β f (un ) f  (un )un dx

f (un ) α f ( v n ) β−2 f ( v n ) f  ( v n ) v n dx.

In the following we first prove that





lim

n→∞

α+β

RN

(3.7)

RN

f (un ) α −2 f ( v n ) β f (un ) f  (un )un dx = 2α α+β



f (u ) α −2 f ( v ) β f (u ) f  (u )u dx.

RN

∗ | f (un√ )|α −2 | f ( v n )|β un , since f (un ), f ( v n ) are bounded in L α +β (R N ) and un is bounded in L 2 (R N ), thus gn is 1+ f 2 ( u n )

Let gn :=

2∗ (α +β)

bounded in L 2∗ (α+β−2)+α+β (R N ). By the continuity of f , we have

gn  g =: and



| f (u )|α −2 | f ( v )|β u  1 + f 2 (u )

2∗ (α +β)

f (un ) α −2 f ( v n ) β f (un ) f  (un )un dx −

RN

f (u ) α −2 f ( v ) β f (u ) f  (u )u dx

  f (un ) α −2 f ( v n ) β f (un ) f  (un )un − f (u ) f (un ) α −2 f ( v n ) β f  (un )un dx

= RN



+ =





RN







in L 2∗ (α+β−2)+α+β R N

RN

 



α −2  f ( v n ) β f  (un )un − f (u ) α −2 f ( v ) β f  (u )u dx

f (u ) f (un )





gn f (un ) − f (u ) dx +

RN

f (u )( gn − g ) dx

RN

=: I + II.

Since gn  g, one can see that II → 0 as n → ∞. And



I=





gn f (un ) − f (u ) dx

RN



= RN

  f (un ) α −2 f ( v n ) β f  (un )un f ( v n ) − f ( v ) dx

338

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339





 1∗ 

2∗

2

|un | dx RN

   C (un , v n )

β 

−2 

1  αα+β  α+β  α+β f (un ) α +β dx f ( v n ) α +β dx f (un ) − f (u ) α +β dx

RN

RN

   f (un ) − f (u ) α+β . H1 L

RN

G

Note that f (un ) → f (u ) in L q (R N ) for 2 < q < 2∗ . Since 4 < α + β < 2 · 2∗ , we can choose θ ∈ (0, 1) such that θ q + (1 − θ)2 · 2∗ , then





f (un ) − f (u ) α +β dx 

RN



q

f (u n ) − f (u )

RN





lim

n→∞

α+β

RN

∗ f (un ) − f (u ) 2·2

f (un ) α f ( v n ) β−2 f ( v n ) f  ( v n ) v n dx = 2α α+β

By (3.7), we have



Ω

1  1−θ

RN

→ 0.

Similarly, we have

 θ1 

α+β =

  2α |∇ u |2 + f  (u ) f (u )u + |∇ v |2 + f  ( v ) f ( v ) v dx  α+β

RN



f (u ) α −2 f ( v ) β f (u ) f  (u )u dx

RN





+

f (u ) α f ( v ) β−2 f ( v ) f  ( v ) v dx.

α+β

f (u ) α f ( v ) β−2 f ( v ) f  ( v ) v dx.

(3.8)

RN

Thus ∃0 < α0  1 such that

ΦΩ



α0 (u , v ) ∈ N Ω and    α0 (u , v )  ΦΩ (u , v ) ,

hence c (Ω)  ΦΩ (α0 (u , v ))  ΦΩ ((u , v ))  limn→∞ Φλn = k < c (Ω). A contradiction.

2

The proof of Theorem 1.1. It is sufficient to prove that (u , v ) ∈ N Ω and ΦΩ (u , v ) = c (Ω). Suppose that {(un , v n )} is a sequence such that (un , v n ) ∈ N λ , Φλn (un , v n ) = c λn , by the proof of Lemma 3.1, we have

∇ un  ∇ u ,

  ∇ v n  ∇ v in L 2 R N ,

f (un ) → f (u ),





for 2 < q < 2∗ ,

f ( v n ) → f ( v ) in L q R N ,

and (u , v )|Ω c = 0. Moreover ΦΩ (u , v )  limn→∞ Φλn (un , v n )  c (Ω), if (u , v ) ∈ N Ω , then ΦΩ (u , v ) = c (Ω). In order to prove (u , v ) ∈ N Ω , it is sufficient to prove





λn a(x) f 2 (un ) dx = 0,

lim

n→∞

RN

|∇ un |2 dx =

n→∞

RN

and

lim

n→∞

RN

n→∞

λn b(x) f 2 ( v n ) dx = 0,

RN





lim

lim

|∇ u |2 dx,

RN





lim

n→∞

|∇ v n |2 dx =

RN

f (un ) f  (un )un dx =



RN

f (u ) f  (u )u dx,

|∇ v |2 dx

RN



f ( v n ) f  ( v n ) v n dx =

lim

n→∞

RN



f ( v ) f  ( v ) v dx.

RN

In fact, if one of the above limits does not hold. By the Fatou lemma, we have



RN



 2α |∇ v |2 + |∇ u |2 + f (u ) f  (u )u + f ( v ) f  ( v ) v dx < α+β +



RN



α+β

f (u ) α −2 f ( v ) β f (u ) f  (u )u dx

f (u ) α f ( v ) β−2 f ( v ) f  ( v ) v dx,

RN

similar as in the proof of Lemma 3.2, there exists α0 ∈ (0, 1) such that α0 (u , v ) ∈ N Ω and c (Ω)  ΦΩ (α0 u , α0 v ) < ΦΩ (u , v )  limn→∞ Φλn (un , v n )  c (Ω). A contradiction. We complete the proof of Theorem 1.1. 2

Y. Guo, Z. Tang / J. Math. Anal. Appl. 389 (2012) 322–339

339

Acknowledgments We thank the referees for their useful comments and suggestions.

References [1] C.O. Alves, Local mountain pass for a class of elliptic system, J. Math. Anal. Appl. 335 (2007) 135–150. [2] C.O. Alves, D.C. de Morais Filho, M.A.S. Souto, On systems of elliptic equations involving subcritical or critical Sobolev exponents, Nonlinear Anal. 42 (2000) 771–787. [3] A. Ambrosetti, M. Badiale, S. Cingolani, Semiclassical states of nonlinear Schrödinger equations, Arch. Ration. Mech. Anal. 140 (1997) 285–300. [4] A. Ambrosetti, A. Malchiodi, S. Secchi, Multiplicity results for some nonlinear Schrödinger equations with potentials, Arch. Ration. Mech. Anal. 159 (2001) 253–271. [5] T. Bartsch, Z.Q. Wang, Multiple positive solutions for a nonlinear Schrödinger equation, Z. Angew. Math. Phys. 51 (2000) 266–284. [6] H. Berestycki, P.L. Lions, Nonlinear scalar field equations I, Arch. Ration. Mech. Anal. 82 (1983) 313–346. [7] J. Byeon, Z.Q. Wang, Standing waves with a critical frequency for nonlinear Schrödinger equations, Arch. Ration. Mech. Anal. 165 (2002) 295–316. [8] J. Byeon, Z.Q. Wang, Standing waves with a critical frequency for nonlinear Schrödinger equations II, Calc. Var. Partial Differential Equations 18 (2003) 207–219. [9] D. Cao, P. Han, High energy positive solutions of Neumann problem for an elliptic system of equations with critical nonlinearities, Calc. Var. Partial Differential Equations 25 (2006) 161–185. [10] Y. Guo, Z. Tang, Ground state solutions for quasilinear Schrödinger equation, preprint. [11] P. Han, The effect of the domain topology on the number of the positive solutions of an elliptic system involving critical Sobolev exponents, Houston J. Math. 32 (2006) 1241–1257. [12] P.L. Lions, The concentration-compactness principle in the calculus of variations. The locally compact case. Part I, Ann. Inst. H. Poincaré Anal. Non Linéaire 1 (1984) 109–145. [13] J.Q. Liu, Y.Q. Wang, Z.Q. Wang, Soliton solutions for quasilinear Schrödinger equation, II, J. Differential Equations 187 (2003) 473–493. [14] Mathieu Colin, Louis Jeanjean, Solutions for a quasilinear Schrödinger equation: a dual approach, Nonlinear Anal. 56 (2) (2004) 213–226. [15] M. Poppenberg, K. Schmitt, Z.Q. Wang, On the existence of solutions to quasilinear Schrödinger equations, Calc. Var. Partial Differential Equations 14 (2002) 329–344. [16] Z. Tang, Sign-changing solutions of critical growth nonlinear elliptic systems, Nonlinear Anal. 64 (2006) 2480–2491. [17] M.M. Rao, Z.D. Ren, Theory of Orlicz Space, Marcel Dekker, New York, 1991.