Asymptotics for cuspidal representations by functoriality from GL(2)

Journal of Number Theory 164 (2016) 323–342

Contents lists available at ScienceDirect

Journal of Number Theory www.elsevier.com/locate/jnt

Asymptotics for cuspidal representations by functoriality from GL(2) Huixue Lao a , Mark McKee b , Yangbo Ye b,∗ a

School of Mathematical Sciences, Shandong Normal University, Jinan, Shandong 250014, China b Department of Mathematics, The University of Iowa, Iowa City, IA 52242-1419, USA

a r t i c l e

i n f o

Article history: Received 8 June 2015 Received in revised form 16 January 2016 Accepted 22 January 2016 Available online 3 March 2016 Communicated by David Goss MSC: 11F70 11F66 11F30

a b s t r a c t Let π be a unitary automorphic cuspidal representation of GL2 (QA ) with Fourier coefficients λπ (n). Asymptotic expansions of certain sums of λπ (n) are proved using known functorial liftings from GL2 , including symmetric powers, isobaric products, exterior squares, and base change. These asymptotic expansions are manifestation of the underlying functoriality and reflect value distribution of λπ (n) on integers, squares, cubes and fourth powers. © 2016 Elsevier Inc. All rights reserved.

Keywords: Cuspidal representation Functoriality GL(2) Fourier coefficient Asymptotic expansion

* Corresponding author. E-mail addresses: [email protected] (H. Lao), [email protected] (M. McKee), [email protected] (Y. Ye). http://dx.doi.org/10.1016/j.jnt.2016.01.008 0022-314X/© 2016 Elsevier Inc. All rights reserved.

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

324

1. Introduction Let π = ⊗πp be a unitary automorphic irreducible cuspidal representation of GL2 (QA ) with Fourier coefficients λπ (n). For each finite p for which πp is unramified we can associate to πp Satake parameters απ (p), βπ (p) ∈ C such that λπ (p) = απ (p) + βπ (p), |απ (p)βπ (p)| = 1. For ramified πp , one can still choose απ (p), βπ (p) ∈ C, by allowing some of them to be zero, such that the finite-part automorphic L-function attached to π is L(s, π) =

∞   −1  −1 λπ (n) = 1 − απ (p)p−s 1 − βπ (p)p−s s n p<∞ n=1

(1.1)

for Re s > 1. If π is associated with a holomorphic cusp form for a congruence subgroup of SL2 (Z), the Ramanujan bound was proved by Deligne [4] |απ (p)| = |βπ (p)| = 1 for p with πp being unramified. For other cases the best known bound toward the Ramanujan conjecture is 7

|απ (p)|, |βπ (p)| ≤ p 64 ,

(1.2)

which is due to Kim and Sarnak [12]. Let us review known functoriality from an automorphic cuspidal representation π of GL2 (QA ) (cf. Kim [11]). 1.1. The symmetric square lift. By Gelbart and Jacquet [5] Sym 2 π is an automorphic representation of GL3 (QA ). It is cuspidal if and only if π is not monomial. Here π is monomial which means that there is a nontrivial grössencharacter ω so that π ∼ = π ⊗ ω. ¯ This in turn means that π corresponds to a dihedral Galois representation Gal(Q/Q). 3 1.2. The symmetric cube. Kim and Shahidi [14] proved that Sym π is an automorphic representation of GL4 (QA ). In addition, Sym 3 π is cuspidal if and only if π does not correspond to a dihedral or tetrahedral Galois representation. We recall π is tetrahedral means the π is not monomial and there exists a grössencharacter η = 1 so that Ad 2 π ∼ = Ad 2 π ⊗ η. Here Ad 2 π = Sym 2 π ⊗ ωπ−1 , where ωπ is the central character of π. 1.3. The symmetric fourth power. By Kim [10] Sym 4 π is an automorphic representation of GL5 (QA ). It is either cuspidal or unitarily induced from cuspidal representations of GL2 (QA ) and GL3 (QA ). More precisely, Sym 4 π is cuspidal unless π corresponds to either a dihedral, tetrahedral, or octahedral Galois representation (Kim and Shahidi [13]). Here π is octahedral means Ad 3 π is cuspidal, but there exists a nontrivial quadratic character μ so that Ad 3 π ∼ = Ad 3 π ⊗ μ.

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

325

1.4. The Rankin–Selberg convolution of GL2 ×GL2 . Let π1 and π2 be unitary automorphic cuspidal representations of GL2 (QA ). By Ramakrishnan [29] their isobaric product π1  π2 is an automorphic representation of GL4 (QA ). In case of π1 and π2 being not associated to characters of a quadratic extension, π1  π2 is cuspidal if and only if π1 and π2 are not twisted equivalent by any grössencharacter. 1.5. The Rankin–Selberg convolution of GL2 × GL3 . Let π2 and π3 be unitary automorphic cuspidal representations of GL2 (QA ) and GL3 (QA ), respectively. Then, by Kim and Shahidi [14], π2  π3 is an automorphic representation of GL6 (QA ). It is isobaric and is cuspidal or irreducibly induced from unitary cuspidal representations. When π2 is not dihedral, π2  π3 is cuspidal unless π3 is a twist of Ad(π2 ) by a grössencharacter (Ramakrishnan and Wang [30]). 1.6. Exterior square of GL4 . Let π4 be a unitary automorphic cuspidal representations of GL4 (QA ). By Kim [12] and Henniart [7] the exterior square lift ∧2 (π4 ) is an automorphic representation of GL6 (QA ). Moreover ∧2 (π4 ) is of the form Ind σ1 ⊗ · · · ⊗ σk where each σi is a cuspidal representation of GLni (QA ), ni > 1. Its cuspidality was classified by Asgari and Raghuram [2]. 1.7. Base change. Let π be a unitary automorphic cuspidal representation of GL2 over Q. Suppose that there is a Galois number field F of prime degree over Q such that π  π ⊗ ηF/Q ,

(1.3)

where ηF/Q is a class character of Q× A attached to F . In other words, ηF/Q is trivial exactly on Q× NF/Q (FA× ). Note that ηF/Q is a character of order . By Langlands [16] and Arthur and Clozel [1], (1.3) implies that −1 π, π ⊗ ηF/Q , . . . , π ⊗ ηF/Q

lift to an automorphic cuspidal representation Π of GL2 over F . This base change Π is stable under the action of Gal(F/Q) and given by

L(s, Π) =

−1 

  j L s, π ⊗ ηF/Q .

(1.4)

j=0

The subject goal of this paper is to establish an asymptotic expansion of a certain sum of Fourier coefficient λπ (n) of the GL2 cuspidal representation π based on its functorial lifting in each of the cases 1.1–1.7. These sums and asymptotic expansions can be regarded as a manifestation of the underlying functoriality. They also provide deep insight into the value distribution of λπ (n) on positive integers, squares, cubes and fourth powers. Asymptotics of this kind are important consequences in Rankin [31] and Selberg [33] as they developed their theory. Similar asymptotics were later obtained by Lao and Sankaranarayanan [17–19], Lau and Lü [20], Lau, Lü and Wu [21], and Lü [24–26].

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

326

Theorem 1.1. Let π be a unitary automorphic irreducible cuspidal representation of GL2 (QA ). Assume that π is not monomial so that Sym 2 π is cuspidal. Then the L-function ∞  |λπ (n2 )|2 L2 (s) = ns n=1

converges absolutely for Re s > 1 and has an analytic continuation to Re s > for a simple pole at s = 1. Moreover, 

1 2

 4 |λπ (n2 )|2 = c2 x + Oπ x 5 ,

except

(1.5)

n≤x

where c2 = Res L2 (s). s=1

Theorem 1.2. Let π be a unitary automorphic irreducible cuspidal representation of GL2 (QA ). Assume that π does not correspond to a dihedral or tetrahedral Galois representation so that Sym 3 π is cuspidal. Then the L-function L3 (s) =

∞  |λπ (n3 )|2 ns n=1

converges absolutely for Re s > 1, and has an analytic continuation to Re s > for a simple pole at s = 1. Moreover, 

21 32

except

 15  |λπ (n3 )|2 = c3 x + Oπ x 17 ,

n≤x

where c3 = Res L3 (s). s=1

Theorem 1.3. Let π be a unitary automorphic irreducible cuspidal representation of GL2 (QA ). Assume that π does not correspond to a dihedral, tetrahedral, or octahedral Galois representation so that Sym 4 π is cuspidal. Then the L-function L4 (s) =

∞  |λπ (n4 )|2 ns n=1

converges absolutely for Re s > 1 and has an analytic continuation to Re s > for a simple pole at s = 1. Moreover,  n≤x

where c4 = Res L4 (s). s=1

 12  |λπ (n4 )|2 = c4 x + Oπ x 13 ,

7 8

except

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

327

Theorem 1.4. Let π1 and π2 be two unitary automorphic cuspidal representations of GL2 (QA ) not associated to characters from a quadratic field. Suppose that π1 and π2 are not twisted equivalent by any grössencharacter so that π1  π2 is cuspidal. Then the L-function L1,1 (s) =

∞  |λπ1 (n)|2 |λπ2 (n)|2 ns n=1

converges absolutely for Re s > 1 and has an analytic continuation to Re s > for a simple pole at s = 1. Moreover, 

1 2

 15  |λπ1 (n)|2 |λπ2 (n)|2 = c1,1 x + Oπ1 ,π2 x 17 ,

except

(1.6)

n≤x

where c1,1 = Res L1,1 (s). s=1

Theorem 1.5. Let π1 and π2 be two unitary automorphic cuspidal representations of GL2 (QA ). Suppose that π1  Sym 2 (π2 ) is cuspidal. Then the L-function L1,2 (s) =

∞  |λπ1 (n)|2 |λπ2 (n2 )|2 ns n=1

converges absolutely for Re s > 1 and has an analytic continuation to Re s > for a simple pole at s = 1. Moreover, 

 35  |λπ1 (n)|2 |λπ2 (n2 )|2 = c1,2 x + Oπ1 ,π2 x 37 ,

21 32

except

(1.7)

n≤x

where c1,2 = Res L1,2 (s). s=1

Theorem 1.6. Let π1 and π2 be two automorphic cuspidal representations of GL2 (QA ) with trivial central characters. Suppose that π1 and π2 are not twisted equivalent by any grössencharacter and that π1  π2 is cuspidal. Then 

|λπ1 (a2 )|2 λπ1 (b2 )λπ2 (b2 )λπ2 (c2 )λπ1 (c2 )|λπ2 (d2 )|2

abcd≤x 35

= x(c1 log x + c0 ) + Oπ1 ,π2 (x 37 log x)

(1.8)

for some constants c1 and c0 . Theorem 1.7. Let π be a unitary automorphic cuspidal representation of GL2 (QA ), and χ a nontrivial Dirichlet character modulo prime . Denote by η the idelic lift of χ (cf. [6]). Assume π  π ⊗ η. Then

328

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342



−1  −1 

|λπ (aij )|2 χj (aij )

i=0 j=0 a ≥1 −1 ij −1 i=0 j=0 aij ≤x

  42 −1 = xP−1 (log x) + Oπ,χ x 42 +1 log−1 x ,

(1.9)

where P−1 is a polynomial of degree − 1. We remark that Theorems 1.1–1.3 are a generalization of Theorem 1.1 in Lao and Sankaranarayanan [17,18] where the same asymptotics (with the same error terms) were proved assuming π corresponds to a self-dual holomorphic cusp form of even integral weight on the full modular group SL2 (Z). The proof of Theorem 1.2 in [18], however, should not have used a subconvexity bound for L(s, Sym 2 f ) proved by Li [22]. Using a subconvexity bound for ζ(s) which was recently improved by Bourgain [3] we can correct the proof and obtain a better result. Theorem 1.8. Let f be a self-dual holomorphic cusp form for a congruence subgroup of SL2 (Z). Then 

 139  λ2f (n2 ) = c2 x + Of x 181 .

(1.10)

n≤x

In the next section we will outline the main techniques which we will use. 2. Some properties of L-functions We need a classical lemma by Landau [15] (cf. Ram Murty [28]). Lemma 2.1. Let an ≥ 0 and set f (s) =

∞  an . ns n=1

Suppose f (s) is convergent in some half-plane and that it has analytic continuation, except for a pole at s = 1 of order k, to the entire complex plane and it satisfies a functional equation cs Δ(s)f (s) = c1−s Δ(1 − s)f (1 − s), where c is a certain positive constant and Δ(s) =

N  i=1

Γ(αi s + βi ),

αi > 0.

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

329

Then 

 2A−1  an = xPk−1 (log x) + O x 2A+1 logk−1 x ,

n≤x

where A =

N i=1

αi , and Pk−1 (y) is a polynomial in y of degree k − 1.

Now we summarize some basic properties of automorphic L-functions. Let π be a unitary automorphic cuspidal representation of GL2 (QA ), with Fourier coefficients λπ (n) normalized by λπ (1) = 1. For simplification, we write απ (p) = αp and βπ (p) = βp . By (1.1), we have λπ (pk ) =

k 

αpk−i βpi .

(2.1)

i=0

Let S be a finite set of primes containing all p at which πp is ramified. Then the mth-symmetric power L-function attached to π L(s, Sym m π) =

∞  λSym m π (n) ns n=1

has an Euler product outside S S

m

L (s, Sym π) =

m 

(1 − αpm−i βpi p−s )−1

p∈S / i=0

for Re s > 1. Similarly the Rankin–Selberg L-function of Sym m π times its contragredient L(s, Sym m π × Sym m π

) =

∞  λSym m π×Sym m π (n) ns n=1

has an Euler product outside S

) = LS (s, Sym m π × Sym m π

m  m 

(1 − αpm−i βpi α ¯ pm−j β¯pj p−s )−1

p∈S / i=0 j=0

for Re s > 1. For p ∈ / S its parameters are  m−i i m−j j    αp βp α ¯ p β¯p  ≤ max |αp |2m , |βp |2m . Consequently

330

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

LS (s, Sym m π × Sym m π

)     |αp |2m −1 |αp |2m−2 αp β¯p −1 = 1− 1 − ps ps p∈S /

 |αp |2m−4 αp2 β¯p2 −1 |αp |2m−2 α ¯ p βp −1  × 1− × 1 − ps ps 2m−4 2 2    α ¯ p βp −1 |αp | |αp |2m−4 −1 × 1− ···. 1 − ps ps

(2.2)

Then we have for p ∈ /S λSym m π×Sym m π (p) ¯ p βp = |αp |2m + |αp |2m−2 αp β¯p + |αp |2m−2 α + |αp |2m−4 αp2 β¯p2 + |αp |2m−4 + |αp |2m−4 α ¯ p2 βp2 + · · ·

(2.3)

λSym m π×Sym m π (pj ) ¯ p βp = |αp |2mj + |αp |2mj−2 αp β¯p + |αp |2mj−2 α + 2|αp |2mj−4 αp2 β¯p2 + 2|αp |2mj−4 + 2|αp |2mj−4 α ¯ p2 βp2 + · · ·

(2.4)

2

for j ≥ 2. We note that there are dm2 (pj ) ≤ (j + 1)m terms on the right hand side of   (2.4) and the absolute value of each term is bounded by max |αp |2mj , |βp |2mj . Lemma 2.2. Let π be a unitary automorphic cuspidal representation of GL2 (QA ). Then for 0 ≤ n ≤ 8  |αp |n + |βp |n  1+ (2.5) pσ p converges for σ > 1. Proof. We know that for 1 ≤ m ≤ 4, the symmetric power representation Sym m π is either cuspidal or unitarily induced from cuspidal representations ([5,10,13] and [14]). Consequently (2.2) converges absolutely for Re s > 1. Without loss of generality, assume |αp | ≥ |βp |. Recall |αp βp | = 1. First (2.5) converges for σ > 1 for n = 2 by the convergence of (2.2) for m = 1. Then for m = 2, the product over p of all factors other than the leading one in (2.2) converges absolutely for σ > 1 because of |αp βp | = 1. The convergence of (2.2) for m = 2 leads to the convergence of (2.5) for n = 4 for σ > 1. Repeating this argument we can show the convergence of (2.5) for n = 6 and 8 for σ > 1. Then we have the convergence of (2.5) for n ≤ 8 for σ > 1. 2 Li and Young [23, p. 1638] proved the case of n = 8 with a bound under the assumption that π is self-dual of trivial central character and Sym 4 π is cuspidal.

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

331

3. Proof of Theorems 1.1–1.3 Define a Dirichlet series Lm (s) =

∞ ∞   |λπ (nm )|2 |λπ (pmj )|2  = s n pjs p n=1 j=0

(3.1)

for Re s sufficiently large. The series Lm (s) is not expected to be automorphic and probably has no analytic continuation beyond Re s > 1/2. For our propose we want to factor a Rankin–Selberg L-function out of Lm (s). Lemma 3.1. Let π be a unitary automorphic cuspidal representation of GL2 (QA ) such that Sym m π is cuspidal. Then we have for m = 2, 3, 4, Lm (s) = L(s, Sym m π × Sym m π

)Um (s), where Um (s) as an Euler product is absolutely convergent and hence nonzero for Re s > 7 for m = 2, Re s > 23 32 for m = 3, Re s > 8 for m = 4.

1 2

Proof. First we factor Lm (s) as in (3.1): Lm (s) =

∞  λSym m π×Sym m π (pj ) p

=

pjs

j=0

−

∞  Pj (αp , βp )  j=1

pjs

∞  λSym m π×Sym m π (pj )  p

×

pjs ∞

j=0

 p

1 − ∞ j=0

j=1

Pj (αp , βp )p−js



λSym m π×Sym m π (pj )p−js

)Um (s) = L(s, Sym m π × Sym m π

(3.2)

by denoting the last product in (3.2) by Um (s). Here Pj (αp , βp ) = λSym m π×Sym m π (pj ) − |λπ (pmj )|2 for j ≥ 1. By (2.1), we have |λπ (p

mj

mj 2    )| =  αpmj−i βpi  2

i=0

= |αp |2mj + |αp |2mj−2 αp β¯p + |αp |2mj−2 α ¯ p βp + |αp |2mj−4 αp2 β¯p2 + |αp |2mj−4 + |αp |2mj−4 α ¯ p2 βp2 + · · ·

(3.3)

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

332

for j ≥ 1. Comparing (2.3), (2.4) and (3.3) we get for p ∈ /S P1 (αp , βp ) = 0, ¯ p2 βp2 + · · · Pj (αp , βp ) = |αp |2mj−4 αp2 β¯p2 + |αp |2mj−4 + |αp |2mj−4 α + |βp |2mj−4 βp2 α ¯ p2 + |βp |2mj−4 + |βp |2mj−4 β¯p2 αp2 ,

j ≥ 2.

Now let us turn to the series in the denominator in (3.2) for p ∈ / S. By (2.2), we get ∞  λSym m π×Sym m π (pj ) −1

pjs

j=0

 |αp |2m  ¯ p βp  |αp |2m−2 αp β¯p  |αp |2m−2 α = 1− 1− 1− s s s p p p 2m−4 2 ¯2    αp βp ¯ p2 βp2  |αp | |αp |2m−4 α |αp |2m−4 × 1− 1 − 1 − ···. ps ps ps Thus for p ∈ /S ∞ − ∞ j=0

j=2

Pj (αp , βp )p−js

λSym m π×Sym m π

(pj )p−js



∞ 

2

(k + 1)O(m

2mk−4 ) |αp |

+ |βp |2mk−4 . (3.4) pkσ

k=2

First let us consider the case of m = 2. Then for p ∈ / S (3.4) becomes

∞ 

|αp |4k−4 + |βp |4k−4 pkσ

(k + 1)N

k=2

∞ |αp |4 + |βp |4  k + 1 N |αp |4k−8 + |βp |4k−8 ≤3 p2σ 3 p(k−2)σ N

(3.5)

k=2

for an integer N > 0, which converges when σ > 7/16 by (1.2). For σ ≥ 7/16 + ε the last series in (3.5) is ≤B(ε), i.e., bounded by a positive constant which may depend on ε. Thus U2 (s) in (3.2) is dominated by 

1 + c3N B(ε)

p∈S /

|αp |4 + |βp |4  p2σ

which is in turn dominated by  p∈S /

|αp |4 + |βp |4 c3 p2σ

N

1+

B(ε)

,

(3.6)

where c > 0 is an implied constant from (3.5). By Lemma 2.2 for n = 4, (3.6) and hence U2 (s) are absolutely convergent and nonzero for σ > 1/2.

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

333

Then let us turn to the case of m = 3. Then for p ∈ / S (3.4) becomes

∞ 

(k + 1)N

k=2

|αp |6k−4 + |βp |6k−4 pkσ

which converges when σ > 21/32 by (1.2). Thus for σ ≥ 21/32 + ε, U3 (s) in (3.2) is dominated by 

|αp |8 + |βp |8 ) 3 p2σ

N

1+

p∈S /

C(ε)

(3.7)

for some C(ε) > 0. Since (3.7) converges absolutely for σ > 1/2 by Lemma 2.2, U3 (s) converges absolutely for σ > 21/32. / S (3.4) is Now let m = 4. Then for p ∈

∞ 

(k + 1)N

k=2

|αp |8k−4 + |βp |8k−4 pkσ

which converges for σ > 7/8 by (1.2). Thus for σ ≥ 7/8 + ε, U4 (s) in (3.2) is dominated by 

|αp |12 + |βp |12 ) 3 p2σ

N

1+

p∈S /

D(ε)

(3.8)

for some D(ε) > 0. Now (3.8) in turn is dominated by 

|αp |8 + |βp |8 3 p2σ−7/16

N

1+

p∈S /

D(ε)

(3.9)

by (1.2). By Lemma 2.2 (3.9) converges absolutely for 2σ − 7/16 > 1, i.e., for σ > 23/32. Consequently, U4 (s) converges absolutely for Re s > 7/8. 2 The L-function L(s, Sym m π ×Sym m π

) satisfies the conditions of Landau’s Lemma 2.1 2 with k = 1 and 2A = (m + 1) . Thus we have 

2  1−  λSym m π×Sym m π (n) = dm x + O x (m+1)2 +1 ,

n≤x

where

). dm = Res L(s, Sym m π × Sym m π s=1

(3.10)

334

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

Expand Um (s) =

∞  um ( )

s

=1

which converges absolutely for Re s > for m = 4 by Lemma 3.1. By (3.2) |λπ (nm )|2 =

1 2



for m = 2, Re s >

23 32

for m = 3, and Re s >

λSym m π×Sym m π (k)um ( ).

7 8

(3.11)

n=k

For m = 2, (3.11) becomes 

|λπ (n2 )|2 =

n≤x



u2 ( )

≤x

=

 ≤x

= xd2



λSym 2 π×Sym 2 π (k)

k≤x/

 x  45  x u2 ( ) d2 + O  u2 ( ) ≤x



 4  |u ( )|  2 + O x5 4 5 ≤x

(3.12)

by (3.10). Using  u2 ( )

>x

x− 2 + , 1

 |u2 ( )| 4

≤x

5

1,

we get  u2 ( ) ≤x



 1  = U2 (1) + O x− 2 + .

(3.13)

By (3.12) and (3.13) we conclude     λπ (n2 )2 = c2 x + O x 45 , n≤x

where c2 = d2 U2 (1) = Res L2 (s) or by a Tauberian argument. s=1

The cases of m = 3, 4 are similar. 2 4. Proof of Theorems 1.4–1.7 Proof of Theorem 1.5. Denote by αp and βp the Langlands parameters of π1 at p, and by γp and δp the Langlands parameters of π2 . Suppose without loss of generality that |αp | ≥ 1 and |γp | ≥ 1. Let T be a finite set of primes containing all p at which either

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

335

π1p or π2p is ramified. Then at p ∈ / T Sym 2 π2 has parameters γp2 , γp δp and δp2 , while 2 π1  Sym π2 has αp γp2 , αp γp δp , αp δp2 , βp γp2 , βp γp δp and βp δp2 . Thus for p ∈ /T λ(π1 Sym 2 π2 )×( π1 Sym 2 π 2 ) (p) = |αp + βp |2 |γp2 + γp δp + δp2 |2 = |λπ1 (p)|2 |λπ2 (p2 )|2

(4.1)

and both λ(π1 Sym 2 π2 )×( π1 Sym 2 π 2 ) (pj ) and |λπ1 (pj )|2 |λπ2 (p2j )|2 for j ≥ 2 are equal to |αp |2j |γp |4j + |αp |2j−4 |γp |4j (αp2 + α ¯ p2 ) + |αp |2j |γp |4j−4 (γp2 + γ¯p2 )   + Oj |αp |2j |γp |4j−4 + |αp |2j−2 |γp |4j−2 + |αp |2j−4 |γp |4j ,

(4.2)

where the implied constant depends on j polynomially of a bounded degree N . Define Qj (αp , βp , γp , δp ) = λ(π1 Sym 2 π2 )×( π1 Sym 2 π 2 ) (pj ) − |λπ1 (pj )|2 |λπ2 (p2j )|2

(4.3)

/ T that for j ≥ 1. Then by (4.1) and (4.2) we have at p ∈ Q1 (αp , βp , γp , δp ) = 0 Qj (αp , βp , γp , δp ) j |αp |2j |γp |4j−4 + |αp |2j−2 |γp |4j−2 + |αp |2j−4 |γp |4j

(4.4)

for j ≥ 2, where the implied constant depends on j polynomially of a bounded degree N . Consequently by (4.3), the L-function

L1,2 (s) =

∞ ∞   |λπ1 (n)|2 |λπ2 (n2 )|2 |λπ1 (pj )|2 |λπ2 (p2j )|2 = , s n pjs p j=0 n=1

for Re s sufficiently large, can be factored as

L1,2 (s) =

∞  λ(π1 Sym 2 π2 )×( π1 Sym 2 π 2 ) (pj ) p

=

pjs

j=0

−

∞  Qj (αp , βp , γp , δp )  j=1

pjs

∞  λ(π1 Sym 2 π2 )×( π1 Sym 2 π 2 ) (pj )  p

pjs

j=0

 × 1 − ∞ p

j=0

∞ j=1

Qj (αp , βp , γp , δp )p−js



λ(π1 Sym 2 π2 )×( π1 Sym 2 π 2 ) (pj )p−js

π1  Sym 2 π

2 ))V1,2 (s) = L(s, (π1  Sym 2 π2 ) × (

(4.5)

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

336

by denoting the last product in (4.5) by V1,2 (s) =

∞  u1,2 ( ) =1

s

.

Note that by (4.4) the sum of Qj in (4.5) is actually taken over j ≥ 2 for p ∈ / T. For p ∈ / T by (4.4) the series ∞  Qj (αp , βp , γp , δp ) j=2

(4.6)

pjs

is dominated by ∞  |αp |2j |γp |4j−4 + |αp |2j−2 |γp |4j−2 + |αp |2j−4 |γp |4j (j + 1)N pjσ j=2

which in turn is dominated by ∞  j=2

(j + 1)N 7 pjσ−(6j−4) 64

by (1.2). Thus (4.6) converges absolutely when σ > Therefore when σ >

21 32 ,

21 32 ,

and its sum is then

|αp |4 |γp |4 + |αp |2 |γp |6 + |γp |8 . p2σ

V1,2 (s) is dominated by

 p∈T /

1+

|αp |4 |γp |4 + |αp |2 |γp |6 + |γp |8  . p2σ

(4.7)

By Lemma 2.2 and Cauchy’s inequality, (4.7) converges absolutely for σ > 12 . Thus V1,2 (s) converges absolutely for Re s > 21 32 . Applying Landau’s Lemma 2.1 to L(s, (π1  Sym 2 π2 ) × (

π1  Sym 2 π

2 )), we have 

 35  λ(π1 Sym 2 π2 )×( π1 Sym 2 π 2 ) (n) = d1,2 x + O x 37 .

n≤x

By the Dirichlet convolution on the right hand side of (4.5) we have |λπ1 (n)|2 |λπ2 (n2 )|2 =

 k=n

and

u1,2 ( )λ(π1 Sym 2 π2 )×( π1 Sym 2 π 2 ) (k)

(4.8)

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342



|λπ1 (n)|2 |λπ2 (n2 )|2 =

n≤x

 ≤x

=

 ≤x

u1,2 ( )



337

λ(π1 Sym 2 π2 )×( π1 Sym 2 π 2 ) (k)

k≤x/

 x  35  x 37 u1,2 ( ) d1,2 + O ,

which leads to (1.7), by (4.8) and the fact that V1,2 (s) converges absolutely for Re s > 21 32 . 2 Proof of Theorem 1.4. is similar to that of Theorem 1.5 with the same notation. We have for p ∈ /T λ(π1 π2 )×( π1  π2 ) (p) = |αp + βp |2 |γp + δp |2 = |λπ1 (p)|2 |λπ2 (p)|2 ;

(4.9)

Rj (αp , βp , γp , δp ) = λ(π1 π2 )×( π1  π2 ) (pj ) − |λπ1 (pj )|2 |λπ2 (pj )|2 j |αp |2j |γp |2j−4 + |αp |2j−2 |γp |2j−2 + |αp |2j−4 |γp |2j

(4.10)

for j ≥ 2, where the implied constant depends on j polynomially of a bounded degree N . Consequently by (4.9) and (4.10), the L-function L1,1 (s) =

∞  |λπ1 (n)|2 |λπ2 (n)|2 = L(s, (π1  π2 ) × (

π1  π

2 ))V1,1 (s) ns n=1

(4.11)

with V1,1 (s) convergent absolutely for Re s > 12 , by an argument similar as below (3.6). Applying Landau’s Lemma 2.1 to L(s, (π1  π2 ) × (

π1  π

2 )), we have   15  λ(π1 π2 )×( π1  π2 ) (n) = d1,1 x + O x 17 . n≤x

By the Dirichlet convolution on the right hand side of (4.11) we have (1.6). 2 Proof of Theorem 1.6. Under the assumption that π1 and π2 have trivial central characters we know αp βp = γp δp = 1. With π1 and π2 as above so that π1  π2 is cuspidal, its exterior square representation ∧2 (π1  π2 ) has Langlands parameters αp2 , 1, βp2 , γp2 , 1, δp2 . Thus ∧2 (π1  π2 ) ∼ = Ind(Sym 2 π1 ⊗ Sym 2 π2 ) and L(s, ∧2 (π1  π2 ) × ∧2 (

π1  π

2 ))

1 )L(s, Sym 2 π1 × Sym 2 π

2 ) = L(s, Sym 2 π1 × Sym 2 π × L(s, Sym 2 π2 × Sym 2 π

1 )L(s, Sym 2 π2 × Sym 2 π

2 ).

(4.12)

338

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

Since π1 and π2 are not twisted equivalent by any grössencharacter, (4.12) has a double pole at s = 1 and is otherwise analytic. The L-function we are interested in is L∧2 (s) = Lπ1 , π1 (s)Lπ1 , π2 (s)Lπ2 , π1 (s)Lπ2 , π2 (s) ∞ ∞  ¯ π (n2 )  |λπ1 (n2 )|2   λπ1 (n2 )λ 2 = s s n n n=1 n=1 ∞ ∞  ¯ π (n2 )   λπ2 (n2 )λ |λπ2 (n2 )|2  1 × ns ns n=1 n=1

=

∞  λ∧2 (n) ns n=1

(4.13)

for Re s sufficiently large. Here 

λ∧2 (n) =

¯ π (b2 )λπ (c2 )λ ¯ π (c2 )|λπ (d2 )|2 . |λπ1 (a2 )|2 λπ1 (b2 )λ 2 2 1 2

a,b,c,d≥1 abcd=n

By Lemma 3.1, the four L-functions in the product of L∧2 (s) in (4.13) can be factored as Lπi ,πj (s) = L(s, Sym 2 πi × Sym 2 π

j )Ui,j (s)

(4.14)

for i, j = 1, 2, where Ui,j (s) as an Euler product converges absolutely for Re s > 12 . Multiplying (4.14) according to (4.13) we can express L∧2 (s) as the product of (4.12) 2 and i,j=1 Ui,j (s). Then using Landau’s Lemma 2.1 with k = 2 and 2A = 36 and Dirichlet convolution we prove (1.8). 2 Proof of Theorem 1.7. We may write η = ηF/Q as in §1.7 for a Galois number field F of prime degree over Q. From (1.3) we can show that π and π ⊗ ηF/Q are not twisted equivalent by any | det |it , t ∈ R. In fact, if π∼ = π ⊗ ηF/Q ⊗ | det |it , for some t ∈ R× , then π∼ = π ⊗ | det |it , which is impossible. Moreover, j k π ⊗ ηF/Q  π ⊗ ηF/Q ⊗ | det |it

for any t ∈ R when 0 ≤ j < k < .

(4.15)

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

339

By (1.4) we have the Rankin–Selberg L-function

= L(s, Π × Π)

−1  −1 

j k L(s, (π ⊗ ηF/Q ) × (

π ⊗ η¯F/Q )).

(4.16)

j=0 k=0

By (4.15) we can see on the right hand side of (4.16), each diagonal L-function (j = k) has a simple pole at s = 1 and each off-diagonal L-function (j = k) is entire. Consequently,

has a pole of order at s = 1 and is analytic elsewhere. It also has a L(s, Π × Π) standard functional equation. Its complete L-function is bounded in vertical strips and is holomorphic except for poles at s = 0, 1 [8,9,27]. The L-function in (4.16) can be simplified to

= L(s, Π × Π)

−1 

j L(s, π × π

⊗ ηF/Q ) .

(4.17)

j=0

Note this L-function has nonnegative coefficients (cf. Rudnick and Sarnak [32, p. 318])

= L(s, Π × Π)

∞  λΠ×Π (n) n=1

ns

,

λΠ×Π (n) ≥ 0.

It converges absolutely for Re s > 1. Its rank is 4 2 and hence the sum of coefficients of s in Γ functions in its Archimedean factor is 2 2 . By Landau’s Lemma 2.1, 

 42 −1  λΠ×Π (n) = xP−1 (log x) + O x 42 +1 log−1 x ,

(4.18)

n≤x

where P−1 is a polynomial of degree − 1. By (4.17), we may express the Fourier coefficients λΠ×Π (n) in terms of λπ (n). By Goldfeld and Hundley [6], ηF/Q is an idelic lift of a nontrivial Dirichlet character χ mod . Therefore by (4.17) 

λΠ×Π (n) = −1 i=0

−1  −1 

|λπ (aij )|2 χj (aij ).

(4.19)

i=0 j=0 a ≥1 ij −1 j=0 aij =n

Then (1.9) follows from (4.18) and (4.19). 2 5. Proof of Theorem 1.8 Let f be a self-dual holomorphic cusp form for a congruence subgroup of SL2 (Z). To improve the error term bound in (1.5), we factor (3.2) further for m = 2: L2 (s) = ζ(s)L(s, Sym 2 f )L(s, Sym 4 f )U2 (s)

(5.1)

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

340

using the local parameters αp4 , αp2 , αp2 , 1, 1, 1, βp2 , βp2 , βp4 of the self-dual L(s, Sym 2 f × Sym 2 f ). As in the proof of Theorem 1.2 in [18], we use  1+ε+iT  1/2+ε+iT Perron’s formula and shift the contour from 1+ε−iT to 1/2+ε−iT to get  n≤x

1 λ2f (n2 ) = 2πi

1 2 +ε+iT



1 2 +ε−iT

1+ε+iT 

+ 1 2 +ε−iT



+ 1 2 +ε+iT

xs L2 (s) ds s

1+ε−iT

 L (s)xs   x1+ε  2 + Res +O s=1 s T  x1+ε  =: I1 + I2 + I3 + c2 x + O . T

(5.2)

Since U2 (s) converges absolutely for Re s > 1/2, 1

I1 x 2 +ε +x

1 2 +ε

T    dt  1  1  + ε + it, Sym 2 f × Sym 2 f U2 + ε + it  L 2 2 t 1

x

1 2 +ε

+x

1 2 +ε

T    dt  1  + ε + it, Sym 2 f × Sym 2 f  . L 2 t 1

By a dyadic subdivision we have 1

I1 x 2 +ε +x

1 2 +ε

 1 T1   1   + ε + it, Sym 2 f × Sym 2 f dt . (log T ) max L T1 ≤T T1 2 T1 2

By (5.1) and Cauchy’s inequality we have 1

1

I1 x 2 +ε + x 2 +ε (log T ) max

T1 ≤T

×

 1  1   + ε + it  max ζ T1 T21 ≤t≤T1 2

T1   1 2  12   + ε + it, Sym 2 f  dt L 2 T1 2

T1   1 2  12   + ε + it, Sym 4 f  dt . × L 2 T1 2

(5.3)

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

341

Now we need Lemma 2.6 in [18] T   2 m+1   1 + ε + it, Sym m f  dt f,ε T 2 +ε L 2

(5.4)

T 2

and a subconvexity bound for ζ(s) by Bourgain [3]  1 13 + ε + it (1 + |t|) 84 +ε . ζ 2

(5.5)

Applying (5.4) and (5.5) to (5.3) we have I1 x 2 +ε + x 2 +ε T −1+ 84 + 4 + 4 +ε x 2 +ε T 84 +ε . 1

1

13

3

5

1

97

(5.6)

For the integrals I2 and I3 over the horizontal segments, we use (5.5) and the convexity bounds for L(s, Sym 2 f ) and L(s, Sym 4 f ) to get I2 + I3

max

1 2 +ε≤σ≤1+ε

13

3

5

xσ T ( 42 + 2 + 2 )(1−σ)−1+ε

 x 1+ε 181  x  12 +ε 181 42 −1+ε + T T 42 −1+ε 181 181 T 42 T 42 1 97 x1+ε + x 2 +ε T 84 +ε . T

(5.7)

By (5.2), (5.6) and (5.7) we get  n≤x 42

λ2f (n2 ) = c2 x + O

 x1+ε  T

Taking T = x 181 in (5.8), we finally get (1.10).

 1  97 + O x 2 +ε T 84 +ε .

(5.8)

2

Acknowledgments The authors would like to thank an anonymous referee for detailed reading and useful comments. This work was completed when the first named author visited the University of Iowa, supported by the International Cooperation Program sponsored by the Shandong Provincial Education Department. References [1] J. Arthur, L. Clozel, Simple Algebras, Base Change, and the Advanced Theory of the Trace Formula, Annals of Math. Studies, vol. 120, Princeton University Press, Princeton, 1980. [2] M. Asgari, A. Raghuram, A cuspidality criterion for the exterior square transfer of cusp forms on GL(4), in: On Certain L-Functions, in: Clay Math. Proc., vol. 13, Amer. Math. Soc., Providence, 2011, pp. 33–53. [3] J. Bourgain, Decoupling, exponential sums and the Riemann zeta function, arXiv:1408.5794v2, 2/22/2016.

H. Lao et al. / Journal of Number Theory 164 (2016) 323–342

342

[4] P. Deligne, La Conjecture de Weil. I, Publ. Math. Inst. Hautes Études Sci. 43 (1974) 273–307. [5] S. Gelbart, H. Jacquet, A relation between automorphic representations of GL(2) and GL(3), Ann. Sci. Éc. Norm. Supér. 11 (4) (1978) 471–542. [6] D. Goldfeld, J. Hundley, Automorphic Representations and L-Functions for the General Linear Group, vol. 1, Cambridge University Press, Cambridge, 2011. [7] G. Henniart, Sur la fonctorialité, pour GL(4), donnée par le carré extérieur, Mosc. Math. J. 9 (1) (2009) 33–45. [8] H. Jacquet, J.A. Shalika, On Euler products and the classification of automorphic representations. I, Amer. J. Math. 103 (3) (1981) 499–558. [9] H. Jacquet, J.A. Shalika, Rankin–Selberg convolutions: Archimedean theory, in: Israel Math. Conf. Proc., vol. 2, 1990, pp. 125–207. [10] H. Kim, Functoriality for the exterior square of GL4 and the symmetric fourth of GL2 , J. Amer. Math. Soc. 16 (1) (2003) 139–183. [11] H. Kim, Automorphic L-functions, in: Lectures on Automorphic L-Functions, in: Fields Inst. Monogr., vol. 20, Amer. Math. Soc., Providence, 2004, pp. 97–201. [12] H. Kim, P. Sarnak, Refined estimates towards the Ramanujan and Selberg conjectures, appendix to [10]. [13] H. Kim, F. Shahidi, Cuspidality of symmetric powers with applications, Duke Math. J. 122 (1) (2002) 177–197. [14] H. Kim, F. Shahidi, Functorial products for GL2 × GL3 and the symmetric cube for GL2 , Ann. of Math. 155 (3) (2002) 837–893 (with an appendix by C.J. Bushnell and G. Henniart). [15] E. Landau, Über die anzahl der gitterpunkte in gewisser bereichen (zweite abhandlung), Nachrichten Ges. d. Wiss. Göttingen, Math.-Phys. Klasse, 1915, pp. 209–243. [16] R. Langlands, Base Change for GL(2), Annals of Math. Studies, vol. 96, Princeton University Press, Princeton, 1980. [17] H.X. Lao, A. Sankaranarayanan, The average behavior of Fourier coefficients of cusp forms over sparse sequences, Proc. Amer. Math. Soc. 137 (8) (2009) 2557–2565. [18] H.X. Lao, A. Sankaranarayanan, The distribution of Fourier coefficients of cusp forms over sparse sequences, Acta Arith. 163 (2) (2014) 101–110. A. Sankaranarayanan, Integral mean square estimation for the error term related to [19] H.X. 2Lao, λ (n2 ), Sci. China Math. 58 (10) (2015) 2125–2132. nx

[20] Y.-K. Lau, G.S. Lü, Sums of Fourier coefficients of cusp forms, Q. J. Math. 62 (3) (2011) 687–716. [21] Y.-K. Lau, G.S. Lü, J. Wu, Integral power sums of Hecke eigenvalues, Acta Arith. 150 (2) (2011) 193–207. [22] X.Q. Li, Bounds for GL(3) × GL(2) L-functions and GL(3) L-functions, Ann. of Math. 173 (1) (2011) 301–336. [23] X.Q. Li, M. Young, Additive twists of Fourier coefficients of symmetric-square lifts, J. Number Theory 132 (7) (2012) 1626–1640. [24] G.S. Lü, The sixth and eighth moments of Fourier coefficients of cusp forms, J. Number Theory 129 (11) (2009) 2790–2800. [25] G.S. Lü, On an open problem of Sankaranarayanan, Sci. China Math. 53 (5) (2010) 1319–1324. [26] G.S. Lü, On higher moments of Fourier coefficients of holomorphic cusp forms, Canad. J. Math. 63 (3) (2011) 634–647. [27] C. Mœglin, J.-L. Waldspurger, Le spectre résiduel de GL(n), Ann. Sci. Éc. Norm. Supér. 22 (4) (1989) 605–674. [28] M. Ram Murty, On the estimation of eigenvalues of Hecke operators, Rocky Mountain J. Math. 15 (2) (1985) 521–533. [29] D. Ramakrishnan, Modularity of the Rankin–Selberg L-series, and multiplicity one for SL(2), Ann. of Math. 152 (1) (2000) 45–111. [30] D. Ramakrishnan, S. Wang, A cuspidality criterion for the functorial product on GL(2) × GL(3) with a cohomological application, Int. Math. Res. Not. IMRN 2004 (27) (2004) 1355–1394. [31] R.A. Rankin, Contributions to the theory of Ramanujan’s function τ (n) and similar arithmetical functions II. The order of the Fourier coefficients of integral modular forms, Proc. Cambridge Philos. Soc. 35 (2) (1939) 357–372. [32] Z. Rudnick, P. Sarnak, Zeros of principal L-functions and random matrix theory, Duke Math. J. 81 (2) (1996) 269–322. [33] A. Selberg, Bemerkungen über eine Dirichletsche Reihe, die mit der Theorie der Modulformen nahe verbunden ist, Arch. Math. Naturvidensk. 43 (1940) 47–50.