A search for exclusive penguin decays of B mesons

Volume 223, number 3,4

PHYSICS LETTERS B

15 June 1989

A SEARCH FOR EXCLUSIVE PENGUIN DECAYS OF B MESONS

P. AVERY, D. BESSON, L. GARREN, J. YELTON University of Florida, Gainesville, FL 32611, USA

T. BOWCOCK, K. KINOSHITA, F.M. PIPKIN, M. PROCARIO, Richard WILSON, J. WOLINSKI, D. XIAO Harvard University, Cambridge, MA 02138, USA

P. BARINGER, P. HAAS, Ha LAM University of Kansas, Lawrence, KS 66045, USA

A. JAWAHERY, C.H. PARK University of Maryland, College Park, MD 20742, USA

D. PERTICONE, R. POLING University of Minnesota, Minneapolis, MN 55455, USA

R. FULTON, M. HEMPSTEAD, T. JENSEN, D.R. JOHNSON, H. KAGAN, R. KASS, F. MORROW, J. WHITMORE Ohio State University, Columbus, OH 43210, USA

W.-Y. CHEN, J. DOMINICK, R.L. McILWAIN, D.H. MILLER, C.R. NG, E.I. SHIBATA, W.-M. YAO Purdue University, West Lafayette, IN 47907, USA

E.H. THORNDIKE University of Rochester, Rochester, N Y 14627, USA

M.S. ALAM, N. KATAYAMA, I.J. KIM, W.C. LI, X.C. LOU, C.R. SUN State University of New York at Albany, Albany, NY 12222, USA

D. BORTOLETTO, M. GOLDBERG, N. HORWITZ, M.D. MESTAYER, G.C. MONETI, V. SHARMA, I.P.J. SHIPSEY, T. SKWARNICKI Syracuse University, Syracuse, N Y 13210, USA

S.E. CSORNA, T. LETSON Vanderbilt University, Nashville, TN 37235, USA

I.C. BROCK, T. FERGUSON Carnegie Mellon University, Pittsburgh, PA 15213, USA

470

0370-2693/89/$ 03.50 © Elsevier Science Publishers B.V. (North-Holland Physics Publishing Division)

Volume 223, number 3,4

PHYSICSLETTERSB

15 June 1989

M. ARTUSO, C. BEBEK, J. BYRD, K. BERKELMAN, D.G. CASSEL, E. CHEU, D.M. COFFMAN, G. CRAWFORD, J.W. DeWIRE, P.S. DRELL, R. EHRLICH, R.S. GALIK, B. GITTELMAN, S.W. GRAY, A.M. HALLING, D.L. HARTILL, B.K. HELTSLEY, J. KANDASWAMY, R. KOWALEWSKI, D.L. K_REINICK, Y. KUBOTA, J.D. LEWIS, N.B. MISTRY, J. MUELLER, R. NAMJOSHI, S. NANDI, E. NORDBERG, C. O'GRADY, D. PETERSON, M. PISHARODY, D. RILEY, M. SAPPER, A. SILVERMAN, S. STONE, H. WORDEN, M. WORRIS Cornell University, Ithaca, NY 14853, USA

and A.J. SADOFF Ithaca College, Ithaca, NY 14850, USA

Received 10 March 1989

We have measured upper limits on branchingfractionsfor rare exclusivedecaysof B mesonsarising fromone-loopdiagrams in the standard modelof electroweak interactions. We also obtain an upper limit for the lepton-number-violatingdecayB°~ ~t-+e+.

In the standard model of electroweak interactions, flavor-changing neutral currents are forbidden at the tree level but may occur at the one-loop level through so-called penguin diagrams. Examples of these diagrams for B decay are illustrated in fig. 1 [ 1 ]. Decays of this type can exhibit direct C P violation and may be responsible for a non-zero value of E' [2 ] in the kaon system. Final states in K decay, however, can come from several tree diagrams making it difficult to identify the penguin contribution. The large longdistance QCD effects in K decay further obscure the issue. In B decay, penguin diagrams give rise to a b ~ s transition, which can yield exclusive ~t B meson decays which do not occur through any other decay mechanism, thus allowing the existence of one-loop terms to be established. An example is B°-+ CK°, which occurs via the penguin mechanism as shown in fig.

2a (charge-conjugate modes are always implied throughout this paper). There are also exclusive modes which can occur through either the penguin or spectator diagrams, but which are suppressed in the case of the spectator mechanism. An example is

-y(,+ W

B* d

B

~t In inclusive studies one must be able to distinguish the direct b--,s transition from the dominant decaychain b--,c--,s. W

!

~

d

K°

~_

d

7r-

W

Bo d

b

~

(a)

~

u]

(b)

+Y:

s

K+ -

B* d q

q

Fig. 1. Penguin diagram contributions to B decay.

d

Fig. 2. (a) The penguin diagram for the decayB ° ~ K °. (b) The penguin diagram and Cabibbo-suppressed spectator diagram for the decayB°--,K+n-. 471

Volume 223, number 3,4

PHYSICSLETTERSB

B°-.K+n -, which may occur as shown in fig. 2b. The spectator contribution in this case is doubly suppressed by the small values of the KobayashiMaskawa mixing matrix elements V,b and F,~. Many of the exclusive charmless decay channels are predicted to have large CP-violating asymmetries [ 3,4 ]; thus it is also important to measure these decay rates in order to determine the feasibility of using them to study CP violation in the B system. In this paper we present results of a search for exclusive B decays involving b ~ s transitions. We also report on two-body leptonic decays of the B meson. Where possible we compare our limits with various theoretical predictions. The data sample used in this study was collected with the CLEO detector at the Cornell Electron Storage Ring (CESR). It consists of 212 pb -1 at the Y(4S) resonance, yielding 240000 BI) events of which we assume 43% are neutral [ 5 ], and 102 p b at energies below BB threshold. The CLEO detector and our hadronic event selection criteria are described in detail elsewhere [6 ]. Here we will briefly describe the central tracking system. Charged particles are tracked inside a superconducting solenoid of radius 1.0 m which produces a 1.0 T magnetic field. Three nested cylindrical drift chambers measure momenta and specific ionization ( d E / d x ) for charged particles. The innermost part of the tracking system is a three-layer straw tube vertex detector which has a position accuracy of 70 pm in the r-O plane. The middle ten-layer vertex chamber measures position with an accuracy of 90 ~tm in the r-O plane and d E / dx to 14%. The main drift chamber [ 7 ] contains 51 layers of wires, eleven of which are stereo layers with angles of 1.9°-3.5 ° to the z axis. This device provides an r-~ position accuracy of 110 Jam and d E / d x to 6.5% and replaces the previous drift chamber which contained only 17 layers. The track coordinates along the beam axis (z) are measured using the stereo layers and cathode strips in the middle vertex detector and the main drift chamber. The momentum resolution achieved by this system is (Sp/p)2= (0.23°/op) 2 + (0.7%) 2, where p is in GeV/c. The new drift chamber, plus the size of the current data set, allow us to greatly improve on our previous limits [ 8 ]. We search for candidate events in two- and threebody decay modes of B mesons without charm but with a strange hadron (see table 1 ). Each track used 472

15 June 1989

is required to have d E / d x in the drift chamber within two standard deviations of that expected for its particle assignment. Muon candidates are required to penetrate the iron absorber. We detect I ~ n + n - and A - , p n - by combining pairs of tracks coming from a secondary vertex separated from the main vertex by at least 5 mm in the r-q~ plane and having an invariant mass within 20 MeV (6 MeV) of the mass of the I ~ (A), where the mass resolution is approximately 4 MeV ( 1.5 MeV). Candidates for resonance decays are selected by finding combinations of tracks with an invariant mass within 0.75Fofthe resonance mass [12] for p°--,n+n-, f o ~ n + n -, K * ° ~ K + n -, and K*+--,I~n + (where /" is the natural resonance width), and within 6 MeV for ¢~--,K+K-. For B decay modes resulting in a final state of a K - plus two or three pions, the K - n + ( K - n + n + ) invariant mass is required to be more than two standard deviations, 25 (20) MeV, away from the D O (D +) mass. For e+e - o B I ] events each B meson will have half of the center-of-mass energy (Eo=Ec~/2), which is equal to the single beam energy ~2, therefore we require that the measured sum of the energies of the candidate B decay products (Etot) be within two standard deviations of the beam energy. The expected resolution of this energy sum is determined by Monte Carlo simulation of the decays in the CLEO detector. It ranges from 25 MeV to 37 MeV for most of the modes in question and is 300 MeV for the K*7 modes. Much of the background in our search comes from continuum events which are also produced at the "~(4S) energy. To suppress these, we compare the sphericity axis of the charged tracks in the event, excluding the tracks from the B candidate decay, with the sphericity axis found using just the candidate tracks. For BI] events these two axes are uncorrelated since the two B mesons are essentially at rest and decay independently. For continuum events, however, there is a strong correlation as a result of the jet-like character of the event. We require Icos0sl <0.7, where 0s is the angle between the two axes. This rejects almost 90% of the continuum background. In addition, since for B's produced in the process e+e - ~ Y ( 4 S ) ~BB, the angle (0B) of the direction of ~z Althoughthe average value of Eo is equal to the beam energy, the spread in Eo is 2.1 MeV while the spread in beam energy is 3.3 MeV; see ref. [ 13].

Volume 223, number 3,4

PHYSICS LETTERS B

15 June 1989

Table 1 A summary of our search for the indicated exclusive decay modes of B mesons plus a comparison with the theoretical predictions from Chau and Cheng (CC) [3], Gavela et al. (GYO) [9], Deshpande et al. (DLT) [ 10], and Deshpande and Trampetic (DT) [ 11 ]. The 90% confidence level upper limit on the number of events is the sum of the mode indicated and its charge conjugate. The 90% confidencelevel upper-limit branching fraction applies to the mode given or its charge conjugate but not the sum. Decay mode

Detection efficiency

Upper limits (90% CL)

Theory

Number of events

Branching ratio

Prediction

I X 1 0 -4 ]

[ X 1 0 -4 ]

1.0 0.7 0.04

CC GYO CC

1.0 0.5 0.4

CC GYO CC

0.5

CC

0.6

CC

0,1

CC

0,5 0,4

CC CC

0.2 0.2 0.006 0.006 0.02 0.02

DLT DLT DT DT DT DT

l]°--,K-n ÷

0.45

8.9

0.9

l]°~I(°p ° I)°~ I(°f0 l]°~K*-n + B°~I~%

0.045 0.027 0.026 0.023

5.3 2.3 2.3 2.3

5.8 4.2 4.4 4.9

B°~K*°9° B°~K-~°fo B°~K*° 0

0.087 0.058 0.044

11.8 2.3 3.9

6.7 2.0 4.4

B- ~I~°n B---,K-n-n~o charm B-~K-p ° B-~K-fo B- ~K*°n B---,K-~

0.10 0.32 0,22 0,13 0,15 0.11

2.3 14.4 4.1 2.3 5.3 2.3

0.9 1.7 0.7 0.7 1.3 0.8

B---*AI5

0.19

2.3

0.5

l]°--'K*°7 B- ~ K*-~/ B-~K-e+e B-~K-~t+ti I]°--*K*°e+ e B°~K*°p+B-

0.090 0.015 0.30 0.21 0.15 0.10

5.3 2.3 4.1 8.3 2.3 3.9

2.8 5.5 0.5 1.5 0.8 1.9

B°~e+e l]°~p-+~t13°--}B±e:~

0.45 0.26 0.31

2.3 2.3 2.3

0.3 0.5 0.4

the B w i t h respect to t h e b e a m axis is d i s t r i b u t e d as sin20a, we r e q u i r e I cos 0al < 0.8. T h i s retains a l m o s t 95% o f real B's while rejecting 20% o f r a n d o m c o m binations, which have a uniform distribution. F o r d e c a y m o d e s o f the B i n t o a v e c t o r particle (~, K* a n d p) a n d a p s e u d o s c a l a r particle, we i m p o s e an a d d i t i o n a l criterion. D e f i n i n g 0* as the angle o f a track f r o m the v e c t o r m e s o n decay ( i n the v e c t o r m e s o n rest f r a m e ) w i t h respect to the d i r e c t i o n o f the v e c t o r m e s o n in the B rest f r a m e , we r e q u i r e [cos 0"1 > 0.5. Since any signal will h a v e a cos20 * d i s t r i b u t i o n a n d b a c k g r o u n d processes are o b s e r v e d to h a v e a uni-

Author(s)

f o r m distribution, i m p o s i n g this criterion reduces the b a c k g r o u n d . Similarly, for decays i n t o K*~/, t h e K* decays w i t h a sin20 * d i s t r i b u t i o n , a n d we r e q u i r e I cos 0"[ < 0.8 for t h o s e m o d e s . F o r each s u r v i v i n g candidate, we c o m p u t e the b e a m 2 2 1/2, c o n s t r a i n e d m a s s M = ( E o --Ptotal ) w h e r e Ptotal is the v e c t o r s u m o f t h e m o m e n t a o f t h e t r a c k c a n d i dates. Since o u r r e s o l u t i o n in Eo is m u c h b e t t e r t h a n o u r r e s o l u t i o n in Etot, the use o f Eo r a t h e r t h a n t h e m e a s u r e d energy substantially i m p r o v e s t h e resolut i o n in M. F o r c h a r g e d ( n e u t r a l ) B - m e s o n d e c a y m o d e s o n e expects a signal c e n t e r e d a b o u t M = 5.279 473

Volume 223, number 3,4

8"0[' 6.0!

~.o 0.o

I''

' I ' ' p°K' ~ ......

2.0

..q

PHYSICS LETTERS B

p

17~l~V] . . . . . .

I (o' 1

.

nn

nl

.'

>=

4.0

LO 2.0 r,i

o.o

, In

n,r-I,

.nn

nn,

,.

~n c o >

46 .. 00

2.o

~ ~

0.0

I

'

'

6.0

-

K* ° -

~n '

'

I(

n I

'

~,fl '

I

'

'

D°Tr -

''

i]

~

I (d

4.0

2.0 0.0

I 5.21

,

,FI 5.23

,R , 5.25

I 5.27

I 5.29

Beam Constrained Mass (GeV) Fig. 3. The mass distributions for B - - * K - n + n - which satisfy

invariant mass cuts for (a) p°K-, (b) foK-, (c) K*°n-, and (d) D0~-. (5.281) GeV [5], with an RMS resolution, determined from Monte Carlo studies, ranging from 2.5 to 3.0 MeV depending on the decay mode. No statistically significant signal is observed in any non-charm decay mode. As one example, the distribution for the two-body components of the K - n + n - mode are shown in fig. 3. There is no evidence for an enhancement in p°K-, foK-, or K*°n -. There is, however, a clear peak for the charmed mode D°n-. To determine a conservative upper limit, we subtract the continuum background, scaling it to account for the integrated luminosity and the energy dependence of the continuum cross-section. We then sum the number of events within + 5 MeV of the expected value a r M in the subtracted distribution. The numbers of events corresponding to 90% confidence level upper limits are given in table 1. We search for events consistent with the B--,K*7 hypothesis in a similar way. However, to calculate the B mass we proceed in a somewhat different manner. Since the photon energy resolution (aE= 21%,v/~) is poor, while the angular resolution (6 mrad) is quite good, we use E ~ , = E o - E K . instead of the measured photon energy to calculate py and the measured position of the candidate photons to determine the direction. For modes containing a lepton pair and a strange meson, we follow the same event-selection procedure described above, except that for muon pairs we re474

15 June 1989

quire only that the higher-momentum one be identified. Since this yields an efficiency that is constant over all areas of phase space, the upper limits for these modes are insensitive to any uncertainties in the theoretical models for the distribution of these events in phase space. We also exclude any candidate where the invariant mass of the lepton pair is within 50 MeV of the ~ mass, where the mass resolution is 25 MeV. To compute upper limits for the branching fractions, we estimate our detection efficiencies using a Monte Carlo simulation of the response of the CLEO detector to a BI3 event in which one of the B's decays into the mode of interest. For the modes containing a lepton pair and a kaon, the events were generated assuming that the distribution is uniform in phase space. The efficiencies are tabulated in table 1 and they include all corrections for unobserved decay modes of the secondary resonances and neutral kaon. Using these efficiencies, we obtain 90% confidence level upper limits to the branching fractions given in the fourth column of table 1. Using our calculated efficiency for B ~ D ° n -, along with our previously measured branching ratio [5], we expect to see 14.1 + 4.8 events within 5 MeV of the B mass for this mode. We observe 14 events. Theoretical predictions of branching fractions have been made for a few of these modes. These are compared with our experimental results in the last columns of table 1. As noted by Gavela et al. [9 ], one expects a suppression of the penguin mode B + ~ ~K ÷ ( B ° ~ g K *°) relative to the mode B+--,~cK + ( B ° ~ c K *°) by one to two orders of magnitude. The branching fraction for B+--,~K ÷ (B°--,~K *°) has been measured [12] to be ( 0 . 8 _ 0 . 2 8 ) X 1 0 -3 ( ( 3.7 + 1.3 ) X 10- 3). Our upper limits are consistent with this expected suppression. Predictions for K*7 depend on the top quark mass [ 10,14], but unless this exclusive mode is a large fraction of the inclusive b--,s7 rate [15] our limit cannot place any constraints on the top quark mass. It has also been suggested [ 1 I, 16 ] that couplings to a fourth generation of quarks could significantly enhance the rate for b - , s. We see no evidence for such an enhancement. The neutral B meson can also decay via loop diagrams into an e r e - or ~t+~t- pair. The branching fractions are predicted to be very small by the standard model, about 10- ~2 and 10- s, respectively [ 17 ]. We search for these modes along with the decay

Volume 223, number 3,4

PHYSICS LETTERSB

B ° ~ ~t-+e :~, which is forbidden in the model by lepton n u m b e r conservation. No events are observed and the 90% confidence level upper limits for the branching ratio into these three modes are given in table 1. I n conclusion, we have searched for several rare decay modes of B mesons and in particular those that could result from penguin-induced decay mechanisms. We find no evidence for tlaese decays. We are grateful for the excellent efforts of the CESR staff which made this work possible. R. Kass thanks the OJI program of the DOE, P.S.D. thanks the PYI program of the NSF, a n d R.P. thanks the A.P. Sloan F o u n d a t i o n for support. This work was supported by the National Science F o u n d a t i o n a n d the US Departm e n t of Energy u n d e r Contract Nos. DE-AC0276ER01428, DE-AC02-76ER03066, DE-AC0276ER03064, DE-AC02-76ER01545, DE-AC0276ER05001, a n d FG05-86-ER40272. The Cornell National Supercomputing Facility, funded in part by the NSF, N Y state and IBM, was used in this research.

References

15 June 1989

[2] H. Burkhart et al., Phys. Lett. B 206 (1988) 169. [ 3 ] L.-L. Chau and H.Y. Cheng,Phys. Rev. Lett. 59 ( 1987) 958. [4] D. Du, Phys. Rev. D 35 (1987) 902. [ 5 ] C. Bebek et al., Phys. Rev. D 36 ( 1987) 1289. [6] D. Andrewset al., Nucl. Instrum. Methods 211 (1983) 47; S. Behrends et al., Phys. Rev. D 31 ( 1985 ) 2161. [7] D.G. Cassel et al., Nucl. Instrum. Methods A 252 (1986) 325. [8] P. Averyet al., Phys. Lett. B 183 (1987) 429. [9] M.B. Gavela et al., Phys. Lett. B 154 (1985) 425. [ 10] N.G. Deshpande et al., Z. Phys. C 40 (1988) 369. [ 11 ] N.G. Deshpande and J. Trampetic, Phys. Rev. Lett. 60 (1988) 2583. [ 12] Particle Data Group, G.P. Yost et al., Review of particle propertiesl Phys. Len. B 204 (1988) 1. [ 13 ] D.G. Casselet al., Proc. 1988 SummerStudyon Highenergy physics in the 1990's (Snowmass,CO, 1988). [ 14] S. Bertolini, F. Borzumati and A. Masiero, Phys. Rev. Lett. 59 (1987) 180. [ 15 ] T. Altomari, Phys. Rev. D 37 ( 1988 ) 677; C.A. Dominquez et al., preprint DESY 88-110 (1988). [16] W.-S. Hou, R.S. Willey and A. Soni, Phys. Rev. Lett. 58 (1987) 1608; W.-S.Hou, A. Soni and H. Steger, Phys. Rev. Lett. 59 (1987) 1521. [ 17] Report of the Working Group on CP violation and rare decays, Proc. 1984 SnowmassSummer Study on the Design and utilization of the Superconducting Super Collider (1984).

[ 1] See e.g.M. Wirbel, Prog. Part. Nucl. Phys. 21 (1987) 33, and referencestherein.

475