Stationarity test for the beta binomial model

Stationarity Test for the Beta Binomial Model Darius J. Sabavala, Cornell Univer.sir;r Donald G. Morrison, Columbia Universio

The beta binomial model has beer? successfully applied to describe brand switching, television viewing, and magazine readership behavior. However, long-term projections fvom this model tend to have systematic biases. These biases can be explained by violations of the stationarity assumption of the model. A statistical test for stationarity is presented. A transition matrix upon which the test is based provides a useful diagrlostic tool for detecting shifts in preference structures versus equilibrium switching behavior.

Stochastic models have been widely applied to describe choice behavior in repetitive choice situations. For many marketing phenomena, particularly brand switching, magazine readership, and television viewing, the beta binomial has proved to be a good descriptive and predictive model over relatively short time periods [ 1, 4, 5, 81. However, for longer-term predictions, the model does not work well and the predictions show a systematic error. Schreiber [9] demonstrates this fact with some examples and provides an explanation for the biased predictions. Basically, Schreiber argues that the stationarity assumption may not be valid over the longer period. A simple method for testing the stationarity assumption is presented here. The transition matrix approach that is used provides a visual diagnostic for detecting the nature and the extent of nonstationarity. While the probabilistic modeling and statistical tests are of interest per se. this method of analysis also has strong managerial implications. By comparing the observed transition matrix with the expected (under the assumption of stationarity) transition matrix, the manager will be able to distinguish between equilibrium brand to brand switching behavior and switching due to long-term changes in underlying preferences. This ability will be particularly important in assessing the effects of disruptions in the marketplace, e.g., entry of a new brand or television show, periods of excessive pricing or advertising activity, or changes in the positioning of a brand or product. The approach is useful in situations where the aggregate model and summary statistics appear not to have changed. It is managerially Address correspondence to: Darius Jai Sabavala, Graduate School of’ Business and Public Administration, Malott Hall, Cornell University, Ithaca, NY 14853. JOURNAL OFBUSINESS 0 Elsevier North Holland, 52 Vanderbilt

RESEARCH Inc., 1981

9, 221-230

Ave.. New York, NY 10017

(1981) 221 0148-2963/81/02221-10$2.50

222

Darius J. Sabavala and Donald G. Morrison

significant to know whether individual parameters have remained constant or have changed in such a way that the aggregate has remained constant. If individual parameters have changed then this would imply a different audience for the television viewing and magazine readership applications, and would imply a different customer segment in brand choice applications. The Beta Binomial Model On each trial the consumer makes a binary choice, e.g., chooses a brand of interest (choice “1”) or some other brand (choice “O”), views a particular TV show or does not view that show, sees an advertisement in a particular issue of a magazine or does not see the advertisement, etc. Stationarity Assumption On each trial the consumer’s probability is p for binary choice “ 1” and 1 - p for binary choice “0”. This consumer’s p value remains constant over time and each trial is an independent event. Beta Assumption Different consumers have different p values. These p values are distributed beta across the population of consumers. Ekpressing this mathematically we have lym + n) f@)

=

Um)r(n)

’

W-1(1

-p)n-l,

forO
1, (1)

where m, n > 0, and f’ = gamma function. If a consumer makes r binary decisions, then by the stationarity assumption the number of l’s for this consumer is a binomial random variable with parameters r and p. The probability that a randomly chosen individual, whose p value is not known, makes k l’s out of a total of r binary decisions has a beta binomial distribution [8]. Namely,

P(k; m, n,r)

=

B(m+k,n+r-k)

’

Bh,n)

fork

=O, l,..., Y,

where B(m, n) =

WW(n) r(m

+ n)’

This may also be interpreted as the proportion population with k 1‘s out of r trials.

of consumers

in the

Beta Binomial Model

223

A word of caution is now appropriate. The remainder of the paper addresses the issue of the long-run stationarity of individual p values. It is assumed that over a short period the beta binomial is a valid model. That is, individuals follow zero-order processes with constant p values and that the p values are distributed beta across the population. We will mention these assumptions again when the empirical results are presented. A Two-Period Transition Matrix Suppose a string of binary choice decisions is divided into two periods of lengths r and s. These two periods can not be overlapping, but they need not be adjacent. For example in the empirical work for this paper r = four TV viewing occasions in 197 1 and s = four TV viewing occasions in 1973. The individuals in the study can be segmented into r + 1 categories based on their number of “ 1” decisions in period 1, since i, the number of 1 decisions, can take on values 0, 1,2, . . . , r. If the beta binomial model is correct what can be said about the number of 1 decisions in period 2 that will be made by an individual who made i 1 decisions in period l? Our prior distribution on this individual’s p value before we observed the period 1 behavior was beta (m, n). However after observing i l’s in r decisions our updated posterior distribution is beta (m + i, II + r - i). Hence, the probability that this individual will makej 1 decisions in period 2 of length s is beta binomial with parameters m + i and n + r - i. Namely, P(i; m + i, n + r-

i, s) =

s 0i

B(m+i+j,n+rTi-t-S-j) B(m + i> n + Y - i)

’

for i = 0, 1, 2, ***,s. (3) The mathematical results used in deriving our “expected” model can be found in Massy , Montgomery, and Morrison [6, Chap. 31, Feller [3], Raiffa and Schlaifer [7], Winkler and Hays [IO], or virtually any text that discusses Bayesian updating (or inference) on Bernoulli processes. Since these details are undoubtedly familiar to most readers we have omitted any explicit derivations. Each row of the transition matrix (i.e., the probability of making j 1 decisions in period 2 given that i 1 decisions were made in period 1) is a beta binomial probability distribution defined by (3). This transition

224

Darius J. Sabavala and Donald G. Morrison

matrix can be pictured as below. “BB” indicates a beta binomial distribution. This matrix will be called the probability transition matrix.

period 2

0

0

1

s

i

2

1

BB(m, n + r)

1

BB(m+l,ntr-1)

?

BB(m + 2,n +r-2)

1

I

period 1 , I

i

BB(m+i, i

1 I I

n+r-i)

I

I I

I

r

BB(m + r, n)

To calculate the expected number of individuals in each cell of the (r + 1) x (s + 1) matrix, multiply each of the row probabilities by the number of individuals in that row. That is, letting n, be the total number of individuals in row i the expected number of individuals in cell (i, ~1is

Eii = ni

s

0 i

B(m+i+j,n+r-ii+--_) B(m +i,n+r-ii)

for i = 0, 1, 2, .*-, r,

’

(4)

and j = 0, 1, 2, *a-,s. The matrix of Eij’s will be called the expected transition matrix. The matrix of Oij’s, the observed numbers in each cell, will be called the observed transition matrix. Testing the Beta Binomial Model for Stationarity Each row of the observed transition matrix will be an empirical histogram with s + 1 cells. Since the parameters for the beta binomial model will be estimated from period 1 and the behavior is in period 2, no degrees of freedom are lost due to parameter estimation. In fact under the

Beta Binomial Model

225

null hypothesis of a beta binomial model the statistic for the ith row, xi2

=

i

cEij

i,oij)’

j=o

,

11

will have a chi-square distribution with s degrees of freedom. The fact that xi? is (asymptotically) distributed as chi-square is shown in Cramer [2]. There are s + 1 cells but one degree of freedom is lost due to matching on the total number of individuals in row i. Since the rows of the matrix are independent, the statistic for the whole matrix is x2

=

xo2

+

x1

2

+

... +

,,,2

=

i i=O

2 (Eij-/‘i): j=O

(6)

‘I

which will have a chi-square distribution with (r + 1)s degrees of freedom under the null hypothesis. Note that in actual applications there may be small cell frequencies so that in aggregating a group of these, additional degrees of freedom will be lost. Let us summarize our test procedure in words. Assume a sample of consumers in two (nonoverlapping but not necessarily adjacent) periods. The observations consist of the actual decisions made on r binary decisions in period 1 and s binary decisions in period 2. Assume that the beta binomial model is valid in the short run, that is, within each of the periods (an assumption which may easily be tested in practice). Based on the data of period 1, we may make predictions of period 2 behavior, assuming stationarity from period 1 to period 2. The test examines the discrepancy of observed frequencies from these predictions and a large significant discrepancy indicates nonstationarity. As mentioned below, the sources of the discrepancy can be detected and will provide some managerial implications. Empirical Examples The test described above was applied to the problem of estimating the frequency of television viewing. A sample of 504 female heads of households were asked, “How many of the last four episodes of did you view?” This question was asked in 1971 and in 1973 to the same individuals. Of the total of 78 prime time shows in 197 1 and 71 prime time shows in 1973, 33 shows were on in both years. A more detailed explanation of the data plus empirical results for virtually all of

226

Darius J. Sabavala and Donald G. Morrison

the shows can be found in Sabavala and Morrison [8]. (The data were generously provided by K. Britney of Merrill, Lynch, Pierce, Fenner and Smith, Inc.) Of these 33 shows, 3 provided ideal settings to test the stationarity assumption. That is, 1) in each period the beta binomial model fit the viewing pattern very well; and 2) the parameters of the model were almost identical in both periods. The observed and expected frequency distributions for the shows based on a separate period-by-period analysis are shown in Table 1. The ti and ri parameter values shown were estimated using the maximum likelihood method; 7 is the average frequency of viewing; and the x2 value is the goodness-of-fit statistic comparing the observed and expected frequency distributions. First, the fit in each period is good. Of the six cases, thep level at which the x2 is significant is greater than 0.10 in five cases and 0.05 > p > 0.01 in the other. Note that there are two degrees of freedom since for each five-cell histogram we need to estimate two parameters and match the total. Second, there is considerable stability in the parameters (fi, ii) and hence in the average viewing frequency (X). Different stochastic models often fit aggregate data equally well and the power to discriminate between these models often depends upon tests of the components, assumptions, or implications of the models. In our case, it might be appropriate to test the zero-order assumption (that is, outcomes on a given occasion do not depend upon the outcomes on any other occasion). These tests have been developed in [6, Chap. 31. However, since the particular sequence of the outcomes (e.g. 1010, 1100, etc.) is not known, these tests could not be applied. There has been some published research supporting the zero-order assumption for television viewing and brand switching. (Additional support is provided by extensive unpublished studies of media exposure by Jerome D. Greene of Marketmath, Incorporated.) Using the methodology of the previous section the expected transitions were calculated based on m and n estimates from period 1 (1971) data. These matrices are shown in Table 2, along with the x2 statistic and initial values. Note that small expected transition cells have been aggregated causing the degrees of freedom to be less than the full (r + 1)s = 5 X 4 = 20. All three shows fail the statistical test for stationarity . This failure to pass the stationarity test is not surprising given the 2-year hiatus between viewing periods. However, a systematic bias may

221

Beta Binomial Model

Table 1: Separate

Period Analyses Number of Last Four Episodes Viewed 1971

~_.. Show

1973

0

1

2

3

4 -

0

1

2

3

4

227 230

108 97

67 69

47 57

5.5 SO

241 243

97 88

55 63

54 54

57 56

Hawaii Five-O Observed Expected

& = .347, ii = .819 X = 1.185 x2 = 2.00

& = .417, ri = ,964 X = 1.196 x2 = 3.56 Mannix Observed Expected

233 235

97 93 i

73 67

43 56

58 53

215 218

107 98

77 62

43 61

62 55

GI = .444, Pi= .934 55= 1.266 x2 = 7.06

= .386, ri = .893 X = 1.198 x2 = 4.26

Owen Marshall Observed Expected

327 327

74 74

52 45

23 33

1.051 &=.23O,ri= Sz= 0.712 x2 = 4.28

28 25

320 320

84 82

47 48

31 32

22 21

r;z = .272, r; = 1.245 Y= 0.712 x2 = 0.17

x2.1o(2) = 4.6 1. x2 &=y = 5.99. x2:#) = 9.21.

be detected. Letting a “ + ” indicate more observed than expected transitions and a “- ” indicate fewer than expected transitions, the pattern for all three shows is found to be

‘i 1

8 0

d

Y

3 3

14 1

6 6

10 12

cl21 29

9 14

13 14

11 15

12 9

_ x2 (14) = 91.17 ~2.~~ (14) = 29.14

6 I

8 12

7 7

11

9 3

13 21

13 18

32 34

e,

13 18

Gf

44 42

;;

55

47

67

108

‘,“:,”

I

6 6 9 2

14 2 6 1 11 7

6 11

12 20

12 18

2

11 16

3 13

9 15

10 8

3

x2 (14) = 207.42 x2.01 (14) = 29.14

23 19

32 30

1

5 12

36 38

0

cl21 32

14 11

9 7

7 2

4

1973 Viewing Occasions

Show: Mannix

58

43

73

97

Total

9 0

10 1

10

iT

40 32

1 8

4 7

7 10

5 5

x2 (13) = 254.65 x2.01 (13) = 27.69

8 4

5 6

3 4

8 2

5 14

7 12

12 14

18 23

2 cl14

1 5

6 4

4 1

Show: Owen Marshall 1973 Viewing Occasions

Note: For all three transition matrices the observed values are the top figures and the expected values are the bottom figures in each cell.

h

7 3

Matrices

Show: Hawaii Five-O 1973 Viewing Occasions

Table 2: Transition

28

23;

52

14 2.

5 h

2

P

rl

3

S

2 x

5

t

b 9

Beta Binomial Model

229

Although less striking it is also interesting to note that of the 15 observed diagonal transitions (5 for each matrix) 14 are less than expected. The results are clear: With respect to a stationary beta binomial model, too few individuals are viewing the same number of episodes in 1973 as they did in 197 1, while too many viewers are making big jumps (both up and down) in the number of viewing occasions. In particular the number of 197 1 O’s becoming 1973 4’s and the number of 197 14’s that become 1973 O’s is much too high. All of these deviations can be explained by changes in individual p values over the 2-year interval between the two sets of viewing occasions.

Summary This paper has presented an approach based on a two-period transition matrix to test the stationarity assumption for the beta binomial model. This test is based on the conjugate property of the beta and binomial distributions plus a Bayesian updating based on the observed behavior in period 1 to predict period 2 behavior. This rather simple notion has not appeared in the literature although we have learned that Jerome D. Greene of Marketmath, Incorporated, and Robert J. Schreiber of Time, Incorporated, have used similar ideas in some of their unpublished analyses. The empirical data used were ideal for illustrating the methodology. In the short run (four consecutive weekly viewing occasions) the beta binomial provides an excellent fit to the data. Examination of the aggregate model and the frequency distribution shows considerable stability over the 2-year period. However, our test statistically verified that individual p values must have changed. By comparing the expected and observed transition frequencies, the nature and degree to which preferences did shift can be learned. It should be noted that a rather technical definition of a change in preference structure has been used above. In fact, a change in an individual’s p value is taken to indicate a change in preference. Clearly exogenous factors such as scheduling changes for TV shows and out of stock conditions for a brand will change p values even though the individual may still like the TV show or brand just as much as before. Therefore the reader should not get the impression that we can really measure changes in “true” preferences by our transition matrix approach. However, changes can be detected due to shifting p values versus the inherently lower level of switching that will occur in a stationary environment. The ability to determine the extent of nonstationarity has important practical implications even if it is not possible

230

Darius J. Sabavala and Donald G. Morrison

to isolate preference changes in a more adequate behaviorial definition of the word “preference. ” In particular, the method is appropriate for before-after analyses that attempt to measure the effect of some activity or disturbance in the system. In future research attempts should be made to quantity the nonstationarity and explicitly model this phenomenon. Better long-term predictions will result. The transition matrix approach that has been presented is a valuable tool for obtaining more insight into nonstationarity, in general, and shifts in preference structures, in particular. References 1.

Chatfield, C., and Goodhardt, G. J., The Beta Binomial chasing Behavior, Appl. S&t. 19 (1970): 240-250.

2.

Cramer, H., Mathematical Princeton, 1946.

3.

Feller, W., An Introduction to Probability 2nd ed., John Wiley, New York, 1971.

4.

Greene, J. D., 1970): 12-18.

5.

Headen, R. S., Klompmaker, J. E., and Teel, J. E., Jr., Predicting Audience Exposure to Spot TV Advertising Schedules, J. MarketingRes. 14 (February 1977): l-9.

6.

Massy, W. F., Montgomery, D. B., and Morrison, Buying Behavior, The MIT Press, Cambridge, 1970.

7.

Raiffa, H., and Schlaifer, R., Applied vard University Press, 1961.

8.

Sabavala, D. J., and Morrison, D. G., A Model of TV Show Loyalty, Res. 17 (December 1977): 35-43.

9.

Schreiber, R. J., Instability (April 1974): 13-17.

10.

Personal

Methods

Media

of Statistics,

Model for Consumer University

Press,

Theory and its Applications

Vol. II,

Probabilities,

Princeton

Pur-

J. Advertising

Res.

D. G., Stochastic Models of

Statistical Decision Theory,

in Media Exposure

10 (October

Habits,

Boston:

J. Aduersiting

J. Advertising

Winkler, R. L., and Hays, W. L., Statistics: Probability, Inference Holt, Rinehart and Winston, New York, 1975.

Har-

Res.

14

and Decision,