An entropy approach to unintrusive targeted advertising on the Web

Computer Networks 33 (2000) 767–774 www.elsevier.com/locate/comnet

An entropy approach to unintrusive targeted advertising on the Web John A. Tomlin 1 IBM Research Division, Almaden Research Center, K53=802, 650 Harry Road, San Jose, CA 95120-6099, USA

Abstract This paper describes the formulation of a new model for unintrusive targeted advertising on the web, extending the linear programming approach taken by Langheinrich et al. [M. Langheinrich, A. Nakamura, N. Abe, T. Kamba and Y. Koseki, Unintrusive customization techniques for Web advertising, in: Proc. of World Wide Web 8, 1999]. A feature of our model is that it avoids unrealistic solutions of the type which may show ads to only a too-narrow group of users. This is accomplished by using a statistically derived entropy maximization model, which incorporates a form of randomization in associating advertisements with targetable groups of users, as well as considering click-through probability. It is then shown that this nonlinear entropy model can be embedded in larger models for the purpose of optimal management of web advertisement portfolios by agencies or brokerages.  2000 Published by Elsevier Science B.V. All rights reserved. Keywords: World Wide Web; Electronic commerce; On-line advertising; Optimization; Statistics; Entropy; Equilibrium

1. Introduction This paper outlines the formulation of some entropy models for directed advertising on the Web [9], and in particular for unintrusive targeted advertising. The initial aim is to build an optimization model which can be used to compute frequencies for showing advertisements to groups of users to stably optimize click-throughs or revenue. A feature of the model is that it is formulated to avoid unrealistic, ‘over-targeted’ solutions. To illustrate what this means, and to motivate the rest of this paper, let us consider a very simple example. Suppose we have 100 identical banner ads to be presented to two distinguishable types, or groups, of users, who view the page on which the ad may be displayed in equal numbers, and who have estimated click-through probabilities of 51% and 49%. How 1

E-mail: [email protected]

many of the ads should we show to each type of user to maximize the expected number of click-throughs? If we let x1 , x2 be the number of ads shown to users of type 1 and 2, this problem can be expressed as a tiny linear program (LP): Maximize 0:51x1 C 0:49x2 subject to x1 C x2 D 100 xi ½ 0 The obvious ‘optimal’ solution is x1 D 100, x2 D 0, in other words to show all the ads to the first group of users, to achieve an expected 51 click-throughs. The second group are shown no ads at all. Is this solution realistic or desirable? If the clickthrough probabilities and ratio of users were known with perfect accuracy, perhaps. But in the real world such data are uncertain. Suppose the uncertainty in the click-through probabilities is a modest 5%. Then in the worst case, the actual probabilities might be 46% and 54%, the coefficients in the function to be

1389-1286/00/$ – see front matter  2000 Published by Elsevier Science B.V. All rights reserved. PII: S 1 3 8 9 - 1 2 8 6 ( 0 0 ) 0 0 0 6 2 - 1

768

J.A. Tomlin / Computer Networks 33 (2000) 767–774

maximized would be 0.46 and the 0.54, rather than 0.51 and 0.49, and the ‘optimal’ solution would be completely different — to show all of the ads to the second group of users (x1 D 0, x2 D 100). This would result in 54 expected click-throughs, whereas our previous solution with x1 D 100 would result in only 46. These drastic differences in solution are clearly unsatisfactory, and we shall refer to ‘all-ornothing’ solutions of this type as ‘over-targeted’. It might be thought that one way out of this dilemma is to specify a minimum allocation of ads to each group — to demand that (say) each group is shown at least 10 ads (i.e. xi ½ 10). However, the effect of this is that once each group has been assigned its minimum 10 ads, the remaining 80 will again be assigned on an all-or-nothing basis as before, and our solution will be (x1 D 90, x2 D 10) — only slightly more stable under changes in the probabilities than before. We might improve the situation by increasing the lower bound further, but we have no guidance on how much. Faced with the above situation, the commonsense approach is to “more or less evenly distribute the ads between the groups of users, but with a bias toward the group(s) with the higher click-through probability”. We must then ask how this is to be expressed mathematically. The results of Section 3 of this paper suggest that instead of solving the tiny linear program (LP)above, we should solve the following tiny non-linear program (NLP): Maximize 0:51x1 C 0:49x2 0:5x1 ln.x1 / 0:5x2 ln.x2 / subject to x1 C x2 D 100 xi ½ 0 The solution (to 4 figures) to this NLP is x1 D 51:00, x2 D 49:00, yielding an expected number of 50.00 click-throughs — one less than in the LP optimum, but much more robust under changes in the data. Under the worst case 5% variation in probabilities considered above, the extreme cases — when the probabilities are (0:56; 0:44) and (0:46; 0:54) — yield solutions of (55:97; 44:03) and (46:01; 53:99) with expected click-throughs of 50.72 and 50.32 — a far more stable outcome. The nonlinear terms in the NLP objective have accomplished the goal of “more or less evenly distributing the ads between the groups of users”, which we might more formally

describe as randomization, but the linear terms still give us a “bias toward the group(s) with the higher click-through probability”. While the concept of randomizing may at first appear odd in the advertising context, any Web surfer will have observed that portals do in fact serve up mixtures of ads. The first priority of this paper is then to formulate optimization models which avoid ‘over-targeted’ solutions by a technique which allows for randomization, as well as considering click-through probability, and which is suitable for much larger (and less contrived) problems than our example, involving many types of users and ads. We use a statistical argument in deriving this model, following the lead given in other modeling areas — notably transportation planning. These models also have the advantage that efficient algorithms are available for their solution, making them attractive in practice. A further advantage of this formulation is that it provides a fundamental building block for more far-reaching objectives in Web-based advertising. In particular we will describe how this linear-logarithmic model can be embedded in a larger model for the purpose of optimal management of Web advertisement portfolios by agencies or brokerages. In the next section we move beyond the highly simplified LP model described above, and outline the quite general linear programming model formulated by Langheinrich et al. [10] for their ADWIZ system. This is followed by a derivation of the entropy approach and a description of the form of the solutions. We then describe how this model may be embedded in a prototype multi-time-period planning model for an advertising campaign. Finally we summarize some of the many interesting areas of further exploration associated with the entropy modeling approach.

2. The ADWIZ LP model Langheinrich et al. [10] have described a system for ‘unintrusive customization’ of Web advertising — unintrusive in the sense that only impersonal information, such as use of a search keyword, or accessing of a page, is required. This system consists of a number of elements. Those of most interest to us here are a Selection Engine and a Learning System.

J.A. Tomlin / Computer Networks 33 (2000) 767–774

The first of these components uses a set of weights in a Relevancy Computation to compute the probability of displaying a particular ad to a user, the user being associated with certain keywords or pages. The component of principle interest here is a ‘learning’ system, which computes display weights based on current estimates of click-through probabilities, the keyword list and the ad inventory. The components and formulation of the model (but with the indices reversed here) are as follows:

Data h j The desired display rate for ad j (normalized over all ads); ki The keyword input probabilities (normalized over all keywords); ci j Click-through probability for ad j and keyword i. Variables di j The display probability of ad j for keyword i.

jD1 m X

.i D 1; : : :; m/I . j D 1; : : :; n/I

ki di j D h j

iD1

(1)

.i D 1; : : :; mI j D 1; : : :; n/:

di j ½ 0 Maximize m X n X ci j ki di j : iD1 jD1

Note that the first equation in (1) above can be multiplied through by ki , followed by a change of variable: ²i j D ki di j to produce the equivalent constraints: n X ²i j D ki .i D 1; : : :; m/I jD1 m X iD1

²i j D h j

²i j ½ 0

. j D 1; : : :; n/I .i D 1; : : :; mI j D 1; : : :; n/:

(3)

iD1 jD1

This is a classic ‘transportation problem’ (see e.g. [3]), and we shall work with this and subsequent models in this form. The model may be complicated in practice by contractual requirements setting a minimum on certain ad display rates. Such constraints, of the form ²i j ½ L i j

Indices i D 1; : : :; m The keywords; j D 1; : : :; n The ads available.

Constraints n X di j D 1

and objective: n m X X Maximize ci j ²i j :

769

(2)

where L i j is a constant, do not significantly complicate the model. This is fortunate, because although efficient algorithms exist for general linear programs, there exist algorithms which are orders of magnitude faster for the transportation problem (see [8]). Since these models may become quite large, or need to be solved frequently, this is an important practical consideration. Despite the virtues of this model — simplicity and fast solution time — it suffers from much the same weakness as the LP model in the introduction. Linear programming theory (see [3]) tells us that a model like this, with .m C n/ linear constraints on mn variables, has optimal solutions in which at most .m C n/ of the variables are away from their lower bound (usually zero). Thus for realistically large values of m and n we expect only a small fraction of the variables to be nonzero. In other words we expect the same phenomenon of over-targeting; on an even larger (if less extreme) scale. Once again we must regard such solutions as unrealistic and unsatisfactory, with many of the ads not being shown at all for most keywords. Langheinrich et al. [10] recognized this, and resorted to the technique of applying rather arbitrary lower bounds on all the variables. As we saw in the small example, this can remove the immediate symptom — where many or most ad=keyword combinations never appear — but it is far from clear in what sense such solutions can be described as ‘optimal’.

3. A statistical theory In this section we shall employ a methodology used in Traffic Theory for the distribution of vehi-

770

J.A. Tomlin / Computer Networks 33 (2000) 767–774

cles from origins (sources) to destinations (sinks). Following the description (but not the notation) of Wilson [13] in the traffic context, let us define a total number of trips Ai originating at origin node i, and require B j trips to end at destination j. Let xi j be the number of trips for the origin-destination pair .i; j/, further let ci j be some generalized cost or impedance of travelling from i to j. Then clearly we require: n X xi j D Ai .i D 1; : : :; m/I jD1 m X

xi j D B j

iD1

xi j ½ 0

. j D 1; : : :; n/I .i D 1; : : :; mI j D 1; : : :; n/

and for feasibility we must have: m n X X XD Ai D Bj iD1

(4)

(5)

jD1

where X is the total number of trips. We shall also assume for the time being that another constraint is satisfied: m X n X ci j xi j D C: (6) iD1 jD1

The basic assumption is that the probability of the distribution fxi j g occurring is proportional to the number of states w.xi j / of the system which give rise to this distribution. The number of distinct arrangements of individuals which give rise to a distribution fxi j g is:  Y  w.xi j / D .X!/ xi j ! : (7) ij

Now finding the maximum of this function is equivalent to finding the maximum of the log of w, which after applying Stirling’s formula, and neglecting constant terms, requires us to: m X n X xi j ln.xi j / (8) Maximize iD1 jD1

subject to (4) and (6). The function appearing in (8) is an entropy function. Without going into detail, the solutions of this entropy maximization problem can be shown to be of the form: xi j D ai Ai b j B j exp. þci j /

(9)

where þ is the Lagrange multiplier for equation (6). In practice it is not necessary to know the value of þ. An efficient iterative (scaling) procedure is available (see [11,5]) which enables us to solve the problem without having to resort to more expensive general nonlinear programming methods. It can also be shown that for practical values of the Ai , B j that this maximum is sharp [13]. The maximum entropy solution discussed above is one form of equilibrium solution. The applicability of this model to the targeted advertising problem should now be clear. If we take trips originating at origins as analogous to ‘buckets’ of users, with Ai users in each bucket, and the destinations as groups of ads to be shown, with B j ads of each type, and consider the ci j as click-through probabilities with C as the total number of clicks required, we obtain an unnormalized extension of the model discussed in Section 2. Once again, contractual lower bounds may be imposed on particular ad=user pairings (as long as feasibility is not lost). However, there is no ‘all-or-nothing’ nature to the solution, and over-targeting may be avoided without any requirement for arbitrary lower bounds. Indeed, since the model involves the logarithm of the xi j ’s, they must necessarily be positive. Another statistical model of interest here has also arisen in traffic theory, and is perhaps even more appropriate (see [12]). We proceed by analogy with the Helmholtz free-energy function, which is at a minimum for a system in equilibrium in conditions of constant volume and temperature. This function is of the form: FDE

K ln w

(10)

where K is a constant, E is the internal energy, and w is the number of states as defined in (7). We are interested in maximizing click-through, or equivalently, minimizing non-click-through probability. Again using Stirling’s formula, and defining Ł ð ci j D max pq c pq ci j (11) if we identify these ‘cost’ values as the analogue of energy we obtain: F D constant C

n m X X iD1 jD1

Ð xi j ci j C ln.xi j / :

(12)

J.A. Tomlin / Computer Networks 33 (2000) 767–774

Here is a constant, replacing K , whose value is yet to be determined. We now assert that the equilibrium distribution is that which minimizes F subject to (4). The constraint (6) is no longer needed, and the parameter accommodates a range of cases, from the extreme of D 0, which gives us the unnormalized version of the LP model in Section 2, to a completely proportional model, giving the solution xi j D Ai B j = X

(13)

when is taken to be arbitrarily large. The general form of the solution to this model can be shown to be of form xi j D aO i bO j exp. ci j = /

m X n X

x i j ci j

models which we may now consider building for the management of Web advertising campaigns. Note that we have made no assumptions on how the ‘buckets’ of users are defined. They may correspond to search keywords, states or histories. Similarly, the assigning of the ads to groups may be by individual or classes of ad. The key pieces of data are the number of users or ads in each bucket or group and the click-through probabilities. The question of maximizing revenue then naturally arises, and can be answered by applying revenue weights to the ci j term in the objective. This will be seen in the more general model of the next section.

(14)

which is of the same form as (9). We must then choose a value for . It is shown in [12] that under certain assumptions the weighted mean of the cost values (as defined in (11)) provides a good fit for the analogous traffic problem, and we shall initially assume so here. This allows us to use an iterative procedure (in ) to solve the problem. A good initial value for has proved to be simply the mean of these cost values for some models, and sometimes this is even a good enough estimate to obtain good agreement between the model and real data. (We used this value in the tiny NLP example in the introduction). Once again the solution to (14) for any can be obtained by an efficient iterative (scaling) procedure. Thus far we have stated this form of the statistical model as a minimization problem. Once has been chosen this is of course equivalent to the maximization problem: Maximize

771

ln.xi j /

Ð

(15)

iD1 jD1

subject to (4), and we shall also use the model in this form. This form of the statistical model offers significant advantages over that stated in (4)–(8). The constraints are those of the classical transportation problem, and the rather arbitrary constraint (6) has been replaced by a parameter in the linear–logarithmic objective function for which we have some rationale for assigning a value. For either case, we have a selfcontained, easily solvable, constrained optimization model which can be embedded in more complex

4. An embedded model We will consider the environment in which our models are to be used — the ad supply chain — to be made up of three segments: Advertisers who hire the agencies to display their ads as effectively as possible to users at the various properties. Agencies=Brokerages who choose and display ads at a property, using what information there is available on the users (if any). Properties The particular pages, typically at a portal, where banner ads are displayed by the agency=broker(s). For concreteness, let us formulate a model which considers only the first two of these specifically — an agency and a number of advertisers who wish to present ads to users in (at least some of) the same buckets. We also broaden the model to multiple time periods. The aim of the agency is to obtain ads from the advertisers which will maximize their net revenue, given the expected number of users in each bucket per time period, and the click-through probabilities for ads in each time period. The components of this model are: Indices i D 1; : : :; m The buckets of users; j D 1; : : :; n The ad types available; k D 1; : : :; K The advertisers;

772

J.A. Tomlin / Computer Networks 33 (2000) 767–774

t D 1; : : :; T Time periods. Data Ait The number of expected users in bucket i in period t; ci j Click-through probability for ad j by user i in period t; Ri jt Revenue from click-through for ad j by user i in period t; Di jt Revenue (or Cost) for displaying ad j by user i in period t; P jtC Penalty for shortfall of shown ads type j at end of period t; P jt Penalty for excess of shown ads type j at end of period t; M jkt Agency’s cost of obtaining ads of type j from advertiser k to be shown in period t; U jkt Upper limit on ads of type j from advertiser k in period t; L jkt Lower limit on ads of type j from advertiser k in period t;

t Entropy weight for period t. Variables xi jt The displays of ad j for user type i in period t; y jkt The number of ads j bought by advertiser k for display in period t; z jt The number of ads j shown to all users in period t; s Cjt Inventory of unshown ads of type j at end of period t; s jt Excess of shown ads of type j at end of period t. Constraints

n X jD1

z jt D

m X

s j;t

1

1C

K X kD1

Ait

y jkt

z jt

for all j; tI (16) for all t:

iD1

Supply and demand: n X xi jt D Ait for all i; tI jD1 m X iD1

(17) xi jt D z jt

Maximize X xi jt .Di jt C Ri jt ci jt t ln.xi jt // i; j;t X X .P jtC s Cjt C P jt s jt / M jkt y jkt : jt

for all j; t:

(18)

(19)

jkt

Although the broader questions of costing of Web advertising (see [1,7]) are beyond the scope of this paper, note that by allowing revenues (or costs) to be associated with ads that are clicked on or otherwise (via the Di jt and Ri jt coefficients), as well as marketing costs M jkt , considerable flexibility is provided. This prototype model needs some further specifications to complete it. For example dummy ads may need to be added to make sure that the second material balance constraint can be satisfied, but the above description gives us a sufficiently detailed model to examine its structure. Clearly it can grow quite large quite quickly if T becomes large, or if m or n are taken to be large. However the model has a definite structure. If we let H represent an m by n transportation problem coefficient matrix, the structure of the entire problem coefficient matrix is of the form: 1 0 .0/ A C B A.1/ H C B .2/ C BA H C B C B :: :: A @ : : A.T /

H .k/

Material balance: s Cjt s jt D s Cj;t

Bounds: s Cjt ; s jt ; xi jt ; z jt ½ 0 for all i; j; t; L i jt  yi jt  Ui jt for all i; j; t:

where the A are coefficients corresponding to the y jkt , z jt , s Cjt and s jt variables only. This structure is well known in the optimization community to be amenable to a technique known as Generalized Benders Decomposition (see [6]). Without going into too much technical detail, for this model the decomposition leads to solving multiple subproblems of the form (4), (15) in only the xi jt variables, with the right hand sides B jt now determined by the ‘master’ z jt values, and a ‘master’ problem in the y, z, s C , s variables, with constraints derived from the A.k/ matrices and the ‘cuts’ generated by the subproblems. We expect the overall solution procedure to very efficient, even for large problems.

J.A. Tomlin / Computer Networks 33 (2000) 767–774

It is important to note that the Generalized Benders approach can be applied in a wider context. It is not limited in applicability to the maximum entropy submodels we have considered. Any procedure for displaying groups of ads to buckets of users which can notionally be expressed as an optimization problem in our xi jt variables, subject to constraints only on those variables, is a candidate for this treatment. Thus some form of Markovian model (see [2]) may be just as amenable as the entropy model(s) we have discussed.

773

modifying it to vary the weight on the entropy terms for the different strata, without losing the matrix structure which promises efficient solution. Other extensions of this model involve examining the structure of the costs which we have so far considered constant. Especially when multiple advertisers and multiple portals are considered, there is an opportunity to use the model to evaluate some forms of nonlinear pricing for yield management (see [4,14]).

6. Conclusion 5. Refinements and extensions There are many possible extensions to the embedded targeting model described above, to encompass more of the ad supply chain. It is relatively straightforward to extend it to consider properties on multiple portals by stratifying the buckets of users (say by adding an index p) and considering not only variables x pi jt , etc., but stratified revenues R pi jk , etc. Such a model, incorporating agencies, advertisers and properties=portals will still have the basic matrix structure shown above, and be amenable to treatment by decomposition. There are other statistical techniques, again grounded in transportation studies, which might well be considered for application in targeted advertising applications. One of these is the ‘intervening opportunities’ model (see [13]) which ranks, in this context, groups of ads in decreasing attractiveness for each bucket of users, and using a probability that opportunities of a certain rank will be passed up, constructs an exponential decay model for associating users with groups of ads. It should be considered that the model(s) formulated here deliberately assume (since they are ‘unintrusive’) that very little is known about individual users — only the bucket to which they belong and the click-through probabilities for that bucket of users. If we relax the unintrusivity requirement it may well be that we can stratify users by information level — some with the information level we have used above, some with limited information available through cookies, and others for which a detailed click-trail is available. Once again, it is possible to extend the model to accommodate this stratification,

We have shown that entropy models, and potentially other modeling techniques from other areas of Operations Research, can provide useful tools in the context of Web advertising campaigns, either in a single time-slice application, or more generally, at a strategic level in the management over time of entire campaigns and portfolios. As in any large scale optimization application, data availability is critical. This work is at an early stage. The next step is the acquisition of suitable data to test and calibrate the models we have described. The author is indebted to Prabhakar Raghavan and Sridhar Rajagopalan for valuable discussions in the course of this work.

References [1] R. Briggs and N. Hollis, Advertising on the Web: is there response before click-through? J. Advert. Res. 37 (2) (1997) 33–45. [2] M. Charikar, R. Kumar, P. Raghavan, S. Rajagopalan and A. Tomkins, On targeting Markov segments, in: Proc. of the ACM Symposium on Theory of Computing, ACM Press, 1999. [3] G.B. Dantzig, Linear Programming and Extensions, Princeton University Press, Princeton, NJ, 1963. [4] S. Daudel and G. Vialle, Yield Management: Applications to Air Transport, Institut du Transport Aerien, Paris, 1994. [5] S. Erlander, Optimal spatial interaction and the gravity model, Lecture Notes in Economics and Mathematical Systems 173, Springer, Berlin, 1980. [6] A.M. Geoffrion, Generalized benders decomposition, J. Optimization Theory Appl. 10 (1972) 237–260. [7] B. Harvey, The expanded ARF model: bridge to the accountable advertising future, J. Advert. Res. 37 (2) (1997) 11–20.

774

J.A. Tomlin / Computer Networks 33 (2000) 767–774

[8] J.L. Kennington and R.V. Helgason, Algorithms for Network Programming, Wiley, New York, 1980. [9] Y. Kodha and S. Endo, Ubiquitous advertising on the WWW: merging advertisement on the browser, in: Fifth International World Wide Web Conference, Paris, France, May 1996, see also: http://www5conf.inria.fr/fich_html/pap ers/P52/Overview.html. [10] M. Langheinrich, A. Nakamura, N. Abe, T. Kamba and Y. Koseki, Unintrusive customization techniques for Web advertising, in: Proc. of World Wide Web 8, 1999, available at: http://www8.org/w8-papers/2b-customizing/unintrusive/ unintrusive.html. [11] R.B. Potts and R.M. Oliver, Flows in Transportation Networks, Academic Press, New York, 1972. [12] J.A. Tomlin and S.G. Tomlin, Traffic distribution and entropy, Nature 220 (1968) 974–976.

[13] A.G. Wilson, A statistical theory of spatial distribution models, Transport. Res. 1 (1967) 253–269. [14] R.B. Wilson, Nonlinear Pricing, Oxford University Press, Oxford and New York, 1993. John Tomlin gained his B.Sc. (Hons) and Ph.D. in Mathematics at the University of Adelaide, South Australia. Since then he has worked continuously in the areas of operations research, applied optimization and optimization software. He joined the IBM Research Division in 1987. He is a member of ACM, INFORMS and the Mathematical Programming Society.