Control of an epidemic spreading in a heterogeneously mixing population

Control of an Epidemic Spreading in a Heterogeneously Mixing Population DAVID GREENHALGH* Co,zrrol and Munugemenr Systems Division, Engineering Department, Mill Lane. Cambridge, CB2 IRX, England Received 29 March 1985; revised 6 January 1986

ABSTRACT A theoretical epidemic model for a heterogeneously mixing community is examined in the context of control by immunization of susceptibles and removal of infected people. Under certain assumptions an optimal control scheme is developed using the ideas of dynamic programming, and is found to have a particularly simple form for either method of control. Counterexamples are discussed to illustrate why the simplifying assumptions made cannot be relaxed. Applications of these results are considered with respect to the accuracy of the assumptions of the model.

1.

INTRODUCTION

Most classical epidemic models assume that the population amongst which a disease is spreading mix homogeneously as outlined in Bailey [l]. However an alternative, more realistic, assumption would be to assume that the population is divided into several distinct groups with homogeneous mixing within the groups but with heterogeneous mixing between groups. This heterogeneity may be due to a variety of factors such as an age structure or some other geographical or social structure in the population. Another possible cause of heterogeneity may be differences in the genetic composition of the population. Examples of this type of approach are given by Watson [2], Rushton and Mautner [3], and Cane and McNamee [4]. Watson considers a stochastic heterogeneously mixing epidemic model by using simulations and analytic approximations, whereas Rushton and Mautner consider the case of a deterministic simple epidemic in m equivalent classes. The model discussed in this paper is based on Cane and McNamee’s model, which will therefore be discussed in detail later.

*Address for correspondence: Glasgow, Gl lXH, Scotland. MA THEMA

TICA L BIOSCIENCES

Department

80:23-45

OElsevier Science Publishing Co., Inc., 1986 52 Vanderbilt Ave., New York, NY 10017

of .Mathematics,

University

of Strathclyde,

23

(1986)

0025-5564/86/$03.50

24

DAVID

GREENHALGH

A separate group of papers suppose that the population amongst which the disease is spreading is divided into several different age classes with a “contact matrix” giving the rates of spread of infection between several different age classes (Longini et al. [5], Anderson and May [6], May and Anderson [7], Schenzle [S]). These papers are aimed more at specific diseases. Longini et al. consider a deterministic model for the spread of influenza and use computational optimization techniques to determine an optimal vaccine distribution between age classes. They consider two types of optimization: to minimize the monetary cost of an epidemic and to minimize the expected number of years of life lost. They find that there is a conflict between the need to stop the disease spreading and the need to protect those members of the community whose becoming infected would incur the highest cost. This corresponds to a conflict between the need to vaccinate the younger age classes who spread the disease fastest and the older age classes who would incur a greater cost by becoming infected. One of our models is similar to, but simpler than, this model: the relationship between the two sets of results will be discussed later. The two papers by Anderson and May discuss the distribution of vaccine amongst several different populations which mix heterogeneously. In one paper an age structured model is used to describe measles outbreaks; in both the optimal vaccination strategy is derived to eradicate the disease with as small an amount of vaccine as possible. However, the results are not directly relevant to the results presented in this paper, as Anderson and May are discussing endemic diseases as opposed to epidemics. Thus there is a continual flow of susceptibles into the population caused by the birth of new susceptibles. Here an optimal vaccination strategy is to allocate fixed amounts of vaccine to the different groups as described in the papers. The paper by Schenzle describes how a detailed age structured model can be used to generate a good fit to measles data in England and Wales. These papers were picked only as examples of models for diseases which involve heterogeneous mixing and themselves contain many references to other such papers. In this paper we take one such epidemic model and introduce an element corresponding to control into it. We wish to determine how to distribute this amongst the different groups of people in the epidemic. The disease is discussed first when controlled only by immunization so as to minimize the expected number of people infected and with the less direct objective of maximizing the expected number of people immunized. For the case where the disease is controlled by removals instead of immunization we discuss controlling the disease so as to minimize the expected number of people infected, minimize the number of people who have ever caught the disease, and maximize the expected number of people removed. The motivations for some of the less usual objectives are discussed later in the paper. A comprehensive study of mathematical models for the control of pests and infectious diseases up to 1977 was detailed by Wickwire [lo]. Although

HETEROGENEOUSLY

MIXING COMMUNITY

25

some of the techniques of control theory, and in particular stochastic optimal control theory, are relevant to the results discussed here, most of the results are for a disease spreading in a homogeneously mixing population, and hence are not directly applicable to the results discussed here. It should perhaps be emphasized that the model discussed in this paper makes many simplifications. It is not intended to be a specific model for one particular disease, as a simulation model might be, nor could it be expected to give precise numerical values for the expected numbers of people infected, immunized, and so on. Rather, this model is used to explain why certain “obvious” policies are optimal under certain conditions. It was necessary to steer a line between those assumptions which would provide a tractable model and more accurate assumptions. Throughout the paper we shall try to point out what these assumptions are, why they were made, when they will be appropriate, and when they can be relaxed. The paper is organized as follows: In the remainder of this section we discuss the model of Cane and McNamee in detail. In the following sections we introduce the two elements of vaccination and removals of infected people into the model and introduce the notation used throughout the paper. In Section 3 we discuss control of the epidemic by immunization, and in the following section we discuss control by removals. Next we discuss a modification of the model and counterexamples to explain why the less realistic assumptions cannot be relaxed. In the final section the applications and limitations of the model are examined. Cane and McNamee [4] outline a model for the spread of infection in which the population is assumed to mix in a heterogeneous manner. This model development was motivated by the need to assess how far a contact rate estimated for one city might reasonably be applied to another, as was implicitly done in a computer model for the spread of influenza in Russia [9]. In Cane and McNamee’s model the population consists of n individuals: Z, , Z2,. . . , Z,,, where n is large; each of these people acts independently. Each individual Z, is assigned a contact parameter X,. This can be thought of as the proportion of his time that this individual spends in contact with the rest of the population, say in some communal meeting place, and 1 - A, can be thought of as the proportion of his time that he spends isolated from contact with other susceptible or infected individuals. Thus we can think of h, as representing the degree of susceptibility or infectiousness of individual Z, . In a small interval of time [t, t + Ar), Pr(Z, meetsI,

in[t,t+At))

=h,A,At+o(Ar).

It is also assumed that a susceptible meeting an infected person is infected instantaneously. If Z, is infected and I, is susceptible, then Z, will infect Z, after a random time interval which has an exponential distribution with parameter X,h,. The homogeneously mixing epidemic is a special case

DAVID GREENHALGH

26

of this with all the X,‘s equal. The precise numerical value of any given X, will depend on the social habits of the individual I,: amongst other things his or her sex, age, temperament, and geographical location. 2.

CONTROL

OF THE EPIDEMIC

We now outline those assumptions which were added to Cane and McNamee’s model concerning control of the epidemic. These naturally fall into the assumptions which concern removals and the assumptions which concern immunization. For control by removals, if an amount p of removal effort is applied to an infected individual, then he or she will be removed after a time which has an exponential distribution with parameter p. Removal effort has effect only when applied to an infected individual. However, we wish to allow the amount of removal effort applied to vary with the time and the state of the epidemic. Hence we suppose that if an amount p, (S, T) of removal effort is applied to the infected individual 1, in the small time interval [T, T -t AT] when the epidemic is in the state S, then he or she will be removed in that time interval with probability

Events in successive time intervals are independent. However, in practice there would be some physical or economic constraint on the rate at which infected people could be removed from the epidemic. To take this into account we suppose that there is a maximum total rate pa at which removal effort can be applied at each instant, so that

Similar assumptions are made concerning immunization effort. If an amount u of immunization effort is applied to a susceptible individual, then he will be immunized after a time which has a exponential distribution with parameter U. More generally if an amount u, (S, T) of immunization effort is applied to the susceptible individual 1, in the small time interval [T, T + AT] when the epidemic is in the state S, then he will be immunized in that time interval with probability u,( S, T) AT + o( AT). Events in successive time intervals are taken to be independent, and immunization effort applied to an individual who is either infected or immunized has no effect. Again there is a limit u0 on the total rate at which immuniza-

HETEROGENEOUSLY

MIXING COMMUNITY

27

tion effort can be applied at time T so that

o< t U,(S,T) GUU,. r=l Whilst it is fairly standard to assume that the infection and removal times are exponential in stochastic epidemic models, we know of few models which consider immunization as a stochastic process. The choice of an exponential distribution for the immunization time here is mainly determined by the fact that it allows a tractable model to be set up and the optimal control strategy to be determined. In fact the more commonly made assumption that the infection time distribution is exponential is often made for the same reason. Although it would be desirable to replace this by a more realistic distribution, an exponential immunization time distribution would be approximately true under the following circumstances. Suppose that a patient is treated with a drug at regularly spaced intervals. Each application is either instantaneously successful with probability p or otherwise has no effect. Successive applications are independent. Then the distribution of the time until the patient is successfully immunized is a geometric distribution which can be approximated by an exponential distribution. NOTA TION

Let n IMM(t) SUS( t) INF( t) REM(t)

= no. = no. = no. = no. =no.

of of of of of

people amongst whom the disease is spreading; immune people at time t; susceptible people at time t; infected people at time t; removed people at time t.

We assume complete knowledge of the state of the epidemic. For control by immunization this is determined by who is infected, susceptible, and immunized, and for control by removals by who is infected, susceptible, and removed. The parameters X, are assumed known. To simplify the statement of results we take each X, > 0. S denotes a state of the epidemic. Suppose that I,, is any susceptible in the state S, and 1, is any infected person in S. Let S, denote the state obtained from S by immunizing I,, let S’ denote the state obtained from S by I,, becoming infected, and let Sq denote the state obtained from S by removing I,. Thus for example S,” denotes the state obtained from S by I, becoming infected and 1, being removed; if 1, is any other susceptible in the state S, then S*’ denotes the state obtained from S by both I, and I, being infected, and if I, is any other infected individual in the state S, then Sy, denotes the state obtained from S by both I, and I, being removed. More

28

DAVID GREENHALGH

generally, suppose that P is any set of susceptibles and Q is any set of infected people in the state S. Then S, denotes the state obtained from S by each person in the set P being immunized, and Sp denotes the state obtained from S by each person in the set P being infected. Sa denotes the state obtained from S by each person in the set Q being removed. The obvious convention applies for multiple combinations of superscripts and subscripts; for example S,’ denotes the state obtained from S by each person in the set P being infected and Z, being removed. u,( S, t) and p,(S, f) denote respectively the amount of immunization or removal effort applied to individual 1, at time t if the epidemic is then in the state S. Throughout this paper we shall assume that for each i and each state S, each of u,(S, t) and p,(S, t) is a continuous function of 1. It is straightforward to extend the results to the case where these functions are piecewise continuous functions of t with only finitely many discontinuities. 3.

CONTROL

MAXIMIZING

BY IMMUNIZATION THE

EXPECTED

NUMBER

OF PEOPLE

IMMUNIZED

We first consider the model where the disease is controlled by immunization only. This means that we are considering a simple epidemic model where infected people stay infected forever and are not removed. It should however be pointed out that vaccines are not available for many diseases where infected people stay infected forever, such as herpes. It is supposed that there is a finite time T over which the epidemic is controlled. The model described is best suited to epidemics which occur in less than a year. For longer time periods it is necessary to include births and deaths, since (as previously stated) births are an important source of new susceptibles. Hence our model is not suitable for modeling endemic diseases over long time periods. Suppose that the epidemic is to be controlled by immunization to maximize the expected number of people immunized at time T, which is short compared with an average lifespan. Then the best policy is to treat the “most susceptible” person (the susceptible person with the largest value of X) all of the time, In fact this policy is still best if the epidemic is controlled so as to minimize the expected number infected at time T. We prove the result in detail only for the first case. Some comments are given here to provide a motivation for taking the objective to be maximizing the expected number of people immunized. The idea is to build up a residual immunity in the population. This is beneficial in two ways: first, those people who are immunized are protected from ever catching the disease in the future (as we have assumed that immunity is permanent), and second, if a large body of people is immunized then this immunity in the population will slow down the future spread of the disease.

HETEROGENEOUSLY

MIXING COMMUNITY

29

Suppose that u is a general policy. Let E[IMM(T), U: S] denote the expected number of people immunized at time T if the policy u is used and the epidemic starts in the state S; E[IMM( T), u; S, ~1 the expected number of people immunized at time T if the policy ZJis used and the epidemic is in the state S at time T. Let u* denote the proposed optimal policy which always applies full immunization effort to the “most susceptible” person in the epidemic. For notational convenience write F(S,T)=E[IMM(T),~*:S]. By time homogeneity F(S,T-T)=E[IMM(T),u*:S,T]. The first result which we shall prove appears intuitively obvious. It is needed as a intermediate step to the main result. It is proved by an induction on the finite set of states of the epidemic as follows. First the result is shown to be true for all states in which the disease has stopped spreading. Given a general state, the result is then assumed to be true for each possible state which the disease can spread to. Then the result is shown to be true for the original state. This provides a valid proof for the two types of epidemic model considered in this paper, as each realization must lead to a terminal state in a finite number of steps and there are only a finite number of states. A similar technique is used repeatedly throughout the paper. For the case where the disease is controlled by immunization only, this reduces to a simple induction on the number of susceptibles in the state S. LEMMA

I

Suppose the state S contains at least one infected person I,; let I, and I, he susceptibles in the state S with A, > A,, and P a set of other suxeptibles in the state S. Then: (i) if I, 4 P, F(S,P,T)


with equality if and only if (a) T = 0 or (b) the set P is empty and A, = A,; (ii) if I,, I, 4 P, F(S,‘P,T)


with equalicv if and only if (a) T = 0 or (b) the set P is empty and A, = A, or (c) the state S/ contains no susceptibles and the set P is empty; (iii) we have F(S,,T)‘F(S,T);

DAVID

30

GREENHALGH

and (iv) we have

F(S,T)

> F(P,T)Y

with equali!y if and only if (a) T = 0 or (b) the set P is empcv. Interpretation. These results say that if the policy u* is used: (i) the expected number of people immunized is greater if the epidemic starts in the state obtained from S by immunizing 1, than if the epidemic starts in the state obtained from S by immunizing J, (and possibly more people becoming infected); (ii) the expected number of people immunized is greater if the epidemic starts in the state obtained from S by immunizing r, and I, becoming infected, than in the state obtained from S by immunizing Z, and 1, becoming infected (and possibly more people becoming infected); (iii) the expected number of people immunized is greater if the epidemic starts in the state obtained from S by f, being immunized than if it starts in the state S; and (iv) the expected number of people immunized is greater if the epidemic starts in the state S than if it starts in a state obtained from S by more people being infected. Proof. We assert that result (i) is true if S contains r susceptibles excluding 1,; result (ii) is true if S contains r susceptibles excluding 1, and I,; result (iii) is true if S contains r susceptibles excluding I,; and result (iv) is true if S contains r susceptibles. This is proved by induction on r. It is straightforward to show the sufficiency of the conditions for equality. We prove the remainder of the lemma by (a) showing the results when r = 0 and (b) using induction hypotheses to prove the results in general. (a) Suppose that r = 0. If the state S contains just one susceptible I,, then in (i) it is necessarily true that A, = A, and the set P is empty. In this case the sets two states S,’ and S, are identical and the result of (i) follows. It is similarly straightforward to show that the result of (ii) follows if 1, and 1, are the only susceptibles in the state S; the result of (iii) follows if 1, is the only susceptible individual in the state S; and the result (iv) follows if there are no susceptibles in the state S. (b) Suppose that r = k. We can assume inductively that (i)-(iv) are true whenever S is replaced by a state where the corresponding value of r is less than k. We advance these induction hypotheses one at a time, starting with (ii). Note that by using the induction hypothesis for (iv) applied to S,l, it is sufficient to show that F(S,J, T) > F( S’, T) with strict inequality as ap-

HETEROGENEOUSLY

31

MIXING COMMUNITY

propriate. Without loss of generality take h, > X, and suppose first that T = At is small. By considering all of the events which are of probability of first order in At it is straightforwards to show that if AI is small enough then the result is true. Define.

F(S;,v)
t=sup{t:forvE(O,tl,

(1)

to be the time up to which the result which we are trying to show is true. t > 0, as the result is true for all At small enough. We now show that if the conditions for equality are not satisfied and t < 00, then F(S,J, t) < F( S,‘, t), Continuity is then used to show that the result is true slightly beyond t. This is a contradiction, and thus 1 must be infinite. To show that F(S,‘, t) < F(S/, t) we compare two epidemics, one starting in the state S,’ and one starting in the state S,J, and introduce a null event into the second so that this comparison will work. Let A denote the set of susceptibles in sf and for Z, E A let [, = E ,, E BXn,h,, be the rate at which I,,, is attacked from the set B of infected people in S. Suppose Z, is the most susceptible person in S. Consider the epidemic starting in S/ when u* is used. Conditioning on the random time at which the next event occurs in the epidemic in [0, t) if it starts in the state S,‘, we have

F(&r)=[[([ c{

E&,+~~X~}F(S/~,~-~)

TilCA

I

(24

+u,F(S:,,vr) i xexp

[

c

i[

IS,+q++])

dT]

ItlEA

+imm( J;‘)exp

[

jz, I

(5,

+x,x,>

(-2.b)

1 119(24 + uo f

where imm(S,‘) is the number of immunes in the state S,‘. The first terms correspond to the susceptible Z, being infected, the second to the susceptible I, being immunized, and the third to nothing happening in [0, t]. Suppose that the epidemic starts in the state S,l, and introduce a null event which happens after a time which has an exponential distribution with parameter C,, E A(X, - hi) h,. This corresponds to no event happening in the epidemic and is introduced as a technical trick to make the argument work.

DAVID GREENHALGH

32

Then conditioning on the next event to occur in the epidemic in [0, t) if the epidemic starts in the state S,!, we have

(3b)

Xexp

1

JA

{ kn +

([

1++idTl

kk?l>

(34

+ imm( S,J) exp t?l‘EA

The first terms correspond to the susceptible I,, becoming infected, the second to the null event occurring, the third to the susceptible I, being immunized, and the fourth to neither an event nor the null event occurring in [0, t]. Comparing these expressions (2) and (3) term by term: (A) F(S,‘“‘, t - T) > F(S;“‘, plied to the state S”‘. (B) F(S,/, t - 7) > F(S,!, t F(S;“‘, t - T) by the induction Thus F( S,‘, t - T) 2 F( S,‘“, t satisfied for the state S, this is (A) and (B) together

t - T) by the induction

hypothesis

(ii) ap-

T) by the definition

of f, and F(S,‘, t - T) >, hypothesis for (iv) applied to the state S,‘. 7). As the conditions for equality are not strict inequality for some m.

imply that the sum of (3a) and (3b) is less than (2a).

(C) F( S,/,, t - T) > F( l$, t - T) by the induction hypothesis for (ii) applied to S,. Thus (3~) is less than (2b). (D) imm(S,‘) = imm(S,J) as the states S,’ and S,J contain the same number of immune people. Thus the term (3d) is equal to the term (2~). Putting these results together, F( S,‘, t) > F( S,‘, t) as required. By continuity this contradicts (1) and hence t = cc. This advances the induction hypothesis for Lemma l(ii). The remainder of the proof is to advance the induction hypotheses for Lemma l(i), (iv), and (iii), which is done in that order along the lines of advancing the induction hypothesis for Lemma l(ii).

HETEROGENEOUSLY

MIXING COMMUNITY

33

A crucial idea in the proof of the above result was the idea of stochastic dominance. Suppose that we are given two stochastic processes (in the above example the epidemic starting in two different states), which we wish to compare so as to maximize some given objective. Suppose that it is possible to pair the set of all possible realizations of the two stochastic processes so that both realizations in any pair have the same probability of occurrence and for each pair the outcome of the first stochastic process is preferable to the outcome of the second. Then it is clear that the overall expected outcome of the first stochastic process is preferable to the overall expected outcome of the second. Note that to set up the situation described above it was necessary to introduce an imaginary null event into the epidemic starting in the state S,! to “balance” the rate at which events happened. The following corollary is immediate. This says that the expected number of people immunized using the policy u * is greater if the epidemic starts in the state obtained from S by immunizing 1, than if it starts in the state S, and greater if the epidemic starts in the state obtained from S by immunizing 1, than if it starts in the state obtained from S by immunizing 5. Thus it is better to have immunized the more susceptible people and better to have immunized people than not. COROLLARY

.?

With the above notation, if T > 0 and A, > A, then (i) F( S, , T) F( S, T) and (4 2 F(S,, T), if and

if A,

Xi.

This yields the main result which always apply treatment effort

are seeking, that the the most susceptible

is in the

3

that the in the S and want to the expected number of people immunized at time T 0. Then u* the best policy. Proof. The result be proved by induction the number of susceptibles the starting state Let u any policy, and the policy which agrees u whilst the epidemic is state S and agrees u*. Let be the policy whilst the epidemic is the state applies the of effort u’ but it all the most susceptible in the the epidemic is not the state u” agrees u*. Thus u” from u* in that it not necessarily apply treatment effort whilst the is in the S. shall

34

DAVID GREENHALGH

prove that u is a worse policy than u’, which is worse than u”, which in turn is worse than u*. The final implication is contained in a separate lemma. Let A [B] denote the set of susceptible [infected] people in the state S. For I,, E A let t,, denote the rate at which I,, is attacked from B. Suppose that whilst the epidemic is in state S the policy u applies an amount ~~(7) of treatment effort to the susceptible individual Z, at time 7. The result is obvious if S contains no susceptibles. So assume inductively that the result is true for any state containing fewer susceptibles than S. F( u, S, T; T) = E[IMM(T), u : S, T] and F( u, S, T; 0) = F( u, S, T). Write Note that by time homogeneity F( I(*, S, T; T) = F( S, T - 7). Then conditioning on the next event to occur in the epidemic starting in the state S,

(4 The first term corresponds to the individual I, becoming infected, the second to the individual I,, being immunized, and the third to no event happening in [0, t). We can similarly write down a corresponding expression for F( u’, S, T) by replacing F( u, Sp, T,

[ tp + y,(T)]}]

h.

Using the induction hypothesis applied to the states S, and SP, we deduce that this change increases the expression. So u’ is a better policy than u. Recall that u” applies the same amount of treatment effort as u’, but applies it all to I,, the most susceptible person in S. We can write down a corresponding expression for F( u”, S, T) by replacing F( u*, Sp, T - T) in

HETEROGENEOUSLY

MIXING COMMUNITY

35

(5) with F( u*, S,, T - 7):

F(u”,S,T)=

~T(~~~{~~F(u*,SP,T:7)+uI’(~)F(u*,S~,T;i)}) I Xexp(

+imm(S)exp

-/,‘(

,I,

-lzAIS,+u,(v)]

[ & + up(T)]]]

By Corollary 2 this increases the expression. Thus u” u’. It remains to show u” is a worse policy than Lemma 4, which completes the proof of Theorem policy u* maximizes the expected number of people LEMMA

dg) dT]

dT.

is a better policy than u*. This follows from 3 and shows that the immunized.

4

With the notation of Theorem 3, F( u”, S, T ) 6

F(u*,S,T).

Proof. The policies u” and u* agree unless the epidemic is in the state S. If they also agree there, then the result is trivial. Otherwise there is some interval [t,, tz) where they disagree, and they agree on [t,, T]. Define a policy u”(s) which agrees with u” on [0, s) if the epidemic is in the state S and otherwise agrees with u*. We shall show that if t > 0 and At is sufficiently small and positive, then F(u”(t-At),S,T) It is straightforward monotonic decreasing

>F(u”(t),S,T).

to show that this implies function of t and thus

F( u”(O), S, T) = F( u*,S,T)

> F(u”(T),S,T)

that

(6) F( u”(t), S, T) is a

= F(u,S,T).

So consider the inequality (6). Note that u”( t - At) and u”(t) agree provided that the time lies between 0 and t - At. Hence we can condition on the state of the epidemic at time t - At when this common policy is used. Unless this state is S, the policies also agree from time t - At onwards. Thus to prove the inequality (6) it is sufficient to show that if the epidemic is in the state S at time t - At, then the expected number of people immunized between time t - At and time T is greater if the policy u”(t - At) is used than if the policy u”(t) is used; in other words, E[w(T),u”(t-At):S,t-At]

aE[~~~(T),u”(t):s,t-At].

DAVID GREENHALGH

36

Suppose that if the epidemic is in the state S, then U” applies an amount v( < uO) of treatment effort at time t. The policy u* applies the maximum amount uu. Conditioning on the events that can occur in [t, t - At), we deduce that

E[IMM( T), u”( t - At) : S, t - At]

1-uAt-

c

&At

E[m~(T),u*:S,t]+o(At).

(7)

PEA

The first term corresponds to an individual Z, becoming infected, the second to the individual Z, being immunized, and the third to no events occurring in [0, t). We can perform a similar decomposition of E[IMM(T), u”(t) : S, T - At], just replacing u in (7) by ua. Subtracting these two expressions, we deduce that E[IMM(T),~“(~-A~):S,~-A~~]-EE[IMM(T),~”(~):S,~-A~] =(u,-u)At{E[m(T),u*:S,t]-E[[IMM(T),u*:S~J]} + o( At). By the second part of Corollary 2 this is negative if At is small enough. This completes the proof. It is straightforward to prove that a necessary and sufficient condition for u to be an optimal policy is that u applies full treatment effort to the group of susceptible people with the highest value of X whilst there are still susceptible people left. Any other policy is strictly worse than this one. This result remains true if the constraint function is a piecewise continuous bounded function rather than a constant and if the policy u can depend on the history of the epidemic up to time t. MINIMIZING

THE

EXPECTED

NUMBER

OF PEOPLE

INFECTED

Suppose that the epidemic starts in the state S and is to be controlled to minimize the expected number of people infected at time T. A necessary and sufficient condition for a policy u to be optimal is that whenever the disease is still spreading, u applies full treatment effort to the most susceptible person. This is proved in the same way as the corresponding result for maximizing the expected number of people immunized.

HETEROGENEOUSLY

4.

CONTROL

MIXING COMMUNITY

37

BY REMOVALS

We next consider an epidemic model with no immunization. Here the population is divided into classes consisting of susceptible, infected, and removed individuals. The disease is controlled by removing infected people and isolating them from the rest of the population as described earlier. Let r* be the policy which always applies full removal effort to the most infectious infected person. This is an obvious candidate for the optimal policy. We consider three criteria for controlling the epidemic: (i) to minimize the expected number of people ever infected (including those removed), (ii) to minimize the expected number of people currently infected, and (iii) to maximize the expected number removed. Whilst the motivation for the first two objectives may seem clear, the third objective needs a little explanation, which is given later. Our results here are less complete than for the corresponding case of controlling the epidemic by immunization. MINIMIZING

THE EXPECTED

NUMBER

OF PEOPLE

EVER INFECTED

First consider trying to minimize the number of people who have ever been infected at time T, or equivalently trying to maximize the number of susceptible people at time T. Consider the stochastic homogeneously mixing epidemic. We suppose that both the infection rate X and the removal rate p are strictly positive. Here r* is the optimal policy. These results are proved in the same way as Theorem 3, although the same method does not work for the heterogeneously mixing epidemic. Any policy r for which there is a possibility that the disease is still spreading and P applies less than full removal effort is strictly worse than +rr*. For the heterogeneously mixing case it remains open whether ?r* is optimal for all values of T. For this case we could in fact prove that the policy # was optimal only for the infinite time horizon (although we suspect that the result will be true for both the finite and the infinite horizon cases). As stated above, the model which we have described is not directly suitable for the infinite time horizon, as it ignores births and deaths in the population. Hence this result may indicate that the policy r* is very nearly optimal if the average infection and immunization times are small compared with the time for which the epidemic is controlled, but this in turn is small compared with the average lifetime of the population in which the disease is spreading. In any case this result supports the plausibility of the conjecture for the finite time horizon. The infinite time horizon result follows from an intermediate lemma in a similar way to that in which Theorem 3 followed from Corollary 2. The technique used to prove this lemma is different to the technique used previously in that it depends explicitly on the product form X,h, of the

DAVID GREENHALGH

38

infection rate. The statement and proof of this lemma are not given here; the interested reader is referred to Greenhalgh [ll]. We briefly mention controlling the epidemic so that as few people as possible are infected at some finite terminal time. Here T* is the optimal policy for the homogeneously mixing epidemic, and a necessary and sufficient condition for a policy to be optimal is that it should apply full removal effort whenever there is at least one infected person. This is proved similarly to the first results of the paper. Again it seems intuitively reasonable that the policy +z* should also be optimal for the heterogeneously mixing case, but as yet this problem remains open. MAXIMIZING

THE EXPECTED

NUMBER

REMOVED

To complete this section of the study we now discuss controlling the epidemic so as to maximize the expected number of people removed at some terminal time. At first sight this is an unusual objective, as it may promote infection in the short term. However, if an epidemic is being treated for a limited time, then it may be preferable for individuals to catch the disease when they can be treated than later when they cannot. Once treated, they are cured and acquire permanent immunity to the disease. Hence this objective function will maximize the expected number of people with immunity to the disease. This is beneficial in two ways: first, it maximizes the number of individuals with permanent immunity when treatment is stopped, and second, this mass of immune people may act to slow down the future spread of the disease. To illustrate this, consider an epidemic such as German measles or mumps spreading through a class of schoolchildren of roughly similar age. Here immunization may well be inconvenient or expensive, and the effect of catching such diseases is often much worse in adult life than in childhood. If the disease spreads through the school in a short time (so that age effects can be ignored), then under these circumstances it may be reasonable to try to maximize the expected number of people removed. (In fact a similar objective is used on a nationwide basis for German measles, but justification of this requires an endemic model with age structure.) We first require the following lemma, which is analogous to Corollary 2 for the case of control by immunization. LEMMA

5

Let p* denote the policy which maximizes the expected number of people removed by time T > 0. Suppose that Ii is a susceptible, and Ik and I, are infected people in the state S. Then if A, > A,,

(i) E[REM(T),P* : Sk] 2 E[REM(T),~* : S,] with equality if and on& if A, = A, or S contains no susceptibles, and (ii) E[REM(T), p* : S’] B E[REM(T), p* : S].

HETEROGENEOUSLY

MIXING COMMUNITY

39

Interpretation. These inequalities say that (i) the expected number of people removed at time T is greater if the epidemic starts in the state obtained from S by removing the less infectious individual Ik than in the state obtained from S by removing I,, and (ii) the expected number of people removed at time T is greater if the epidemic starts in the state obtained from S by Z, becoming infected than if it starts in the state S. Note. Here we are implicitly assuming that such a policy exists. It is possible to prove by induction that p* exists and has the properties that we claim. Proof. (i): If A, = A, then the result is clearly true with equality. Suppose that A, < A,, and consider the epidemic starting in the state S,. Label each infected person as “red.” Each new person infected from Ik is “red” with probability (Y and “blue” with probability 1 - (Y, where OL= A, /A,. Each new person infected from another “red” person is labeled red, and “red” people may infect “blue” people, turning them red just as if they were susceptible. “Blue” infecteds infect susceptibles similarly, turning them “blue.” Thus the larger epidemic of red and blue infected people is the same stochastic process as the epidemic starting in the state S,, and the smaller epidemic of red infected people is the same stochastic process as that starting in the state S,. So given a policy to use in the epidemic of red infected people, this policy defines a policy p’ to use in the larger epidemic with

E[REM(T),~*:S,]

=E[REM(T),~‘:&]

OE[-(T),p*:&],

as p* is the optimal policy for the epidemic starting in the state S, There is a slight technical difficulty in that the policy p’ may depend on random variables independent of the current state of the epidemic, which can be overcome by allowing this in the definition of a permissible policy. Moreover, if A, > A, and S contains susceptible people, then this inequality is strict which proves (i). The result (ii) is proved similarly. We can use the above lemma to show, as in the proof of Theorem 3 from Corollary 2, that the optimal policy p* to maximize the expected number of people removed at time T is characterized by a series of switching times, alternately applying no removal effort and full removal effort to the least infectious infected person in the epidemic and ending by applying full removal effort. It should however be pointed out that there may be practical difficulties in implementing such a policy, particularly for a disease such as influenza which can sweep through a city in six weeks.

DAVID GREENHALGH

40

This ends the main results section of the paper. In the next section of the paper we shall examine the circumstances when these results can or cannot be extended. 5.

EXTENSIONS

OF RESULTS

AND COUNTEREXAMPLES

In this section we shall briefly discuss the extensions which are possible to the results presented in this paper. First we discuss the situation where the disease is controlled simultaneously by immunization of susceptible people and removal of infected people. Then we discuss a possible modification of the model for the spread of the disease, and finally we discuss the problems involved in generalising the immunization time distribution. CONTROL.

BY IMMUNIZATION

AND

REMOVALS

First consider the model for a disease in a heterogeneously mixing community due to Cane and McNamee [4], and suppose that this disease is controlled independently both by immunization of susceptibles and by removal of infected people. Suppose that the epidemic is controlled either so as to minimize the expected number of people infected at some terminal time T, or to minimize the expected number of people ever to have caught the disease by time T. It might have been expected that for this situation the optimal policy is to apply full treatment effort to the most susceptible individual in the epidemic and full removal effort to the most infectious infected individual in the population. However if one attempts to prove this along the lines of the proof of Theorem 3, then one runs into difficulties which are analogous to the difficulties encountered when one considers the situation with control by removals only. On the other hand, it is possible to prove these results for the simpler homogeneous mixing case, even though these results appear fairly obvious. THE

SEMIFiOMOGENEOCJSLY

MIXING

EPIDEMIC

The model of Cane and McNamee [4] assumed that each individual was assigned a parameter X, corresponding to the proportion of his time that he spends in some communal meeting place. However, a person would be more likely to mix with the rest of the population when he was well than when he was ill. An obvious way to model this would be to assign each person two parameters X, and X,. If 1, is infected and Z, is susceptible, then Z, will infect I, after a time which has the exponential distribution with parameter X,:h,. The parameter X, corresponds to the proportion of his time that a person spends in the communal meeting place when he is well, and X, to the proportion of his time that he spends in this communal meeting place when he is ill.

HETEROGENEOUSLY

MIXING

41

COMMUNITY

Suppose first that the epidemic is being controlled by removing infected people so as to either minimize the expected number of people infected at time T or minimize the expected number of people ever to have been infected at time T, The same techniques can be used to show that the results are true provided only that the parameters X,, h,, . . . , X, are equal. This is appropriate if the people who are ill and not mixing with the rest of the population are more likely to behave similarly than the rest of the population. For controlling the epidemic by immunization, suppose that we are either trying to maximize the number of people immunized or minimize the number of people infected. LEMMA 6 Consider controlling the epidemic with any of the above objectives. Suppose that I1 and I2 are two susceptibles in S with X, < X, and X; < X,. Then the outcome

of the epidemic

which starts in the state obtained from S by immuniz-

ing I, is preferable to the outcome of the epidemic obtained from S by immunizing Il.

which starts

in the state

This lemma is proved like Lemma 5. As before, we deduce that if the epidemic starts in the state S it is better to treat person I2 than person Ii. However, the following counterexamples show that the optimal policy to minimize the expected number of people infected at some terminal time T must take account of the values of both X and x’ and therefore has a more complicated form. Counterexample I. Suppose that an epidemic is spreading amongst three people. In the starting state S, I1 is infected and I2 and I3 are susceptible. The values of (X,x’) for these three individuals are (l,O), (3,6), and (4,0) respectively. The doctor who is treating the epidemic can immunize people at rate ZL.The expected number of people infected at time T if the epidemic starts in the state obtained from S by immunizing I2 is zero, whilst the expected number of people infected at time T if the epidemic starts in the state obtained from S by immunizing Z3 is [6/(6 + p)][l - exp{ - (6 + p)T }]. Thus when the epidemic is in the state S, it is preferable to treat individual I, rather than individual 13. Hence it is not always optimal to treat the most infectious person (i.e. the infected person with the largest value of X). Counterexample II. The epidemic is spreading amongst four people: I,, who is infected, and I,, Z3, and 14, who are susceptible in the starting state S. The values of (X, 2) for these individuals are (l,O), (0, l), (0, l), and (K, 1 - S) respectively, where K is large and 6 is small and positive. The doctor who is controlling the epidemic can immunize people at rate CL.Note that if the epidemic starts in the state obtained from S by immunizing I,,

42

DAVID GREENHALGH

then the optimal policy next treats Z3 and then Z4, whilst if the epidemic starts in the state obtained from S by immunizing Z,, then the optimal policy next treats either Z2 or Z3. We find that if K and T are large enough and 6 is small enough, then the expected number of people infected at time T if the epidemic starts in the state obtained from S by immunizing Z, is less than the expected number of people infected at time T if the epidemic starts in the state obtained from S by immunizing Z2. Hence it is preferable to treat individual Z4 rather than individual Z, if the epidemic is in the state S. Thus it is not always optimal to treat the most susceptible person in the epidemic (i.e. the person with the largest value of X). Note that the situation described in Counterexample II is in some sense the worst possible case, as the infectiousness of Z, is very much larger than that of anyone else in the epidemic, whilst his susceptibility is just a little bit smaller than that of Z2 and Z3. The situations described in Counterexamples I and II also can be used to show that the optimal policy to maximize the expected number of people immunized depends on the relative values of X and A’.

GENERALIZING

THE IMMUNIZATION

TIME

DISTRIBUTION

Consider the model for an epidemic in a heterogeneously mixing population which was considered at the start of the paper and is controlled by immunization only. The assumption was made that a susceptible patient would be immunized after being treated for an exponentially distributed immunization time. However, if an alternative distribution is assumed, the problem immediately becomes much more complicated. This is because it is no longer possible to specify the state of the epidemic by knowledge of those individuals who are susceptible, infected, and immune. The time for which the current patient has been treated must also affect the decisions concerning the control of the disease. A perhaps more realistic assumption would be to assume that a susceptible patient would be immunized after being treated for a constant immunization time. For this situation the policy u* which applies full treatment effort to the most susceptible individual is no longer optimal if we wish to maximize the expected number of people immunized. This is because this person also has a high probability of catching the disease. To illustrate this, consider the situation where the disease is spreading amongst three individuals: Zi, who is infected, and I, and Z,, who are susceptible. The parameters of infectiousness and susceptibility associated with these individuals are hi, A,, and h, respectively, where h, > A,. The disease is controlled so as to maximize the expected number of people immunized at time T, where T > 0 is the constant immunization period. Then if individual I2 is treated first the expected number of people im-

HETEROGENEOUSLY

MIXING COMMUNITY

43

mtmized is E, = exp( - X,X,T), whereas if individual I3 is treated first, the expected number of people immunized is E3 = exp( - X,h, T). As E3 > E, , we deduce that it is not always optimal to treat the susceptible person with the largest value of A. It is at the moment open whether the policy u* is optimal if the objective is taken to be to minimize the expected number of people infected at time T. FURTHER

MODIFICATIONS

Recall the results of Longini et al. [5]. They considered a model for influenza which is similar to the model discussed in this ‘paper, with the population divided into distinct age groups. Longini et al. consider two different types of objective function and use a computational optimization technique to determine the optimal policy. Either objective function corresponds to assigning different costs to individuals in different age groups catching the disease. There are two conflicting factors in the optimal policy. The first is a tendency to slow down the spread of the disease by immunizing those age classes who spread the disease fastest, and the second is a tendency to directly reduce costs by immunizing those people who would incur a high cost were they to catch the disease. The model presented here is simpler than that studied by Longini et al. in that it has assumed that the cost of each person catching the disease is the same, and hence it does not capture the second trend. It is straightforward to show that if the model presented in this paper is modified so that a different cost is incurred for each different individual who catches the disease, and the objective function is taken to be to minimize the expected total cost, then it is no longer always optimal to apply full immunization effort to the most susceptible individual Instead there is a similar conflict to that described above, between trying to immunize those susceptible individuals who would spread the disease fastest and those susceptible individuals who would incur the highest cost were they to become infected. 6.

SUMMARY

AND DISCUSSION

This paper has used applied probability and operational research techniques to find optimal control policies for a disease spreading in a heterogeneously mixing population. The major conclusions are as follows: For the case of control by immunization only, the optimal policy is to apply full immunization effort to the most susceptible people in the epidemic. This is true whether one is trying to maximize the expected number immunized or minimize the expected number infected. For the case of control by removals only, consider trying to (a) minimize the expected number of people infected at some terminal time or (b) minimize the expected number of people ever to catch the disease by this time. In this case it is conjectured that the optimal

44

DAVID

GREENHALGH

policy is to apply full removal effort to the group of the most infectious infected individuals in the epidemic. This has been proved for the homogeneous mixing case, and for the heterogeneous mixing case with an infinite time horizon and objective (b). For the heterogeneous mixing case suppose that it is desired to maximize the expected number of people at some terminal time. Then the optimal policy is characterized by a series of switching times, alternately applying no removal effort and applying full removal effort to the least infectious infected person in the epidemic. One question that remains unanswered is the finite time horizon problem for controlling the heterogeneously mixing epidemic by removals, and whether or not n* is optimal here for either minimizing the expected number of people infected at the end of the epidemic or minimizing the expected number of people who have ever caught the disease at the end of the epidemic. We feel that it would be interesting to resolve this numerically. Consider the epidemic controlled by immunization only. It is more usual to assume that vaccination of the population takes place in advance, so that immunity is developed before the epidemic starts [5]. For this situation it is sensible to consider minimizing the expected number of people infected. The epidemic proceeds with no immunization, and so we take u0 = 0. Then the result which corresponds to Corollary 2 indicates that the optimal policy is again to immunize those people with the highest value of X (who would spread the disease fastest). In practice it would of course be difficult to measure the parameters Xi and also difficult to determine the precise state of the epidemic at each instant. However, it might be useful if the people who are most prone to infection and the people who will infect the most others were identified, as our results show that these are precisely the people who should be immunized or removed first. A related point, and one which is taken into consideration in most of the related work on this subject, is that as infected people become fewer it becomes harder to find and isolate them. Another important factor to take into account is the cost of treatment. Our model has ignored these factors. This work was funded by the S. E. R. C. We are grateful to Eddie Anderson and Richard Weber for much discussion and criticism of this work and to Roy Anderson for comments on the manuscript.

REFERENCES 1 2

N. T. J. Bailey, The Marhemarical Theory of Infectious Diseases, 2nd ed.. Macmillan, New York, 1975. R. K. Watson, On an epidemic in a stratified population, J. Appl. Prohah. 9:659-666 (1972).

HETEROGENEOUSLY S. Rushton

MIXING

and A. J. Mautner,

45

COMMUNITY The deterministic

simple epidemic

for more than one

community, Biometriku 42:126-132 (1955). V. R. Cane and R. McNamee, The spread of an infection in a heterogeneous population, J. Appl. Probub. 19A:173-184 (1982). I. M. Longini Jr., E. Ackerman, and L. R. Elveback, An optimisation model for influenza A epidemics, Math. Biosci. 38:141-157 (1978). R. M. Anderson and R. M. May, Spatial, temporal, and genetic heterogeneity in host populations and the design of immunisation programs, IMA J. Math. Appl. Med. Bid. 1:233-266 (1984). R. M. May and R. M. Anderson, Spatial heterogeneity tion programs, Math. Biosci. 72:83-111 (1984).

8 D. Schenzle, An age structured 9

10 11

and the design of immuniza-

model of pre- and post-vaccination measles transmission. IMA J. M&h. Appl. Med. Biol. 1:169-191 (1984). 0. V. Baroyan, L. A. Genchikov, L. A. Rvachev, and V. A. Shaskov, An attempt at large scale influenza modelling by means of a computer, Bull. Internut. Epidentiol. A.woc. 18:22-31 (1969). K. Wickwire, Mathematical models for the control of pests and infectious diseases: A survey. Theoret. Populution Biol. 11:182-238 (1977). D. Greenhalgh, Stochastic models for control of epidemics, Ph.D. Dissertation, Cambridge

Univ., 1984.