- Email: [email protected]
Refinements of Paltiel and Kaplan's Decision-theoretic model of AIDS clinical trials
Socio-Econ. Plann. Sci. Vol. 31, No. 2, pp. 87-101, 1997
~ ) Pergamon
PII: S0038-0121(96)00030-4
© 1997 ElsevierScience Ltd All rights reserved. Printed in Great Britain 0o38-Ol21/97 $17.oo+ o.oo
Refinements of Paltiel and Kaplan's Decision-theoretic Model of AIDS Clinical Trials ROBERT A. BOSCHI~" and NATASHA K. STOUT 2 tOberlin College, Department of Mathematics, King Building 205, 10 North Professor St, Oberlin, OH 44074-1019, U.S.A. 2Epic Systems, Madison, WI 53707, U.S.A.
Abstraet--ln this paper we present three decision-theoretic models for determining, amongst all of the courses of action available to a government agency such as the Food and Drug Administration at a certain stage of its examination of a new AIDS drug, the one that places the least economic burden on society. Our models are based on a model devised by Paltiel and Kaplan. With their model, Paltiel and Kaplan concluded that the agency's optimal course of action depends on its initial estimate of a drug's efficacy: A drug that appears to be not-at-all promising should be rejected immediately. A drug that appears to be very promising should be accepted immediately. All other drugs should be put through a clinical trial. The length of a clinical trial should depend on the agency's estimate of the drug's efficacy (the more promising the drug, the shorter the clinical trial). With our models, which are structurally more realistic than Paltiel and Kaplan's model, we obtained mostly similar results. The differences we observed have to do with the relationship between the optimal length of a clinical trial and the preliminary estimate of the drug's efficacy. © 1997 Elsevier Science Ltd
INTRODUCTION Before a new AIDS drug is submitted to the Food and Drug Administration (FDA) for approval, it is put through a three-phase examination process. In the first phase of this process, the only concern is safety. In this phase, researchers look for side effects, estimate toxicity, and determine safe dosages. Usually, the Phase I clinical trial has no more than 20 participants and lasts for less than a year. Once Phase I is completed, the focus of the examination process shifts from safety to performance. In the remaining two phases, researchers obtain estimates of the drug's efficacy--preliminary estimates in Phase II and final estimates in Phase III. Usually, the Phase II and Phase III clinical trials have several hundred and several thousand participants, respectively, and last upwards of two and four years. In a recent article [1], Paltiel and Kaplan devised a decision-theoretic model for determining, amongst all of the courses of action available to a government agency such as the FDA at the end of Phase II, the one that places the least economic burden on society. Their model (which we will refer to as PK) allows for the agency to reject the drug at the end of Phase II, to accept the drug at the end of Phase II, or to postpone the decision to reject or accept until after the drug has been put through a single Phase III clinical trial. PK allows for the agency to use the information gathered in Phase II to help it select the length of the Phase III clinical trial, but not to help it choose the number of Phase III participants. Paltiel and Kaplan conducted a number of numerical experiments with PK. For the most part, their results confirm what one's intuition might expect: Which policy is optimal depends on the preliminary (i.e. Phase II) estimate of the drug's efficacy. A drug that appears to be not-at-all promising should be rejected immediately. A drug that seems very promising should be accepted immediately. A drug that fits into neither of the previous two categories should be put through tAuthor for correspondence. 87
88
Robert A. Bosch and Natasha K. Stout
a Phase III clinical trial, and the length of this trial should depend on the preliminary estimate of the drug's efficacy. In our opinion, Paltiel and Kaplan's results are counter-intuitive in only one respect: the specific relationship between the optimal length of a Phase III clinical trial and the preliminary estimate of the drug's efficacy. Their results indicate that the more promising the drug appears to be at the end of Phase II, the shorter should be its Phase III clinical trial. We were surprised by this. We would expect the optimal length of a Phase III clinical trial to be short for not one, but two classes of drugs: drugs that seem promising at the end of Phase II but do not seem promising enough to warrant immediate acceptance, and drugs that seem to be a failure at the end of Phase II but not enough of a failure to warrant immediate rejection. In this article, we present three refinements of PK, each one structurally more realistic than the original. (For the most part, we did not alter the values of the numerous parameters of PK.) In our two-trial model (TT), we allow for the drug to be rejected at the end of Phase II, accepted at the end of Phase II, or for the decision to reject or accept to be postponed until after one or two Phase III clinical trials have been performed. In PK, at the end of the Phase III clinical trial, the drug must be rejected or accepted. In practice, however, when the results of a Phase III clinical trial are inconclusive, the drug might be put through another Phase III clinical trial. In our second refinement of PK, our treatment-control model (TC), we allow for the standard statistical practice of dividing the participants of the Phase III clinical trial into a treatment group and a control group. The participants who are assigned to the treatment group are given the drug being tested. Those who are assigned to the control group are given either a placebo or an already approved drug. In PK, there is no control group; all participants of the Phase III trial are given the drug being tested. Our third and final refinement of PK is simply a combination of TT and TC. In TT/TC, we allow for the drug to be rejected at the end of Phase II, accepted at the end of Phase II, or for the decision to be postponed until after one or two Phase III clinical trials. In each trial, the participants are divided into a treatment group and a control group. This article is organized as follows: The first section is devoted to a discussion of the assumptions and mathematics that underlie Paltiel and Kaplan's model, and our two refinements of it. The next four sections contain descriptions of PK, TT, TC, and TT/TC. In the final section, we describe the numerical experiments conducted to compare the four models, and then suggest conclusions that can be drawn from the results. PRELIMINARIES Paltiel and Kaplan designed their model for the purpose of examining the epidemiological and economic consequences of AIDS clinical trials. Realizing that it would be difficult (if not impossible) to construct a simple model of how certain policies would affect the entire population of the United States, they decided to focus on how these policies would affect a much smaller and more homogeneous group of people: a population of sexually active homosexual males. In this section, we review how Paltiel and Kaplan modeled the effects of two types of policies on this population: policies that allow the AIDS epidemic to run its course, and policies that involve the implementation of screening and treatment programs. We refer the reader who desires more detailed explanations of this material to Ref. [1] (pp. 181-183). Readers who wish to examine the work that forms some of the foundations of Paltiel and Kaplan's work are referred to [2-4].
The population Let X(t), Y(t), and Z(t) denote the numbers of susceptibles, unidentified seropositives, and identified seropositives, respectively, at time t in a certain population of homosexual males who engage in unprotected anal intercourse. A population member is considered to be a susceptible if he is uninfected with the HIV virus, a seropositive if he is infected but asymptomatic, an unidentified seropositive if he is unaware that he is infected, and an identified seropositive if he has learned (by means of a blood test) that he is infected. We make the following assumptions: • New members enter the population at the rate of U per year. All new members are susceptibles. • Members leave the population owing to AIDS-related and non-AIDS-related causes. An
Decision-theoretic models of AIDS clinical trials
• • • • •
89
AIDS-related departure takes place whenever a seropositive progresses to AIDS. A non-AIDS-related departure takes place whenever a population member dies from non-AIDS-related causes. For seropositives who receive no treatment, the AIDS-related departure rate is y per person per year, the natural AIDS incubation rate. The non-AIDS-related departure rate is # per person per year. The only way that a susceptible can contract the virus is to have sexual intercourse with a seropositive partner. Each member of the population has c distinct sexual partners per year. The probability is fl that a susceptible will contract the virus from one particular seropositive partner.
Allowing the epidemic to run its course We now consider how the population would be affected if the AIDS epidemic were allowed to run its course from time t = 0 (the present) until time t = h (the end of the time horizon). We assume that during this time period, no effort is made to identify or treat seropositives. Thus, the only members of the population who are given any care whatsoever are the members who have progressed to AIDS, and they are treated for symptoms of AIDS and for secondary infections. We also assume that no new infections occur after time t = h. (Although one need not do so, one can think of time t = h as the time at which a vaccine is introduced.) It can be shown that under these assumptions and the assumptions described in the previous subsection, the following system of differential equations describes how the population changes over time:
dX dt - U dY
cflXY
I~X
(1)
(~ + y) Y
(2)
X+ Y
cflXY
-d-[ = x + r
Since there is no screening for the HIV virus, Z(t) = 0 for all 0 ~< t ~ h. It is easy to show that if the epidemic is allowed to run its course, the cost to society will be given by ~ ( y ) : = CARDS7i ~ Y(t)dt + CA,DS"~--~Y "Y(h), where CARDSis the cost of providing medical care to one AIDS patient for the rest of his life and Y(.) is obtained by solving (1) and (2). The first term of ~ ( y ) represents the cost of caring for the members of the population who progress to AIDS before time t = h. The second term represents the cost of caring for those who progress to AIDS after time t = h. It is also easy to show that if the decision to let the epidemic run its course is made not at time t - - 0 but at time t = t*, the moment at which an experimental drug emerges from the drug examination process, then the cost to society will be given by C/(y): = C°(y) + CCL,N,ct*, where CcuN~C is the cost of conducting a clinical trial for one year.
Implementing screening and treatment programs We conclude this section by considering how the population would be affected if at time t = t* a new AIDS drug is accepted, a screening program is put into effect, and all members who are thus identified as seropositives are given the drug. We assume that the drug affects the AIDS incubation rate by changing it from y to 6, and the infectivity of the virus by changing it from fl to Off, where 0 = 6/7. (We refer readers who desire an explanation of the reasonableness of the latter assumption to Ref. [1] (p. 181).) We also assume that in the screening program, each susceptible and unidentified seropositive is tested v times per year for the virus. It can be shown that under these assumptions and under the assumptions described in the first subsection of this
90
Robert A. Bosch and Natasha K. Stout
section, the following system of differential equations describes how the population changes over time:
dX dr-
cflX(Y + OZ) X+ Y+Z
U
d Y cflX(Y + OZ) dt - X + Y + Z
/iX
(l~+7+I,.z)Y
dZ -d-[ = I,.z Y - (# + 6)Z.
(3)
(4)
(5)
Here, it is understood that Z(t) = 0 for all t ~< t* and that L. is a function of t that takes on the value 1 if t > t* and 0 otherwise. Under the assumption that the drug is put through a clinical trial from time t = 0 until time t = t*, it can be shown that the cost to society is given by
C;(]~, 6): =
CCLIN,Ct*
+ CTESTZ (X(t) + Y(t))dt + CrEsT(X(h) + Y(h))
+ CDRUGfl + CMDS 7
Z(t)dt +
r(t)dt + 6
CDRUG"~--~~ "(Y(h) -~- Z(h))
Z(t)dt
+ CARDS"~ ' - ~ "(r(h) + Z(h)),
where CCLINICis the cost of conducting a clinical trial for one year, CTESTis the cost of screening one individual for the virus, CDRUCis the cost of giving the drug to one person for one year, and X(.), Y(.), and Z(.) are obtained by solving (3), (4) and (5). The first term of C~(7, 3) represents the cost of conducting a clinical trial from time t = 0 to time t = t*. The second and third terms represent the cost of running the screening program; the fourth and fifth terms, the cost of administering the drug to identified seropositives; and the sixth and seventh terms, the cost of providing medical care to the members of the population who progress to AIDS. The reader should note that we assume that at time t = h all unidentified persons are tested. The reader should recall that all seropositives are given the drug from the moment they are identified until the moment they leave the population, and that those seropositives who leave the population owing to AIDS-related causes are treated for symptoms of AIDS and secondary infections. Finally, the reader should note that we assume that C*tos is the same for every AIDS patient; it does not depend on if, or on how long, the patient received the drug. PALTIEL AND KAPLAN'S MODEL Suppose that a new drug has been developed for reducing the AIDS incubation rate--the per capita rate at which HIV-positive, AIDS-asymptomatic individuals progress to AIDS. The sponsor of the drug conducts a Phase I clinical trial and judges the drug to be safe. The sponsor then puts the drug through a Phase II clinical trial and obtains preliminary estimates of its efficacy (the AIDS incubation rate it produces). Paltiel and Kaplan consider the problem of determining, amongst all the courses of action available to a government agency such as the F D A at this point in time, the one that places the least economic burden on society. Their approach to the problem is displayed in the decision tree in Fig. 1. Paltiel and Kaplan assume that at the end of Phase II, the agency has three options--to reject the drug immediately, to accept the drug immediately, or to require the sponsor to conduct a single Phase III clinical trial. They allow the agency to select the Phase III trial length (T), but not the number of Phase III participants (n). They assume that at the beginning of the trial, all of the
Decision-theoretic models of AIDS clinical trials
91
participants will be HIV-positive and AIDS-asymptomatic; that for the duration of the trial, all of the participants will be given the same dosage of the drug; and that at the end of the trial, the results (denoted A'" when T = t*) will be expressed in terms of the number of participants who progressed to AIDS. They also assume that at the end of the trial, the agency must either reject or accept the drug.
Rejecting the drug immediately Paltiel and Kaplan assume that the drug being tested is the 'only hope' of AIDS researchers. Rejecting the drug, therefore, is tantamount to allowing the AIDS epidemic to run its course. Paltiel and Kaplan also assume that y, the natural AIDS incubation rate, is known with certainty. Thus, if the agency rejects the drug immediately, the expected cost to society will be CR: = CR(?).
(6)
Accepting the drug immediately Paltiel and Kaplan assume that acceptance of the drug is followed immediately by implementation of screening and treatment programs. Thus, if the agency accepts the drug immediately and the drug has efficacy 3, the cost to society will be ~ ( ? , 3). However, the agency does not know the drug's efficacy at the end of Phase II; at this point in time, the sponsor has only preliminary estimates of the drug's efficacy. To account for this, Paltiel and Kaplan assume that the drug's efficacy is a random variable A with known density function fa('). Accordingly, if the agency accepts the drug immediately, the expected cost to society will be
CA: =
C°A(V,6)fA(6)d6.
(7)
Conducting a phase III clinical trial Paltiel and Kaplan assume that at the end of a Phase III clinical trial, the agency must either reject or accept the drug. Thus, if the agency requires the sponsor to conduct a Phase III clinical trial of length t* and observes that a of the n participants progressed to AIDS during the trial, the expected cost to society will be
the minimum of the expected cost of rejecting the drug after performing the trial and the expected cost of accepting the drug after observing a of the participants progress to AIDS during the trial. Thus, if the agency requires the sponsor to conduct a Phase III clinical trial of length t*, the expected cost to society will be
C~;,: = ~ ¢,.(a)P(A"= a). a~O
Reject
C~. ('y)
~ Accep-~ tA =
6) Reject
Phase III Trial
C~t*('y)
Accep-~ t
&=6[A
k.
Fig. 1. Decisiontree for PK.
t"
=
a
t* cA (v, 6)
Robert A. Bosch and Natasha K. Stout
92
~
Reject C°(7) /Accept
~A=6
cO (7,
Trial First
~ - \;
tl ~
Phase III - ~ - 2"1 =
at
6) /Reject A'I~ = a l - i ~ A c c e p t
\
L
C ~ (7)
zx=61£1
s
(7, 6)
-
~,Phase
III',
,,_T_~. . . . . ;
Fig. 2. Decisiontree for TT.
Finding the optimal policy To determine which policy is optimal, it is necessary to compute CR, CA, and C~ for each possible trial length t*. If CR is the smallest of these costs, the optimal policy is to reject the drug immediately. If CA is the smallest, the optimal policy is to accept the drug immediately; and if C~, is the smallest, the optimal policy is to conduct a Phase III clinical trial of length t* and make the decision to reject or accept when the results are available. Probabilities Suppose that the drug as efficacy 6. Let p,.(6) denote the probability that an HIV-positive, AIDS-asymptomatic individual who is taking a drug of efficacy 6 progresses to AIDS during a trial of length t*. Since each of the n participants of the Phase III clinical trial is given the drug for the duration of the trial, and since each of the participants will progress to AIDS independently of the other participants, P(A'" = a) =
;(:)
p,.(f)a(1 - p,.(6))" - °fA(f)d6.
(9)
Paltial and Kaplan assume that AIDS-related and non-AIDS-related departure times are independent and are exponentially distributed with parameters 6 and p, respectively. Using this assumption, it is easy to show that
p,,(6) = ~
6
(1 -- e -(* +~)").
THE TWO-TRIAL MODEL In PK, the government agency can require the sponsor of a new AIDS drug to conduct at most one Phase III clinical trial; at the end of this trial, the agency must either reject or accept the drug. However, in practice, when the results of a Phase III clinical trial are inconclusive, the drug might be put through another Phase III clinical trial. We developed TT to allow for this additional option. Our approach to the problem is displayed in the decision trees in Figs 2 and 3. Like Paltiel and Kaplan, we assume that at the end of Phase II, the agency has three options--to f-
"1
I
Reject at
T2 = t 2 I
C q +q(7~
I
g~
22=a2 AtlX--__-al
I Accept
=6 A'll=al,A2 = a 2
t.
CA~+q(%6 .d
Fig. 3. Decisiontree for the secondPhase III clinicaltrial (TT).
Decision-theoretic models of AIDS clinical trials
93
reject the drug immediately, to accept the drug immediately, or to require the sponsor to conduct a Phase III clinical trial. In TT, at the end of the first Phase III clinical trial, the agency can reject the drug, accept the drug, or require the sponsor to conduct a second Phase III clinical trial. At the end of the second Phase III clinical trial, the agency must either reject or accept the drug. Like Paltiel and Kaplan, we assume that the agency can select the length of a Phase III clinical trial, but not the number of participants. We let T~ and 7"2 denote the lengths of the first and second Phase III clinical trials, respectively, and n denote the number of participants in each trial. We assume that different participants are used in each trial; that at the beginning of each trial, each participant is HIV-positive and AIDS-asymptomatic; and that for the duration of each trial, each participant is given the same dosage of the drug. We denote the results of the first trial by A[t when T~ = t* and the results of the second trial by Air when T2 = t*. (The subscript indicates the number of the trial; the superscript, the length.) Like Paltiel and Kaplan, we express the results of a trial in terms of the number of participants who progress to AIDS during that trial. Rejecting or accepting the drug immediately
The expected costs of rejecting the drug immediately and accepting the drug immediately are CR and CA, respectively--the same as in PK. See (6) and (7). Conducting the first Phase I I I clinical trial
We assume that at the end of the first Phase III clinical trial, the agency can reject the drug, accept it, or require the sponsor to conduct a second Phase III clinical trial. Thus, if the first Phase III clinical trial lasts for t* years, the expected cost to society will be
O,r(a,)P(A[ t = a,),
Gill,. = a I =0
where
0o) the minimum of the expected costs of rejecting the drug, accepting the drug, and conducting a second Phase III clinical trial, given that the first Phase III clinical trial lasted for t* years and that a~ of its participants progressed to AIDS during that time period. A comparison of (10) and (8) shows that the 'reject' and 'accept' terms of ~,r(a~) are essentially the same as the 'reject' and 'accept' terms of q~,,(a) in Paltiel and Kaplan's model. Since the agency must reject or accept the drug at the end of the second Phase III clinical trial, reasoning similar to that used in deriving C[;~in Paltiel and Kaplan's model can be used to show that the individual terms of the 'second Phase III clinical trial' term of ¢,r(a~) are given by CxI~(t, , a~): = ~ min Cg+'r(7), a2 = 0
C2+'~(7, 6)f~lAfT.A1~(61a,,a2)d6
~.
•P(A'2 r = az]A'l t = al). Finding the optimal policy
To determine which policy the agency should pursue at the end of Phase II, it is necessary to compute CR, CA, and C~h~for each possible length t* of the initial Phase III clinical trial. If Ca is the smallest of these costs, the optimal policy is to reject the drug immediately. If CA is the smallest, the optimal policy is to accept the drug immediately. If C~t~,is the smallest, the optimal policy is to conduct an initial Phase III clinical trial of length t* and make the decision to reject or accept or conduct a second Phase III clinical trial when the results of the first trial become available.
94
Robert A. Bosch and Natasha K. Stout
Reject - ~
r=7
Accept - ~
F=7 A=;~
Phase ]]I Trial
c[(7)
T=t*
7+
f_ _ ~
Reject
F="7
CR (7)
A t. = a - I I
B" =b
/- r = T A" =a
Accept - ~
A = i B*" = b
cf (7, 7 +
Fig. 4. Decision tree for TC.
Probabilities It should be noted that even though the events A~r = a,[A = 6 and A~~ = a21A = b are independent, the events A[t = a~ and A~~ = a2 are not independent. The result of the first trial can be used to make inferences about A, which, in turn, can be used to make revisions of probabilities pertaining to the second trial.
THE TREATMENT-CONTROL MODEL In PK and TT, it is assumed that the natural AIDS incubation rate, ?, is known with certainty. This assumption eliminates the need for a control group. (If the assumption holds, then it is possible to evaluate a drug by (1) administering the drug to a group of HIV-positive, AIDS-asymptomatic individuals for a period of one or more years, (2) recording how many of them progress to AIDS, (3) using this figure to estimate the AIDS incubation rate the drug produces, and (4) comparing this rate with 7.) In this section, we describe one way of removing this assumption from PK. Our approach to the problem is displayed in the decision tree in Fig. 4. In TC, we make many of the assumptions made by Paltiel and Kaplan: that at the end of Phase II, the agency may reject the drug, accept the drug, or require the sponsor to conduct a single Phase III clinical trial; that the agency may select the Phase III trial length (T), but not the number of Phase III participants; and that at the beginning of a Phase III clinical trial, the participants are HIV-positive and AIDS-asymptomatic. We do not, however, assume that each participant is given the same dosage of the drug. Instead, we assume that n of the participants are given the drug. These participants form the treatment group. They are each given the same dosage of the drug. The remaining n participants form the control group. They are each given a placebo. (Alternately, each member of the control group can be given an already approved drug.) We let A'" and B'* denote the number of members of the treatment group and control group, respectively, who progress to AIDS during a Phase III clinical trial of length t*.
How the treatment-control model differs from Paltiel and Kaplan's model As mentioned earlier, in both PK and TT, it is assumed that the natural AIDS incubation rate, ~, is known with certainty. In TC, we replace this assumption with a much milder one: that the natural AIDS incubation rate is given by a random variable F with known density function fr('). To allow for A to be dependent on F, we assume that A = F + A, where A is a random variable with known density function f^(.) and F and A are independent. We view A as the change in the incubation rate induced by the drug. As a result of these changes, in
Decision-theoretic models of AIDS clinical trials
95
TC, the expected cost of rejecting the drug immediately and of accepting the drug immediately are no longer given by (6) and (7), respectively. Instead, they are given by c~: =
f
~(~/f~(~,)d~,
and CA: =
I
f0f0
CA(7, y + 2)fr(y)f^(2)dy d2,
respectively. The expected cost of conducting a Phase III clinical trial of length t* is then given by
C~[I:= ~ f ~,.(a,b)P(A'" = a, B"= b) a=0
b=O
where
~,,(a,b):=min{fo~C'~(~)frfB,'(Tlb)dT, fo~fo°°C'~(y,y+2)fr.A~A,'.B,'(y,21a,b)dTd2}. Finding the optimalpolicy As in PK, determining which policy is optimal involves comparing CR, CA, and all of the Cm"'s. THE TWO-TRIAL/TREATMENT-CONTROL MODEL For our final model, we combined the features of TT and TC. Like TT, TT/TC has a two-piece decision tree. (To save space, we decided against including a picture of the main piece.) The main piece resembles the decision tree for TC, but has Tts in place of Ts, A't's in place of A"s, etc. and has not two, but three branches--a 'Reject' branch, an 'Accept' branch, and a 'Second Phase III Trial' branch---extending from each node corresponding to an outcome of the first Phase III clinical trial. The second piece of the decision tree for TT/TC is displayed in Fig. 5. The notation is similar to that used in TT and TC. (The 'A' random variables give the numbers of AIDS cases observed in the treatment groups, while the 'B' random variables give the numbers observed in the control groups. Subscripts indicate the number of the Phase III trial--first or second--and superscripts indicate the length.) The costs of accepting the drug immediately and rejecting the drug immediately are exactly the same as in TC. The expression for the expected cost of postponing the decision until after conducting a (possibly initial) Phase III clinical trial of length t~* is quite long and involved, but is easy to obtain provided that one keeps in mind that (1) the expected cost of rejecting the drug after observing at and bt AIDS cases in the treatment and control
/_ q
.
, q
/
r Reject
r= lB r=bl,B; =b2
_,l,_A2=a21Al=al..~ t2
gl
UAccept-~_A=A]B;~=bl,B;
~ b,
Fig. 5. Decision tree for the second Phase III clinical trial (TT/TC).
+
96
Robert A. Bosch and Natasha K. Stout
groups of a Phase III trial lasting t~* years, and a2 and b2 AIDS cases in the treatment and control groups of a second Phase III trial lasting t2* years is given by
o~ Cff +'~(v)fr~slr.Bp(ylb~,b2)dv, and (2) the expected cost of accepting the drug under the same conditions is given by
2°°fo~C'ar÷'~(7,),+~)fr.A,~LB,r.~,~B,~(7,).la,,b~,a2, b2)dTd2.
NUMERICAL EXPERIMENTS The numerical experiments of Paltiel and Kaplan indicate that the optimal course of action for the agency at the end of Phase II depends on the Phase II estimate of the efficacy of the drug being tested. If the Phase II clinical trial is conclusive, the agency should be aggressive: drugs that seem not-at-all promising at the end of Phase II should be rejected at the end of Phase II, and drugs that seem quite promising at that point in time should be accepted at that point in time. In other words, as one's intuition might expect, if the Phase II clinical trial is conclusive, there is no need to perform a Phase III clinical trial. Paltiel and Kaplan's numerical experiments also indicate that if the Phase II clinical trial is inconclusive, the agency should put the drug through a Phase III clinical trial. The optimal length of this trial depends on how promising the drug seems at the end of Phase II: the more promising it seems, the shorter the trial. We were surprised by the final portion of this conclusion. We would expect the optimal length of a Phase III clinical trial to be directly proportional to the degree to which the Phase II clinical trial is inconclusive. That is, we would expect the optimal length of a Phase III clinical trial to be short when the Phase II clinical trial indicates (inconclusively) that the drug is effective and when it indicates (inconclusively) that the drug is ineffective, not just in the former case. We designed our numerical experiments to determine whether our three refinements of Paltiel and Kaplan's model would lead us to the same conclusions as those of Paltiel and Kaplan, or if they would lead us to conclusions that would agree more with our intuition, as discussed above. We begin this final section of the article by describing the details of our implementations of PK, TT, TC, and TT/TC. We then present our results and conclusions.
Implementing the models In order to implement PK, TT, TC, and TT/TC, it is necessary to choose values for their parameters. Since we were interested in the effects of making structural changes to PK, we made every effort to run the models with their parameters set at values used by Paltiel and Kaplan. The only parameter we set at a value different from those used by Paltiel and Kaplan was n, the number of participants who are given the drug being tested. Paltiel and Kaplan used n = I00. We used n = 50. The reason was that our largest model, TT/TC, was intractable with n = 100. Even with n = 50, a single run of T T / T C required about an hour of CPU time on a DECsystem 5000/260. (A single run of PK required only a few seconds of CPU time.) The values we used are listed in Table 1. In order to implement the models, it is also necessary to choose set(s) of possible values for their decision variable(s) and to choose distributions for their random variables. In P K and TC, the only decision variable is T, the length of a Phase III clinical trial. In TT and TT/TC, there are two decision variables: /'1 and /'2, the lengths of the first and second Phase III clinical trials. In our implementations of PK and TC, we followed the example of Paltiel and Kaplan and restricted T to integers between 0 and 3, inclusive. In our implementations of T T and TT/TC, we restricted both TI and T2 to values from this set.
Decision-theoretic models of AIDS clinical trials
97
Table 1. Settings of the parameters Parameters
Epidemiological c fl p 7 z U
contact rate infectivity non-AIDS-related departure rate natural AIDS-related departure rate screening intensity level new member arrival rate
5 partners per year 0.075 per infected partner 1/60 per person per year 1/10 per person per year 1 screening per person per year 1667 per year
cost cost cost cost
$500,000 per year $10 per test $1,000 per person per year $100,000 per person
Cost CCL~N*C CxEsT CDRUG Calm
of of of of
conducting a trial screening for HIV administering the drug AIDS treatment
Other number of participants given the drug being tested per trial length of planning horizon
50 persons 10 years
For both PK and TT, we assumed that the random variable A follows a 'truncated' normal distribution with parameters 3" and aa. In particular, we assumed that 1
if 16 - 81 ~< 2aa, otherwise,
A(a)= K x / ~ a a 0 where K=
e -<
d~
d8- 2aa N / ~ f f a
The parameter 5 should be interpreted as the Phase II estimate of the AIDS incubation rate produced by the drug being tested. The parameter aA should be viewed as a measure of how uncertain the agency is about the accuracy of this estimate. Note that our assumption that AIDS-related departure times are exponentially distributed implies that 1/3"is the Phase II estimate of the mean AIDS incubation time for HIV-positive, AIDS-asymptomatic individuals who are given the drug. For TC and TT/TC, we assumed that F and A follow truncated normal distributions with parameters ?" and ar and parameters Z and aA, respectively. Here, ~7is an estimate of the natural AIDS incubation rate, and ~ is the Phase II estimate of the effect of the drug being tested on the AIDS incubation rate. (So 5, the Phase II estimate of the AIDS incubation rate induced by the drug being tested, is given by 5 = ~7+ )7.) The parameters ar and a^ are measures of how uncertain the agency is about the accuracy of ~ and £, respectively. We coded the four models in C and ran them on a DECsystem 5000/260 and on several Silicon Graphics Indigo2 200Mz MIPS R4400 computers. To solve the systems of differential equations given by (1) and (2) and then (3), (4), and (5), we used a Runga-Kutta method. Each time we solved these systems of differential equations, we used X(0) = 70,000, Y(0) = 20,000, and Z(0) = 0 as initial conditions. To evaluate the integrals that appear in the definitions of C°(7) and C~(?, a), we used Simpson's method. Results for PK and TT We ran PK and T T for numerous settings of a- between ~ and }. By doing this, we were limiting our attention to drugs that appeared, from the information available at the end of Phase II, to be able to produce mean AIDS incubation times ranging from 7 years to 15 years. For each setting of 5, we considered three settings of aa: 0.000, 0.015, and 0.030. We did this to determine how the level of uncertainty about the accuracy of the Phase II estimate affects the optimal policy. The results of our experiments with PK and T T are displayed in Figs 6 and 7. In these
98
Robert A. Bosch and Natasha K. Stout o',x = 0.030
RI
3
I
2
I1 12111
O'A -- 0.015
R
cr/x = 0.000 8
9
10
t I
11
12
A A A
13
14
P h a s e II e s t i m a t e of m e a n i n c u b a t i o n time (years) Fig. 6. Optimal policies for PK.
figures--and in all subsequent ones--the symbols 'R' and 'A' stand for the policies 'reject the drug immediately' and 'accept the drug immediately' respectively. The numerical symbols represent policies in which the drug is put through a Phase III clinical trial; the number gives the length of the trial in years. Suppose, for instance, that the Phase II estimate of the mean AIDS incubation time produced by a certain drug was 8.7 years and that the uncertainty about the accuracy of this estimate was high (i.e. aA = 0.030). PK would recommend that the drug be put through a three-year Phase III clinical trial (see Fig. 6). At the end of this trial, the government agency would examine the results and decide whether to reject or accept the drug. TT would come to a different conclusion; it would recommend a one-year Phase III clinical trial (see Fig. 7). At the end of this trial, the agency would examine the results and decide whether to reject the drug, accept it, or put it through a second Phase III clinical trial. The results displayed in Fig. 6 confirm what Paltiel and Kaplan reported: that a drug that seems not-at-all promising at the end of Phase II should be rejected at the end of Phase II, that a drug that seems quite promising at that point in time should be accepted at that point in time, that a drug that seems neither not-at-all promising nor quite promising at the end of Phase II should be put through a Phase III clinical trial, and that if a drug is to be put through a Phase III clinical trial, the length of the trial should depend on how promising the drug seems to be (the more promising the drug, the shorter the Phase III clinical trial). Also, the results displayed in Fig. 6 indicate that as uncertainty about the Phase II estimate of the drug's efficacy increases, longer trials are needed. The results we obtained with our two-trial model are, for the most part, similar. A comparison of Figs 6 and 7 shows that most of the drugs that are rejected immediately by PK are rejected immediately by TT as well. Similar statements can be made about the drugs that are accepted immediately and the drugs that are put through Phase III clinical trials. Essentially, the two models differ only in their recommendations concerning the lengths of Phase III clinical trials. From Fig. 6, it is apparent that if the Phase II estimate of a drug's efficacy is very low, but not quite low enough to warrant immediate rejection, and if the agency is quite uncertain about the accuracy of this estimate, then PK recommends that the drug be put through a long Phase III clinical trial. From Fig. 7, it is clear that under the same conditions, TT recommends a short Phase III clinical trial. Of the two recommendations, we find the recommendation of TT the more intuitive. We would expect that all 'borderline' drugs---drugs that would be rejected at the end of Phase II if only they seemed
o'A = 0.030 ~t, = 0.015 t r A = 0.000
R 10
[ 11
12
A 13
P h a s e II e s t i m a t e of m e a n incubation time (years) Fig. 7. Optimal policies for TT.
14
Decision-theoretic models of AIDS clinical trials
99
O"F = O"A
A
o'r' = 0.8O'A
A
O"F = 0.6o"A
A
o"F = 0.40"A
R
A
O'F = 0.2o'.4. 8
9
10
11
12
13
14
P h a s e II e s t i m a t e o f m e a n i n c u b a t i o n t i m e ( y e a r s ) Fig. 8. Optimal policies for TC, aA = 0.030.
to be a little less promising and drugs that would be accepted at the end of Phase II if only they seemed to be a little more promising--would be given brief Phase III clinical trials. In our opinion, it would seem that if, at the end of Phase II, the agency has a strong suspicion about the ultimate fate of the drug (i.e. rejection or acceptance) it would be a waste of money to put the drug through a long Phase III clinical trial. Figures 6 and 7 also make it clear that TT recommends brief (one-year) Phase III clinical trials much more often than does PK. This is to be expected. In TT, each Phase III clinical trial that is performed at the end of Phase II is assumed to be an initial Phase III clinical trial; if the outcome is inconclusive, a second, follow-up Phase III clinical trial can be performed.
Results for TC and TT/TC To facilitate comparison with PK and TT, we ran TC and T T / T C for 3-between ~ and 3. For each setting of 3-, we considered two settings of a~: 0.015 and 0.030 (the 0.000 setting isn't meaningful for this model). We kept ~'fixed at ~0(the estimate of the natural AIDS-related departure rate used by Paltiel and Kaplan). For each setting of a~, we considered several pairs of values of ar and aA that resulted in a ~ ~ that setting. (Note that since A = F + A and F and A are .~ 2 independent, we have aA ,,,, x/a~2 + a~. Also note that if the as were the standard deviations of the random variables rather than approximations of the standard deviations, we would have aa = x/~-r + a~.) We always chose values of ar and aA that satisfied ar ~< aA. We did this in order to model our assumption that more is known about the natural AIDS incubation rate than is known about the effect of the drug being tested on the AIDS incubation rate. The results of our experiments with TC are displayed in Figs 8 and 9. Note that, for the most
R
O'~ = ( 7 " A ~V
--
1[
A
0.SqA
err -- 0.6qh
R
gr = 0.4ah
lq.
A
R
A
ffF " - 0.2CrA
9
2[ 1
10
11
12
A
13
P h a s e II e s t i m a t e o f m e a n i n c u b a t i o n t i m e ( y e a r s ) Fig. 9. Optimal policies for TC, aA = 0.015.
14
100
Robert A. Bosch and Natasha K. Stout
O-r --O- A o'r
=
0.8trA
o-F
_--
0.60- A
crF
=
0.40" A
o'r
=
0.2~rA
R R
8
13_[ Ill 1
2 z
9
10
I
]
i
[
1
1
1 11
A I
A I
12
A
13
14
Phase II estimate of mean incubation time (years) Fig. 10. Optimal policies for TT/TC, ~r~= 0.030. part, the results are similar, from a qualitative standpoint, to the results obtained with P K - - i n nearly every case, we encountered ' . . . a continuum of optimal trial lengths that runs from immediate rejection, through long testing periods, through shorter tests, to immediate approval . . . . ' Ref. [1] (p. 189). The notable exception is the aA = 0.030, ar -- 0.6aA case. Also note that in both the aA = 0.030 cases and the aA = 0.015 cases, as ar decreases (and aA increases), policies involving putting the drug through a Phase III clinical trial are optimal more and more often. This means that if more and more of the variation in A is owing to variation in A, policies involving Phase III trials will be optimal a greater and greater proportion of the time. The results of our experiments with T T / T C are displayed in Figs 10 and 11. Note that the results obtained with T T / T C resemble both the results obtained with TT (continuums of optimal trial lengths that run from immediate rejection, through short testing periods, through longer tests, through shorter tests once again, to immediate approval) and the results obtained with TC (an increase in the fraction of the variation in A owing to variation in A leads to an increase in the amount of optimal policies that involve putting drugs through Phase III clinical trials). CONCLUSIONS In this article we have described three refinements of Paltiel and Kaplan's decision-theoretic model of A I D S clinical trials, each one structurally more realistic than the original. The results of our numerical experiments with these models confirm much of what Paltiel and Kaplan reported in their article. Like Paltiel and Kaplan, we found that the optimal policy depends on the preliminary estimate of a drug's efficacy. Like Paltiel and Kaplan, we found that a drug that
o-F -- o-A crr =
R
0.80- A
Ill
A
1%
crr -=--0 . 6 c r A
A
o'r = 0.4O'A
R
I l l
A
o-F -- 0.2O-A
R
1 1 1
A
8
9
10
11
12
13
Phase II estimate of mean incubation time (years) Fig. 11. Optimal policies for TT/TC, aA = 0.015.
14
Decision-theoretic models of AIDS clinical trials
101
appears to be not-at-all promising should be rejected immediately, a drug that appears to be very promising should be accepted immediately, and a drug that fits into neither o f the previous two categories should be put t h r o u g h a clinical trial. A n d like Paltiel and Kaplan, we f o u n d that the optimal length o f a clinical trial increases as uncertainty a b o u t the preliminary estimate o f the drug's efficacy increases. Where our results differ f r o m those obtained by Paltiel and K a p l a n is in the relationship between the optimal length o f a clinical trial and the preliminary estimate o f a drug's efficacy. O u r two-trial and two-trial/treatment-control models r e c o m m e n d short trials for all 'borderline' drugs, i.e. both those that would be rejected immediately if only they seemed to be a little less promising and those that would be accepted immediately if only they seemed to be a little more promising. Acknowledgements--The authors are grateful to Professor Edward Kaplan and Professor A. David Paltiel for making numerous helpful comments about an early draft of this paper, to Derek Bosch of Silicon Graphics, Inc. for providing access to several Silicon Graphics Indigo2 computers, and to two anonymous referees for their excellent comments and suggestions.
REFERENCES 1 Paltiel A. D. and Kaplan, E. H., The epidemiological and economic consequences of AIDS clinical trials. Journal of Acquired Immune Deficiency Syndrome, 1993, 6, 179-190. 2 May, R. M. and Anderson, R. M., Transmission dynamics of HIV infection. Nature (London), 1987, 326, 137-142. 3 Anderson, R. M., Gupta, S. and May, R. M., Potential of community-wide chemotherapy or immunotherapy to control the spread of HIV-I. Nature (London), 1991, 350, 356-359. 4 Paltiel, A. D. and Kaplan, E. H., Modeling zidovudine therapy: a cost-effectivenessanalysis. Journal Acquired Immune Deficiency Syndrome, 1991, 4, 795-804.
~ ) Pergamon
PII: S0038-0121(96)00030-4
© 1997 ElsevierScience Ltd All rights reserved. Printed in Great Britain 0o38-Ol21/97 $17.oo+ o.oo
Refinements of Paltiel and Kaplan's Decision-theoretic Model of AIDS Clinical Trials ROBERT A. BOSCHI~" and NATASHA K. STOUT 2 tOberlin College, Department of Mathematics, King Building 205, 10 North Professor St, Oberlin, OH 44074-1019, U.S.A. 2Epic Systems, Madison, WI 53707, U.S.A.
Abstraet--ln this paper we present three decision-theoretic models for determining, amongst all of the courses of action available to a government agency such as the Food and Drug Administration at a certain stage of its examination of a new AIDS drug, the one that places the least economic burden on society. Our models are based on a model devised by Paltiel and Kaplan. With their model, Paltiel and Kaplan concluded that the agency's optimal course of action depends on its initial estimate of a drug's efficacy: A drug that appears to be not-at-all promising should be rejected immediately. A drug that appears to be very promising should be accepted immediately. All other drugs should be put through a clinical trial. The length of a clinical trial should depend on the agency's estimate of the drug's efficacy (the more promising the drug, the shorter the clinical trial). With our models, which are structurally more realistic than Paltiel and Kaplan's model, we obtained mostly similar results. The differences we observed have to do with the relationship between the optimal length of a clinical trial and the preliminary estimate of the drug's efficacy. © 1997 Elsevier Science Ltd
INTRODUCTION Before a new AIDS drug is submitted to the Food and Drug Administration (FDA) for approval, it is put through a three-phase examination process. In the first phase of this process, the only concern is safety. In this phase, researchers look for side effects, estimate toxicity, and determine safe dosages. Usually, the Phase I clinical trial has no more than 20 participants and lasts for less than a year. Once Phase I is completed, the focus of the examination process shifts from safety to performance. In the remaining two phases, researchers obtain estimates of the drug's efficacy--preliminary estimates in Phase II and final estimates in Phase III. Usually, the Phase II and Phase III clinical trials have several hundred and several thousand participants, respectively, and last upwards of two and four years. In a recent article [1], Paltiel and Kaplan devised a decision-theoretic model for determining, amongst all of the courses of action available to a government agency such as the FDA at the end of Phase II, the one that places the least economic burden on society. Their model (which we will refer to as PK) allows for the agency to reject the drug at the end of Phase II, to accept the drug at the end of Phase II, or to postpone the decision to reject or accept until after the drug has been put through a single Phase III clinical trial. PK allows for the agency to use the information gathered in Phase II to help it select the length of the Phase III clinical trial, but not to help it choose the number of Phase III participants. Paltiel and Kaplan conducted a number of numerical experiments with PK. For the most part, their results confirm what one's intuition might expect: Which policy is optimal depends on the preliminary (i.e. Phase II) estimate of the drug's efficacy. A drug that appears to be not-at-all promising should be rejected immediately. A drug that seems very promising should be accepted immediately. A drug that fits into neither of the previous two categories should be put through tAuthor for correspondence. 87
88
Robert A. Bosch and Natasha K. Stout
a Phase III clinical trial, and the length of this trial should depend on the preliminary estimate of the drug's efficacy. In our opinion, Paltiel and Kaplan's results are counter-intuitive in only one respect: the specific relationship between the optimal length of a Phase III clinical trial and the preliminary estimate of the drug's efficacy. Their results indicate that the more promising the drug appears to be at the end of Phase II, the shorter should be its Phase III clinical trial. We were surprised by this. We would expect the optimal length of a Phase III clinical trial to be short for not one, but two classes of drugs: drugs that seem promising at the end of Phase II but do not seem promising enough to warrant immediate acceptance, and drugs that seem to be a failure at the end of Phase II but not enough of a failure to warrant immediate rejection. In this article, we present three refinements of PK, each one structurally more realistic than the original. (For the most part, we did not alter the values of the numerous parameters of PK.) In our two-trial model (TT), we allow for the drug to be rejected at the end of Phase II, accepted at the end of Phase II, or for the decision to reject or accept to be postponed until after one or two Phase III clinical trials have been performed. In PK, at the end of the Phase III clinical trial, the drug must be rejected or accepted. In practice, however, when the results of a Phase III clinical trial are inconclusive, the drug might be put through another Phase III clinical trial. In our second refinement of PK, our treatment-control model (TC), we allow for the standard statistical practice of dividing the participants of the Phase III clinical trial into a treatment group and a control group. The participants who are assigned to the treatment group are given the drug being tested. Those who are assigned to the control group are given either a placebo or an already approved drug. In PK, there is no control group; all participants of the Phase III trial are given the drug being tested. Our third and final refinement of PK is simply a combination of TT and TC. In TT/TC, we allow for the drug to be rejected at the end of Phase II, accepted at the end of Phase II, or for the decision to be postponed until after one or two Phase III clinical trials. In each trial, the participants are divided into a treatment group and a control group. This article is organized as follows: The first section is devoted to a discussion of the assumptions and mathematics that underlie Paltiel and Kaplan's model, and our two refinements of it. The next four sections contain descriptions of PK, TT, TC, and TT/TC. In the final section, we describe the numerical experiments conducted to compare the four models, and then suggest conclusions that can be drawn from the results. PRELIMINARIES Paltiel and Kaplan designed their model for the purpose of examining the epidemiological and economic consequences of AIDS clinical trials. Realizing that it would be difficult (if not impossible) to construct a simple model of how certain policies would affect the entire population of the United States, they decided to focus on how these policies would affect a much smaller and more homogeneous group of people: a population of sexually active homosexual males. In this section, we review how Paltiel and Kaplan modeled the effects of two types of policies on this population: policies that allow the AIDS epidemic to run its course, and policies that involve the implementation of screening and treatment programs. We refer the reader who desires more detailed explanations of this material to Ref. [1] (pp. 181-183). Readers who wish to examine the work that forms some of the foundations of Paltiel and Kaplan's work are referred to [2-4].
The population Let X(t), Y(t), and Z(t) denote the numbers of susceptibles, unidentified seropositives, and identified seropositives, respectively, at time t in a certain population of homosexual males who engage in unprotected anal intercourse. A population member is considered to be a susceptible if he is uninfected with the HIV virus, a seropositive if he is infected but asymptomatic, an unidentified seropositive if he is unaware that he is infected, and an identified seropositive if he has learned (by means of a blood test) that he is infected. We make the following assumptions: • New members enter the population at the rate of U per year. All new members are susceptibles. • Members leave the population owing to AIDS-related and non-AIDS-related causes. An
Decision-theoretic models of AIDS clinical trials
• • • • •
89
AIDS-related departure takes place whenever a seropositive progresses to AIDS. A non-AIDS-related departure takes place whenever a population member dies from non-AIDS-related causes. For seropositives who receive no treatment, the AIDS-related departure rate is y per person per year, the natural AIDS incubation rate. The non-AIDS-related departure rate is # per person per year. The only way that a susceptible can contract the virus is to have sexual intercourse with a seropositive partner. Each member of the population has c distinct sexual partners per year. The probability is fl that a susceptible will contract the virus from one particular seropositive partner.
Allowing the epidemic to run its course We now consider how the population would be affected if the AIDS epidemic were allowed to run its course from time t = 0 (the present) until time t = h (the end of the time horizon). We assume that during this time period, no effort is made to identify or treat seropositives. Thus, the only members of the population who are given any care whatsoever are the members who have progressed to AIDS, and they are treated for symptoms of AIDS and for secondary infections. We also assume that no new infections occur after time t = h. (Although one need not do so, one can think of time t = h as the time at which a vaccine is introduced.) It can be shown that under these assumptions and the assumptions described in the previous subsection, the following system of differential equations describes how the population changes over time:
dX dt - U dY
cflXY
I~X
(1)
(~ + y) Y
(2)
X+ Y
cflXY
-d-[ = x + r
Since there is no screening for the HIV virus, Z(t) = 0 for all 0 ~< t ~ h. It is easy to show that if the epidemic is allowed to run its course, the cost to society will be given by ~ ( y ) : = CARDS7i ~ Y(t)dt + CA,DS"~--~Y "Y(h), where CARDSis the cost of providing medical care to one AIDS patient for the rest of his life and Y(.) is obtained by solving (1) and (2). The first term of ~ ( y ) represents the cost of caring for the members of the population who progress to AIDS before time t = h. The second term represents the cost of caring for those who progress to AIDS after time t = h. It is also easy to show that if the decision to let the epidemic run its course is made not at time t - - 0 but at time t = t*, the moment at which an experimental drug emerges from the drug examination process, then the cost to society will be given by C/(y): = C°(y) + CCL,N,ct*, where CcuN~C is the cost of conducting a clinical trial for one year.
Implementing screening and treatment programs We conclude this section by considering how the population would be affected if at time t = t* a new AIDS drug is accepted, a screening program is put into effect, and all members who are thus identified as seropositives are given the drug. We assume that the drug affects the AIDS incubation rate by changing it from y to 6, and the infectivity of the virus by changing it from fl to Off, where 0 = 6/7. (We refer readers who desire an explanation of the reasonableness of the latter assumption to Ref. [1] (p. 181).) We also assume that in the screening program, each susceptible and unidentified seropositive is tested v times per year for the virus. It can be shown that under these assumptions and under the assumptions described in the first subsection of this
90
Robert A. Bosch and Natasha K. Stout
section, the following system of differential equations describes how the population changes over time:
dX dr-
cflX(Y + OZ) X+ Y+Z
U
d Y cflX(Y + OZ) dt - X + Y + Z
/iX
(l~+7+I,.z)Y
dZ -d-[ = I,.z Y - (# + 6)Z.
(3)
(4)
(5)
Here, it is understood that Z(t) = 0 for all t ~< t* and that L. is a function of t that takes on the value 1 if t > t* and 0 otherwise. Under the assumption that the drug is put through a clinical trial from time t = 0 until time t = t*, it can be shown that the cost to society is given by
C;(]~, 6): =
CCLIN,Ct*
+ CTESTZ (X(t) + Y(t))dt + CrEsT(X(h) + Y(h))
+ CDRUGfl + CMDS 7
Z(t)dt +
r(t)dt + 6
CDRUG"~--~~ "(Y(h) -~- Z(h))
Z(t)dt
+ CARDS"~ ' - ~ "(r(h) + Z(h)),
where CCLINICis the cost of conducting a clinical trial for one year, CTESTis the cost of screening one individual for the virus, CDRUCis the cost of giving the drug to one person for one year, and X(.), Y(.), and Z(.) are obtained by solving (3), (4) and (5). The first term of C~(7, 3) represents the cost of conducting a clinical trial from time t = 0 to time t = t*. The second and third terms represent the cost of running the screening program; the fourth and fifth terms, the cost of administering the drug to identified seropositives; and the sixth and seventh terms, the cost of providing medical care to the members of the population who progress to AIDS. The reader should note that we assume that at time t = h all unidentified persons are tested. The reader should recall that all seropositives are given the drug from the moment they are identified until the moment they leave the population, and that those seropositives who leave the population owing to AIDS-related causes are treated for symptoms of AIDS and secondary infections. Finally, the reader should note that we assume that C*tos is the same for every AIDS patient; it does not depend on if, or on how long, the patient received the drug. PALTIEL AND KAPLAN'S MODEL Suppose that a new drug has been developed for reducing the AIDS incubation rate--the per capita rate at which HIV-positive, AIDS-asymptomatic individuals progress to AIDS. The sponsor of the drug conducts a Phase I clinical trial and judges the drug to be safe. The sponsor then puts the drug through a Phase II clinical trial and obtains preliminary estimates of its efficacy (the AIDS incubation rate it produces). Paltiel and Kaplan consider the problem of determining, amongst all the courses of action available to a government agency such as the F D A at this point in time, the one that places the least economic burden on society. Their approach to the problem is displayed in the decision tree in Fig. 1. Paltiel and Kaplan assume that at the end of Phase II, the agency has three options--to reject the drug immediately, to accept the drug immediately, or to require the sponsor to conduct a single Phase III clinical trial. They allow the agency to select the Phase III trial length (T), but not the number of Phase III participants (n). They assume that at the beginning of the trial, all of the
Decision-theoretic models of AIDS clinical trials
91
participants will be HIV-positive and AIDS-asymptomatic; that for the duration of the trial, all of the participants will be given the same dosage of the drug; and that at the end of the trial, the results (denoted A'" when T = t*) will be expressed in terms of the number of participants who progressed to AIDS. They also assume that at the end of the trial, the agency must either reject or accept the drug.
Rejecting the drug immediately Paltiel and Kaplan assume that the drug being tested is the 'only hope' of AIDS researchers. Rejecting the drug, therefore, is tantamount to allowing the AIDS epidemic to run its course. Paltiel and Kaplan also assume that y, the natural AIDS incubation rate, is known with certainty. Thus, if the agency rejects the drug immediately, the expected cost to society will be CR: = CR(?).
(6)
Accepting the drug immediately Paltiel and Kaplan assume that acceptance of the drug is followed immediately by implementation of screening and treatment programs. Thus, if the agency accepts the drug immediately and the drug has efficacy 3, the cost to society will be ~ ( ? , 3). However, the agency does not know the drug's efficacy at the end of Phase II; at this point in time, the sponsor has only preliminary estimates of the drug's efficacy. To account for this, Paltiel and Kaplan assume that the drug's efficacy is a random variable A with known density function fa('). Accordingly, if the agency accepts the drug immediately, the expected cost to society will be
CA: =
C°A(V,6)fA(6)d6.
(7)
Conducting a phase III clinical trial Paltiel and Kaplan assume that at the end of a Phase III clinical trial, the agency must either reject or accept the drug. Thus, if the agency requires the sponsor to conduct a Phase III clinical trial of length t* and observes that a of the n participants progressed to AIDS during the trial, the expected cost to society will be
the minimum of the expected cost of rejecting the drug after performing the trial and the expected cost of accepting the drug after observing a of the participants progress to AIDS during the trial. Thus, if the agency requires the sponsor to conduct a Phase III clinical trial of length t*, the expected cost to society will be
C~;,: = ~ ¢,.(a)P(A"= a). a~O
Reject
C~. ('y)
~ Accep-~ tA =
6) Reject
Phase III Trial
C~t*('y)
Accep-~ t
&=6[A
k.
Fig. 1. Decisiontree for PK.
t"
=
a
t* cA (v, 6)
Robert A. Bosch and Natasha K. Stout
92
~
Reject C°(7) /Accept
~A=6
cO (7,
Trial First
~ - \;
tl ~
Phase III - ~ - 2"1 =
at
6) /Reject A'I~ = a l - i ~ A c c e p t
\
L
C ~ (7)
zx=61£1
s
(7, 6)
-
~,Phase
III',
,,_T_~. . . . . ;
Fig. 2. Decisiontree for TT.
Finding the optimal policy To determine which policy is optimal, it is necessary to compute CR, CA, and C~ for each possible trial length t*. If CR is the smallest of these costs, the optimal policy is to reject the drug immediately. If CA is the smallest, the optimal policy is to accept the drug immediately; and if C~, is the smallest, the optimal policy is to conduct a Phase III clinical trial of length t* and make the decision to reject or accept when the results are available. Probabilities Suppose that the drug as efficacy 6. Let p,.(6) denote the probability that an HIV-positive, AIDS-asymptomatic individual who is taking a drug of efficacy 6 progresses to AIDS during a trial of length t*. Since each of the n participants of the Phase III clinical trial is given the drug for the duration of the trial, and since each of the participants will progress to AIDS independently of the other participants, P(A'" = a) =
;(:)
p,.(f)a(1 - p,.(6))" - °fA(f)d6.
(9)
Paltial and Kaplan assume that AIDS-related and non-AIDS-related departure times are independent and are exponentially distributed with parameters 6 and p, respectively. Using this assumption, it is easy to show that
p,,(6) = ~
6
(1 -- e -(* +~)").
THE TWO-TRIAL MODEL In PK, the government agency can require the sponsor of a new AIDS drug to conduct at most one Phase III clinical trial; at the end of this trial, the agency must either reject or accept the drug. However, in practice, when the results of a Phase III clinical trial are inconclusive, the drug might be put through another Phase III clinical trial. We developed TT to allow for this additional option. Our approach to the problem is displayed in the decision trees in Figs 2 and 3. Like Paltiel and Kaplan, we assume that at the end of Phase II, the agency has three options--to f-
"1
I
Reject at
T2 = t 2 I
C q +q(7~
I
g~
22=a2 AtlX--__-al
I Accept
=6 A'll=al,A2 = a 2
t.
CA~+q(%6 .d
Fig. 3. Decisiontree for the secondPhase III clinicaltrial (TT).
Decision-theoretic models of AIDS clinical trials
93
reject the drug immediately, to accept the drug immediately, or to require the sponsor to conduct a Phase III clinical trial. In TT, at the end of the first Phase III clinical trial, the agency can reject the drug, accept the drug, or require the sponsor to conduct a second Phase III clinical trial. At the end of the second Phase III clinical trial, the agency must either reject or accept the drug. Like Paltiel and Kaplan, we assume that the agency can select the length of a Phase III clinical trial, but not the number of participants. We let T~ and 7"2 denote the lengths of the first and second Phase III clinical trials, respectively, and n denote the number of participants in each trial. We assume that different participants are used in each trial; that at the beginning of each trial, each participant is HIV-positive and AIDS-asymptomatic; and that for the duration of each trial, each participant is given the same dosage of the drug. We denote the results of the first trial by A[t when T~ = t* and the results of the second trial by Air when T2 = t*. (The subscript indicates the number of the trial; the superscript, the length.) Like Paltiel and Kaplan, we express the results of a trial in terms of the number of participants who progress to AIDS during that trial. Rejecting or accepting the drug immediately
The expected costs of rejecting the drug immediately and accepting the drug immediately are CR and CA, respectively--the same as in PK. See (6) and (7). Conducting the first Phase I I I clinical trial
We assume that at the end of the first Phase III clinical trial, the agency can reject the drug, accept it, or require the sponsor to conduct a second Phase III clinical trial. Thus, if the first Phase III clinical trial lasts for t* years, the expected cost to society will be
O,r(a,)P(A[ t = a,),
Gill,. = a I =0
where
0o) the minimum of the expected costs of rejecting the drug, accepting the drug, and conducting a second Phase III clinical trial, given that the first Phase III clinical trial lasted for t* years and that a~ of its participants progressed to AIDS during that time period. A comparison of (10) and (8) shows that the 'reject' and 'accept' terms of ~,r(a~) are essentially the same as the 'reject' and 'accept' terms of q~,,(a) in Paltiel and Kaplan's model. Since the agency must reject or accept the drug at the end of the second Phase III clinical trial, reasoning similar to that used in deriving C[;~in Paltiel and Kaplan's model can be used to show that the individual terms of the 'second Phase III clinical trial' term of ¢,r(a~) are given by CxI~(t, , a~): = ~ min Cg+'r(7), a2 = 0
C2+'~(7, 6)f~lAfT.A1~(61a,,a2)d6
~.
•P(A'2 r = az]A'l t = al). Finding the optimal policy
To determine which policy the agency should pursue at the end of Phase II, it is necessary to compute CR, CA, and C~h~for each possible length t* of the initial Phase III clinical trial. If Ca is the smallest of these costs, the optimal policy is to reject the drug immediately. If CA is the smallest, the optimal policy is to accept the drug immediately. If C~t~,is the smallest, the optimal policy is to conduct an initial Phase III clinical trial of length t* and make the decision to reject or accept or conduct a second Phase III clinical trial when the results of the first trial become available.
94
Robert A. Bosch and Natasha K. Stout
Reject - ~
r=7
Accept - ~
F=7 A=;~
Phase ]]I Trial
c[(7)
T=t*
7+
f_ _ ~
Reject
F="7
CR (7)
A t. = a - I I
B" =b
/- r = T A" =a
Accept - ~
A = i B*" = b
cf (7, 7 +
Fig. 4. Decision tree for TC.
Probabilities It should be noted that even though the events A~r = a,[A = 6 and A~~ = a21A = b are independent, the events A[t = a~ and A~~ = a2 are not independent. The result of the first trial can be used to make inferences about A, which, in turn, can be used to make revisions of probabilities pertaining to the second trial.
THE TREATMENT-CONTROL MODEL In PK and TT, it is assumed that the natural AIDS incubation rate, ?, is known with certainty. This assumption eliminates the need for a control group. (If the assumption holds, then it is possible to evaluate a drug by (1) administering the drug to a group of HIV-positive, AIDS-asymptomatic individuals for a period of one or more years, (2) recording how many of them progress to AIDS, (3) using this figure to estimate the AIDS incubation rate the drug produces, and (4) comparing this rate with 7.) In this section, we describe one way of removing this assumption from PK. Our approach to the problem is displayed in the decision tree in Fig. 4. In TC, we make many of the assumptions made by Paltiel and Kaplan: that at the end of Phase II, the agency may reject the drug, accept the drug, or require the sponsor to conduct a single Phase III clinical trial; that the agency may select the Phase III trial length (T), but not the number of Phase III participants; and that at the beginning of a Phase III clinical trial, the participants are HIV-positive and AIDS-asymptomatic. We do not, however, assume that each participant is given the same dosage of the drug. Instead, we assume that n of the participants are given the drug. These participants form the treatment group. They are each given the same dosage of the drug. The remaining n participants form the control group. They are each given a placebo. (Alternately, each member of the control group can be given an already approved drug.) We let A'" and B'* denote the number of members of the treatment group and control group, respectively, who progress to AIDS during a Phase III clinical trial of length t*.
How the treatment-control model differs from Paltiel and Kaplan's model As mentioned earlier, in both PK and TT, it is assumed that the natural AIDS incubation rate, ~, is known with certainty. In TC, we replace this assumption with a much milder one: that the natural AIDS incubation rate is given by a random variable F with known density function fr('). To allow for A to be dependent on F, we assume that A = F + A, where A is a random variable with known density function f^(.) and F and A are independent. We view A as the change in the incubation rate induced by the drug. As a result of these changes, in
Decision-theoretic models of AIDS clinical trials
95
TC, the expected cost of rejecting the drug immediately and of accepting the drug immediately are no longer given by (6) and (7), respectively. Instead, they are given by c~: =
f
~(~/f~(~,)d~,
and CA: =
I
f0f0
CA(7, y + 2)fr(y)f^(2)dy d2,
respectively. The expected cost of conducting a Phase III clinical trial of length t* is then given by
C~[I:= ~ f ~,.(a,b)P(A'" = a, B"= b) a=0
b=O
where
~,,(a,b):=min{fo~C'~(~)frfB,'(Tlb)dT, fo~fo°°C'~(y,y+2)fr.A~A,'.B,'(y,21a,b)dTd2}. Finding the optimalpolicy As in PK, determining which policy is optimal involves comparing CR, CA, and all of the Cm"'s. THE TWO-TRIAL/TREATMENT-CONTROL MODEL For our final model, we combined the features of TT and TC. Like TT, TT/TC has a two-piece decision tree. (To save space, we decided against including a picture of the main piece.) The main piece resembles the decision tree for TC, but has Tts in place of Ts, A't's in place of A"s, etc. and has not two, but three branches--a 'Reject' branch, an 'Accept' branch, and a 'Second Phase III Trial' branch---extending from each node corresponding to an outcome of the first Phase III clinical trial. The second piece of the decision tree for TT/TC is displayed in Fig. 5. The notation is similar to that used in TT and TC. (The 'A' random variables give the numbers of AIDS cases observed in the treatment groups, while the 'B' random variables give the numbers observed in the control groups. Subscripts indicate the number of the Phase III trial--first or second--and superscripts indicate the length.) The costs of accepting the drug immediately and rejecting the drug immediately are exactly the same as in TC. The expression for the expected cost of postponing the decision until after conducting a (possibly initial) Phase III clinical trial of length t~* is quite long and involved, but is easy to obtain provided that one keeps in mind that (1) the expected cost of rejecting the drug after observing at and bt AIDS cases in the treatment and control
/_ q
.
, q
/
r Reject
r= lB r=bl,B; =b2
_,l,_A2=a21Al=al..~ t2
gl
UAccept-~_A=A]B;~=bl,B;
~ b,
Fig. 5. Decision tree for the second Phase III clinical trial (TT/TC).
+
96
Robert A. Bosch and Natasha K. Stout
groups of a Phase III trial lasting t~* years, and a2 and b2 AIDS cases in the treatment and control groups of a second Phase III trial lasting t2* years is given by
o~ Cff +'~(v)fr~slr.Bp(ylb~,b2)dv, and (2) the expected cost of accepting the drug under the same conditions is given by
2°°fo~C'ar÷'~(7,),+~)fr.A,~LB,r.~,~B,~(7,).la,,b~,a2, b2)dTd2.
NUMERICAL EXPERIMENTS The numerical experiments of Paltiel and Kaplan indicate that the optimal course of action for the agency at the end of Phase II depends on the Phase II estimate of the efficacy of the drug being tested. If the Phase II clinical trial is conclusive, the agency should be aggressive: drugs that seem not-at-all promising at the end of Phase II should be rejected at the end of Phase II, and drugs that seem quite promising at that point in time should be accepted at that point in time. In other words, as one's intuition might expect, if the Phase II clinical trial is conclusive, there is no need to perform a Phase III clinical trial. Paltiel and Kaplan's numerical experiments also indicate that if the Phase II clinical trial is inconclusive, the agency should put the drug through a Phase III clinical trial. The optimal length of this trial depends on how promising the drug seems at the end of Phase II: the more promising it seems, the shorter the trial. We were surprised by the final portion of this conclusion. We would expect the optimal length of a Phase III clinical trial to be directly proportional to the degree to which the Phase II clinical trial is inconclusive. That is, we would expect the optimal length of a Phase III clinical trial to be short when the Phase II clinical trial indicates (inconclusively) that the drug is effective and when it indicates (inconclusively) that the drug is ineffective, not just in the former case. We designed our numerical experiments to determine whether our three refinements of Paltiel and Kaplan's model would lead us to the same conclusions as those of Paltiel and Kaplan, or if they would lead us to conclusions that would agree more with our intuition, as discussed above. We begin this final section of the article by describing the details of our implementations of PK, TT, TC, and TT/TC. We then present our results and conclusions.
Implementing the models In order to implement PK, TT, TC, and TT/TC, it is necessary to choose values for their parameters. Since we were interested in the effects of making structural changes to PK, we made every effort to run the models with their parameters set at values used by Paltiel and Kaplan. The only parameter we set at a value different from those used by Paltiel and Kaplan was n, the number of participants who are given the drug being tested. Paltiel and Kaplan used n = I00. We used n = 50. The reason was that our largest model, TT/TC, was intractable with n = 100. Even with n = 50, a single run of T T / T C required about an hour of CPU time on a DECsystem 5000/260. (A single run of PK required only a few seconds of CPU time.) The values we used are listed in Table 1. In order to implement the models, it is also necessary to choose set(s) of possible values for their decision variable(s) and to choose distributions for their random variables. In P K and TC, the only decision variable is T, the length of a Phase III clinical trial. In TT and TT/TC, there are two decision variables: /'1 and /'2, the lengths of the first and second Phase III clinical trials. In our implementations of PK and TC, we followed the example of Paltiel and Kaplan and restricted T to integers between 0 and 3, inclusive. In our implementations of T T and TT/TC, we restricted both TI and T2 to values from this set.
Decision-theoretic models of AIDS clinical trials
97
Table 1. Settings of the parameters Parameters
Epidemiological c fl p 7 z U
contact rate infectivity non-AIDS-related departure rate natural AIDS-related departure rate screening intensity level new member arrival rate
5 partners per year 0.075 per infected partner 1/60 per person per year 1/10 per person per year 1 screening per person per year 1667 per year
cost cost cost cost
$500,000 per year $10 per test $1,000 per person per year $100,000 per person
Cost CCL~N*C CxEsT CDRUG Calm
of of of of
conducting a trial screening for HIV administering the drug AIDS treatment
Other number of participants given the drug being tested per trial length of planning horizon
50 persons 10 years
For both PK and TT, we assumed that the random variable A follows a 'truncated' normal distribution with parameters 3" and aa. In particular, we assumed that 1
if 16 - 81 ~< 2aa, otherwise,
A(a)= K x / ~ a a 0 where K=
e -<
d~
d8- 2aa N / ~ f f a
The parameter 5 should be interpreted as the Phase II estimate of the AIDS incubation rate produced by the drug being tested. The parameter aA should be viewed as a measure of how uncertain the agency is about the accuracy of this estimate. Note that our assumption that AIDS-related departure times are exponentially distributed implies that 1/3"is the Phase II estimate of the mean AIDS incubation time for HIV-positive, AIDS-asymptomatic individuals who are given the drug. For TC and TT/TC, we assumed that F and A follow truncated normal distributions with parameters ?" and ar and parameters Z and aA, respectively. Here, ~7is an estimate of the natural AIDS incubation rate, and ~ is the Phase II estimate of the effect of the drug being tested on the AIDS incubation rate. (So 5, the Phase II estimate of the AIDS incubation rate induced by the drug being tested, is given by 5 = ~7+ )7.) The parameters ar and a^ are measures of how uncertain the agency is about the accuracy of ~ and £, respectively. We coded the four models in C and ran them on a DECsystem 5000/260 and on several Silicon Graphics Indigo2 200Mz MIPS R4400 computers. To solve the systems of differential equations given by (1) and (2) and then (3), (4), and (5), we used a Runga-Kutta method. Each time we solved these systems of differential equations, we used X(0) = 70,000, Y(0) = 20,000, and Z(0) = 0 as initial conditions. To evaluate the integrals that appear in the definitions of C°(7) and C~(?, a), we used Simpson's method. Results for PK and TT We ran PK and T T for numerous settings of a- between ~ and }. By doing this, we were limiting our attention to drugs that appeared, from the information available at the end of Phase II, to be able to produce mean AIDS incubation times ranging from 7 years to 15 years. For each setting of 5, we considered three settings of aa: 0.000, 0.015, and 0.030. We did this to determine how the level of uncertainty about the accuracy of the Phase II estimate affects the optimal policy. The results of our experiments with PK and T T are displayed in Figs 6 and 7. In these
98
Robert A. Bosch and Natasha K. Stout o',x = 0.030
RI
3
I
2
I1 12111
O'A -- 0.015
R
cr/x = 0.000 8
9
10
t I
11
12
A A A
13
14
P h a s e II e s t i m a t e of m e a n i n c u b a t i o n time (years) Fig. 6. Optimal policies for PK.
figures--and in all subsequent ones--the symbols 'R' and 'A' stand for the policies 'reject the drug immediately' and 'accept the drug immediately' respectively. The numerical symbols represent policies in which the drug is put through a Phase III clinical trial; the number gives the length of the trial in years. Suppose, for instance, that the Phase II estimate of the mean AIDS incubation time produced by a certain drug was 8.7 years and that the uncertainty about the accuracy of this estimate was high (i.e. aA = 0.030). PK would recommend that the drug be put through a three-year Phase III clinical trial (see Fig. 6). At the end of this trial, the government agency would examine the results and decide whether to reject or accept the drug. TT would come to a different conclusion; it would recommend a one-year Phase III clinical trial (see Fig. 7). At the end of this trial, the agency would examine the results and decide whether to reject the drug, accept it, or put it through a second Phase III clinical trial. The results displayed in Fig. 6 confirm what Paltiel and Kaplan reported: that a drug that seems not-at-all promising at the end of Phase II should be rejected at the end of Phase II, that a drug that seems quite promising at that point in time should be accepted at that point in time, that a drug that seems neither not-at-all promising nor quite promising at the end of Phase II should be put through a Phase III clinical trial, and that if a drug is to be put through a Phase III clinical trial, the length of the trial should depend on how promising the drug seems to be (the more promising the drug, the shorter the Phase III clinical trial). Also, the results displayed in Fig. 6 indicate that as uncertainty about the Phase II estimate of the drug's efficacy increases, longer trials are needed. The results we obtained with our two-trial model are, for the most part, similar. A comparison of Figs 6 and 7 shows that most of the drugs that are rejected immediately by PK are rejected immediately by TT as well. Similar statements can be made about the drugs that are accepted immediately and the drugs that are put through Phase III clinical trials. Essentially, the two models differ only in their recommendations concerning the lengths of Phase III clinical trials. From Fig. 6, it is apparent that if the Phase II estimate of a drug's efficacy is very low, but not quite low enough to warrant immediate rejection, and if the agency is quite uncertain about the accuracy of this estimate, then PK recommends that the drug be put through a long Phase III clinical trial. From Fig. 7, it is clear that under the same conditions, TT recommends a short Phase III clinical trial. Of the two recommendations, we find the recommendation of TT the more intuitive. We would expect that all 'borderline' drugs---drugs that would be rejected at the end of Phase II if only they seemed
o'A = 0.030 ~t, = 0.015 t r A = 0.000
R 10
[ 11
12
A 13
P h a s e II e s t i m a t e of m e a n incubation time (years) Fig. 7. Optimal policies for TT.
14
Decision-theoretic models of AIDS clinical trials
99
O"F = O"A
A
o'r' = 0.8O'A
A
O"F = 0.6o"A
A
o"F = 0.40"A
R
A
O'F = 0.2o'.4. 8
9
10
11
12
13
14
P h a s e II e s t i m a t e o f m e a n i n c u b a t i o n t i m e ( y e a r s ) Fig. 8. Optimal policies for TC, aA = 0.030.
to be a little less promising and drugs that would be accepted at the end of Phase II if only they seemed to be a little more promising--would be given brief Phase III clinical trials. In our opinion, it would seem that if, at the end of Phase II, the agency has a strong suspicion about the ultimate fate of the drug (i.e. rejection or acceptance) it would be a waste of money to put the drug through a long Phase III clinical trial. Figures 6 and 7 also make it clear that TT recommends brief (one-year) Phase III clinical trials much more often than does PK. This is to be expected. In TT, each Phase III clinical trial that is performed at the end of Phase II is assumed to be an initial Phase III clinical trial; if the outcome is inconclusive, a second, follow-up Phase III clinical trial can be performed.
Results for TC and TT/TC To facilitate comparison with PK and TT, we ran TC and T T / T C for 3-between ~ and 3. For each setting of 3-, we considered two settings of a~: 0.015 and 0.030 (the 0.000 setting isn't meaningful for this model). We kept ~'fixed at ~0(the estimate of the natural AIDS-related departure rate used by Paltiel and Kaplan). For each setting of a~, we considered several pairs of values of ar and aA that resulted in a ~ ~ that setting. (Note that since A = F + A and F and A are .~ 2 independent, we have aA ,,,, x/a~2 + a~. Also note that if the as were the standard deviations of the random variables rather than approximations of the standard deviations, we would have aa = x/~-r + a~.) We always chose values of ar and aA that satisfied ar ~< aA. We did this in order to model our assumption that more is known about the natural AIDS incubation rate than is known about the effect of the drug being tested on the AIDS incubation rate. The results of our experiments with TC are displayed in Figs 8 and 9. Note that, for the most
R
O'~ = ( 7 " A ~V
--
1[
A
0.SqA
err -- 0.6qh
R
gr = 0.4ah
lq.
A
R
A
ffF " - 0.2CrA
9
2[ 1
10
11
12
A
13
P h a s e II e s t i m a t e o f m e a n i n c u b a t i o n t i m e ( y e a r s ) Fig. 9. Optimal policies for TC, aA = 0.015.
14
100
Robert A. Bosch and Natasha K. Stout
O-r --O- A o'r
=
0.8trA
o-F
_--
0.60- A
crF
=
0.40" A
o'r
=
0.2~rA
R R
8
13_[ Ill 1
2 z
9
10
I
]
i
[
1
1
1 11
A I
A I
12
A
13
14
Phase II estimate of mean incubation time (years) Fig. 10. Optimal policies for TT/TC, ~r~= 0.030. part, the results are similar, from a qualitative standpoint, to the results obtained with P K - - i n nearly every case, we encountered ' . . . a continuum of optimal trial lengths that runs from immediate rejection, through long testing periods, through shorter tests, to immediate approval . . . . ' Ref. [1] (p. 189). The notable exception is the aA = 0.030, ar -- 0.6aA case. Also note that in both the aA = 0.030 cases and the aA = 0.015 cases, as ar decreases (and aA increases), policies involving putting the drug through a Phase III clinical trial are optimal more and more often. This means that if more and more of the variation in A is owing to variation in A, policies involving Phase III trials will be optimal a greater and greater proportion of the time. The results of our experiments with T T / T C are displayed in Figs 10 and 11. Note that the results obtained with T T / T C resemble both the results obtained with TT (continuums of optimal trial lengths that run from immediate rejection, through short testing periods, through longer tests, through shorter tests once again, to immediate approval) and the results obtained with TC (an increase in the fraction of the variation in A owing to variation in A leads to an increase in the amount of optimal policies that involve putting drugs through Phase III clinical trials). CONCLUSIONS In this article we have described three refinements of Paltiel and Kaplan's decision-theoretic model of A I D S clinical trials, each one structurally more realistic than the original. The results of our numerical experiments with these models confirm much of what Paltiel and Kaplan reported in their article. Like Paltiel and Kaplan, we found that the optimal policy depends on the preliminary estimate of a drug's efficacy. Like Paltiel and Kaplan, we found that a drug that
o-F -- o-A crr =
R
0.80- A
Ill
A
1%
crr -=--0 . 6 c r A
A
o'r = 0.4O'A
R
I l l
A
o-F -- 0.2O-A
R
1 1 1
A
8
9
10
11
12
13
Phase II estimate of mean incubation time (years) Fig. 11. Optimal policies for TT/TC, aA = 0.015.
14
Decision-theoretic models of AIDS clinical trials
101
appears to be not-at-all promising should be rejected immediately, a drug that appears to be very promising should be accepted immediately, and a drug that fits into neither o f the previous two categories should be put t h r o u g h a clinical trial. A n d like Paltiel and Kaplan, we f o u n d that the optimal length o f a clinical trial increases as uncertainty a b o u t the preliminary estimate o f the drug's efficacy increases. Where our results differ f r o m those obtained by Paltiel and K a p l a n is in the relationship between the optimal length o f a clinical trial and the preliminary estimate o f a drug's efficacy. O u r two-trial and two-trial/treatment-control models r e c o m m e n d short trials for all 'borderline' drugs, i.e. both those that would be rejected immediately if only they seemed to be a little less promising and those that would be accepted immediately if only they seemed to be a little more promising. Acknowledgements--The authors are grateful to Professor Edward Kaplan and Professor A. David Paltiel for making numerous helpful comments about an early draft of this paper, to Derek Bosch of Silicon Graphics, Inc. for providing access to several Silicon Graphics Indigo2 computers, and to two anonymous referees for their excellent comments and suggestions.
REFERENCES 1 Paltiel A. D. and Kaplan, E. H., The epidemiological and economic consequences of AIDS clinical trials. Journal of Acquired Immune Deficiency Syndrome, 1993, 6, 179-190. 2 May, R. M. and Anderson, R. M., Transmission dynamics of HIV infection. Nature (London), 1987, 326, 137-142. 3 Anderson, R. M., Gupta, S. and May, R. M., Potential of community-wide chemotherapy or immunotherapy to control the spread of HIV-I. Nature (London), 1991, 350, 356-359. 4 Paltiel, A. D. and Kaplan, E. H., Modeling zidovudine therapy: a cost-effectivenessanalysis. Journal Acquired Immune Deficiency Syndrome, 1991, 4, 795-804.