Decision analysis in oncology: Panacea or chimera?

Clinical Oncology(1992) 4:306-312 © 1992The RoyalCollegeof Radiologists

Clinical Oncology

Original Article Decision Analysis in Oncology: Panacea or Chimera? A. J. Munro Department of Radiotherapy, St Bartholomew's Hospital, West Smithfield, London EC1A 7BE, UK

Abstract. A review of the literature on the use of decision analysis in clinical oncology shows that, although these techniques have been available for more than 25 years, they have not been widely applied: only 19 decision analyses of therapeutic management in clinical oncology were found. The main disadvantages concern the difficulty of accurately assessing probabilities and defining measures of outcome. Time-consuming analysis may produce results that are either equivocal or simply confirm the expectations of common sense. If the basic design fails to include all relevant factors then any conclusions will be of little value. The main advantages are that, by demanding that problems be explicitly stated and analysed in a logical fashion, deficiencies in current knowledge, belief and practice are identified. The usefulness of these techniques lies in formulating management guidelines, either for treatment or for follow-up. They have only a limited role in decision making for individual patients. Keywords: Decision analysis; Decision support systems

Decision

tree;

INTRODUCTION Panacea: a universal remedy Chimera: a mere wild fancy, an unfounded conception, a monster with a lion's head and a goat's body The cluster of techniques, collectively known as decision analysis, purports to facilitate decisionmaking under conditions of uncertainty [1, 2], but in spite of considerable promise, has found little acceptance in clinical medicine. First applied to a clinical problem in oncology nearly 25 yeas ago [3], it has not been adopted into routine clinical practice. Decision analysis has been defined [2] as "a systemic approach to decision making under conditions of uncertainty". This is too loose, most of us like to think our decisions are made systematically. Decision analysis provides a formalized system but even inserting the word formal into the definition is inadequate. For example the I Ching [4] provides a formal system to guide decisions, but patients might Correspondence and offprint requests to: A. J. Munro, Department of Radiotherapy,St Bartholomew'sHospital, West Smithfield, LondonEC1A 7BE, UK.

feel uneasy if they felt their clinical management was dictated by randomly cast patterns of yarrow stalks interpreted through a dictionary of hexagrams. A better definition might be that 'decision analysis provides a systematic formalized method for decision making under conditions of uncertainty in which a linear, logical, approach governs both the structuring and dissection of the the problem'. The techniques are tedious to perform by hand but the necessary calculations are easily performed by microcomputers. Software for decision analysis is readily available and, as well as performing straightforward analyses, most programs permit more sophisticated analyses using Markov processes, Boolean nodes, cost effectiveness estimations and Monte Carlo simulations.

USE IN THERAPY The easiest way to approach decision analysis is via the consideration of an oversimplified clinical problem expressed in the form of a decision tree (Fig. 1). This tree fairly crudely models alternate management strategies for a patient with a stage TiNoM0 carcinoma of the vocal cord. For the sake of demonstration the management alternatives are total laryngectomy and radical radiotherapy. By convention the points at which the decision tree branches are termed nodes. There are two types of node: decision nodes, at which actions are determined by conscious choices and chance nodes where the outcomes are beyond direct control and governed simply by chance or probability. Thus, choice of treatment is a decision node but the possible outcomes of that treatment are represented by chance nodes. The decision tree is constructed so that each path through the tree eventually leads, via the final branches of the tree (the terminal branches), to the final outcomes. Each outcome has to be defined in terms of its 'utility'. Utility can be expressed as quality adjusted survival in which the years of survival resulting from a given policy are multiplied by a quality factor, ranging from 0 to 1, which is a measure of the extent to which the quality of that survival is impaired. This results in the well known unit, the QALY (quality adjusted life year) [5]. Once utilities have been defined the analysis of the decision tree proceeds by a simple process called 'folding back'. Starting peripherally, the utility of

Decision Analysis in Oncology

307 p1 CURE [ Surgical cu~-]

1-p2

FAILSALV I

SURGERY FAIL 1-pl

]

I

Su~ive after salvage Rx

]

p3 CONTROL I

cured by radiotherapy

[

sALvcu~ p2

CHOICE

Die

XRT

p4 SURGICAL SALVAGE

[cure after XRT + S[

SALVAGE FAILS

] die after XRT + S ]

FAIL

l-p3

1-p4 Fig. 1. A decision analysis for management of stage I T1NoM~ carcinoma of the larynx. Salvage therapy may succeed in curing patients who have failed primary therapy. These events are governed by the probabilities: p1=0.95, p2=0.15, p3=0.90, p4=0.50.

each branch is multiplied by the probability of its occurrence until branches of the central decision node are reached. Each branch will have a utility value associated with it which represents the overall value, appropriately weighted, of all the possible outcomes occurring distal to it. In the example given, the choice is for radical radiotherapy since it yields 17.8 QALYs compared with 14.4 for radical surgery. Decision analysis includes the fact that we do not know anything with certainty. We are unsure, for example whether an individual patient will survive disease-free 5 years after laryngectomy for Tt carcinoma of the larynx. We may use a probability of 0.9 in our analysis, but only choose this because it represents a compromise within the realistic range of values extending from 0.85 to 0.95. The techniques of sensitivity analysis permit the incorporation of this type of uncertainty into the decision. Having performed the analysis using the baseline value (0.9) we can then perform a series of analyses varying the value between 0.85 and 0.95. This may not affect the choice; alternatively, at some threshold value, the balance may tip from favouring radiotherapy to favouring surgery. In the example, surgery becomes preferable when the primary control rate with radiotherapy falls below 59.8%. More sophisticated versions of sensitivity analysis enable more than one variable to be varied at a time. For example, it is possible to analyse simultaneously (1) the effect of varying the probability of disease-free survival after radiotherapy and (2) the quality factor by which survival must be multiplied for those patients who require laryngectomy. Only when the quality adjustment factor is >0.95 and the primary control rate with radiotherapy is <91% is surgery preferable. The simplicity of this example is both deceptive and instructive. Superficially it may seem adequate, but closer scrutiny reveals its shortcomings: no provision is made for perioperative mortality or morbidity, the role of chemotherapy for patients who relapse is ignored, and there is no consideration of the implications of distant, as opposed to local, recurrence.

USE IN D I A G N O S I S The second major technique in decision analysis is the use of Bayes' theorem [6]. This finds its niche less in making decisions about treatment than in the analysis of the evidential weight that can be put on the result of a diagnostic test. Obviously, in a decision tree which incorporates both investigation and treatment strategies a Bayesian analysis may be embedded within the tree. The primary use of Bayes' theorem is to indicate the probability of disease given a positive, or indeed a negative, test result. This is usually termed the 'post test probability of disease'. The following information is required for a Bayesian analysis: 1. The pre-test (a priori) probability of disease: in assessment of screening tests this is the prevalence of the disease in the tested population. 2. The sensitivity of the test, the true positive rate (TPR): the proportion of patients with the disease who have a positive test result. 3. The specificity of the test: the proportion of individuals without disease who have a normal test result. The specificity is related to the false positive rate (FPR) of a test by: FPR=l-specificity. Bayes' theorem can be derived in a variety of ways and expressed in a variety of forms but the simplest expression is: Probability of disease given a positive test = TPR x prevalence [TPR x prevalence] + [FPR x (1-prevalence)] Bayes' formula depends upon knowledge of the FPR and TPR for a diagnostic test and these rates will depend upon the thresholds used to discriminate between normal and abnormal test results. An assessment of discriminatory thresholds may be necessary to ensure that a test is being used optimally [7]. This can be accomplished graphically using a receiver operating characteristic (ROC) curve. TPR

308

is plotted against FPR for a wide range of threshold values. A line drawn at 45° from the origin defines a test which conveys no information, since TPR always is equalled by FPR. An informative test shows a steep rise in TPR for a minimal rise in FPR; once TPR reaches its maximum then FPR also starts to increase. A similar approach has been used in therapeutic decision making; control rate is plotted against complication rate for a variety of dose levels [8-10].

OTHER USES A decision tree can, through the use of a Markov process, be adapted to incorporate the natural history of disease [11]. A Markov process is cyclical and the cycle length can be set appropriately for the clinical circumstances. During each cycle patients, having entered the first cycle in an initial state, for example " W E L L " , undergo transitions between states. A patient could move during one cycle from " W E L L " to " R E L A P S E " , remain in the state " R E L A P S E " for several cycles and then enter the state " D E A D " . The state " D E A D " would be defined as an absorbing state; once it has been entered the patient is removed from the Markov process. The transitions between states are governed by probabilities built into the model. In some respects a Markov process is similar to a life table and, as such, its potential usefulness in modelling the natural history of a disease can be readily appreciated. The optimal design of follow-up policies is a neglected, but important, area of clinical oncology. Cost effectiveness analysis using Markov processes incorporated into decision trees offers a means of approaching the problem logically. We have used such a model for Stage I non-seminomatous germ cell tumours of the testis and shown that the surveillance policies in current clinical use show a threefold variation in costs, but have approximately equal effectiveness [49]. Such models can be used prospectively to design efficient schedules.

LITERATURE REVIEW A literature search using Medline was carried out for the period from January 1965 to July 1990. The search criteria 'decision support systems', 'decision theory', and 'decision tree' were used to identify articles on clinical decision analysis. Only articles whose titles or keywords dealt with malignant disease were selected. Two main clinical journals regularly publish decision analyses: Medical Decision Making and the Journal of Clinical Epidemiology (formerly the Journal of Chronic Diseases). The contents pages of all issues of these journals between January 1965 and July 1990 were inspected to make sure that no relevant articles had been missed. No attempt was made to locate any unpublished articles. The articles unearthed by the search fell into three main groups:

A . J . Munro

1. Decision analytic approaches to therapeutic decision making in cancer 2. Decision analytic approaches to the investigation of patients with cancer 3. Decision analytic approaches to the the detection of cancer in the general population The current review deals only with those articles in the first category. There were 19 papers which, broadly speaking, dealt with decisions concerning management policy. These are listed in Table 1. Three additional articles dealt, in detail, with therapeutic decisions for individual patients [27-29]. These latter studies were published as clinical decision making rounds of the New England Medical Center and, presumably, their intent was as much didactic as practical. Two other articles dealt mainly with basic principles and contained limited practical information [30, 31]. The three articles dealing with individual patients are now briefly documented. 1. A 47-year-old male smoker presented with a myasthenic syndrome resembling the Eaton-Lambert syndrome. Bronchoscopy was normal. The question posed was whether he should be immediately treated with combined chemotherapy and radiotherapy, as if he had small cell lung cancer, or whether treatment should be delayed until a firm diagnosis could be made [27], The option of giving radical radiotherapy to the chest as sole therapy was not considered. The unequivocal conclusion was that an expectant policy was preferable to immediate treatment. 2. The second example [28] contains 19 figures and 78 references. It deals with a 62-year-old woman with stage TeNxM0 poorly-differentiated carcinoma of the bladder who had an uncomplicated myocardial infarction 7 days before a scheduled cystectomy. The question is, is it safe to delay surgery or should she, in spite of her recent infarct, be operated upon immediately? The possibility that this lady could be managed by radical radiotherapy with salvage cystectomy for failure was not considered. The ultimate conclusion is that a preoperative delay of about 5 weeks is the best management for this patient. 3. The third analysis concerned a 58-year-old HIVpositive male with Stage B carcinoma of the prostate [50]. The question concerned the relative risks both to patient and surgeon should the patient be treated with surgery or with radiotherapy. The analysis showed that, were the-patient to receive radiotherapy, he would lose 8 quality-adjusted months of life; were he to be treated by surgery then the surgeon would lose one quality adjusted life day.

DISCUSSION The conclusions from the analyses summarized in Table 1 are less than earth-shattering. Dogmas have not been overthrown, paradigms have not been shifted. The tools exist to perform, with minimal effort, complex and rigorous analyses. The question is, is any of this worthwhile? Are we witnessing a

Decision Analysis in Oncology

309

Table 1. A summary of decision analyses concerning therapeutic decision making in clinical oncology Reference

Disease

Problem

Method

End-point

Result

Comments

Corder and Ellwein 1984 [12]

Hodgkin's Stage I and II

Tree

Utility (01000)

(A) 822-878

Corder and Ellwein 1984 [13]

Hodgkln's Stage I and II

Tree

Utility (01000)

Results based on the utility values for morbidity assigned by 3 physicians Utilities overall were higher than for MOPP study

Leibenhaut et al. 1984 [14]

Hodgkin's CSI and II (supradiaphragmatic) Hodgkln's Stage IB and IIB

(A) Laparotomy and directed Rx vs. (B) MOPP for all (A) Laparotomy and directed Rx vs. (B) ChIVPP for all Prediction of presence or absence of disease below diaphragm (A) Laparotomy and directed Rx vs. (B) MOPP for all

Tree

Definition of low risk/high risk groups 5-year survival

Rutherford et al. 1980 [15]

(B) 807-921 (A) 848-934 (B) 851-954

Bayes

Not classical DA: treebased partitioning of clinical data Defines combinations of clinical findings for which (A) or (B) are appropriate

VS.

(C) TNI Fineberg 1980 [16]

Plante et al. 1987 [ 1 7 ]

18-yr old male Hodgkin's NS IIA LAG neg.

AJC Stage III carcinoma of pyriform sinus

(A) Laparotomy and directed Rx vs. (B) Mantle + upper abdominal XRT (A) Surgery vs. (B) Radiotherapy vs. (C) S + postop XRT

Tree

5-year survival

(C) Never indicated (A) 90.29% (B) 89.14%

Tree

Quality adjusted survival (weeks)

(A) 97

Utility not considered, Thresholds: for LAG false neg rate of 2%, for Lap mortality <1.8% Did not allow for salvage surgery in patients failing XRT All relapses were equivalent to death

(B) 72

(c) 108

VS.

Stalpers et al. 1989 [ 1 8 ]

T2NoM0 carcinoma of larynx

(D) Preop XRT + S (A) Surgery vs. (B) Radiotherapy

Tree

5-year-survival

(B) 85%

Henschke and Carcinoma of Flehinger 1967 the oral tongue [3]

(A) Neck dissection vs. (B) No dissection

Table

Andrews, 1985 Carcinoma of [10] glosso-palatine sulcus

Dose-optimization

ROC

Utility loss scaled to 100 (utility loss in N + patient who is not dissected) N/A

Metz et al. 1983 [8]

Carcinoma of nasopharynx

Dose-optimization

ROC

N/A

NcNeil et al. 1981 [19]

T3 carcinoma of the larynx

(A) Laryngectomy (B) Radiotherapy

Expected utility

% of pts who should choose XRT

(A) scan and operate if nodule present vs. (B) clinical follow-up

Tree

QALYs

Tree

QA months

Stockwell et al. Abnormalities of 1984 [20] the thyroid in adults irradiated as children

Simes 1985 [21] Treatment of (A) No specific Stage III nontreatment debulked vs. ovarian cancer in (B) oral alkylator a 60-yr-old vs. (C) combination i.v. chemotherapy

(D) 102 (A) 86%

(A) (A) (B) (B)

<2 >2 <2 >2

cm, 40 cm, 40 cm, 30 cm, 65

2000 rets

When utility for voice loss is added there is a threshold at 0.986 for voice quality after surgery (0-1) Neck dissection is only indicated for primary >2 cm

This "optimal' dose corresponds to 84% local control rate and 24% complication rate 2000-2200 rets No optimal dose for T3 for T I and T 2 ? o r T 4 'Optimal" dose for T 3 and T4 T 1 and T2 produces 75%92% local control and 10%-15% complication rates 3% when 30% 3- Flrefighters and yr survival, 19% executives differed in when 40% 3-yr their attitudes survival Sensitivity analysis showed importance of operative mortality in defining better decision (A) 37.97 Difference is equivalent to 10 days Economic costs were not (B) 37.998 considered Sensitivity analysis showed that the decision was robust (A) 23 Sensitivity analysis showed that the conclusion was fairly (B) 37 robust

(C) 34

310

A . J . Munro

Table 1. (Continued) Reference

Disease

Twiggs and Potish 1 9 8 4 [22]

Levine et al. 1985 [ 2 3 ]

Problem

Method

End-point

Result

Role of surgical (A) surgical stage staging in cancer vs. of the cervix (B) clinical stage

Tree

3-year survival

(A) (B) (A) (B)

Diagnosis and treatment of carcinoma of unknown primary

Tree

(A) Limited investigation and hmited therapy vs. (B) Comprehensive investigation and empirical chemoRx VS.

(C) Comprehensive investigation and limited therapy VS,

ZwetslootSchonk et al. 1989 [ 2 4 ]

(D) Limited investigation and empirical chemoRx Management of (A) Non-curative Rx clinical stage B vs. carcinoma of the (B) Lymphprostate adenectomy with selection

Tree

Comments

76% 63% 6.5% 6.1%

Data based on their own experience but patients Risk of enteric were carefully selected morbidity for surgical staging (i.e. finding cannot be generalized) $ cost per 1000 (A) $761 000, Sensitivity analysis patients 11% showed that two assumptions influenced results: 1-yr survival (B) $8.6M, 1. That symptomatic care 11.50% and empirical chemotherapy produced the same survival when (C) $2.7M, primary site was not 11.50% found 2. Equal probabilities of 1-year survival for limited (D) $7.3M, 11% and comprehensive search strategies when primary not found Also looked carefully at 10-year (A) 56.2% burdens associated with survival lymphadenectomy and (B) 66.1% with curative Rx defined threshold ratios

VS.

ZwetslootSchonk et al. 1989 [ 2 4 ]

Management of clinical Stage B carcinoma of the prostate

(C) Curative Rx (A) LAG followed, if Utlhty negative, by thresholds lymphadenectomy vs. (B) Lymphadenectomy

(C) 65.7% Too complex to summarize

Choice depended upon likelihood of metastases and upon choice of curative Rx (whether XRT or surgery)

VS.

Moskowltz and Limb sarcomas Pauker 1985 in adults [251

(C) Curative Rx (A) Limb sparing Rx vs.

Tree

Life-years, QUALYs

(B) Amputation

(A) 16.1 13 1 (QA)

(B) 18.3, 12.8 (QA)

McNeil et al. 1978 [26]

Stage I or II lung (A) Surgery cancer vs. (B) Radical radiotherapy

Expected utility

% of patients who should choose XRT

71% of 70-yrolds 64% of 60-yrolds

Extensive sensitivity analysis showing that risk preferences affect the decision The baseline analysis hinges on quality of life considerations When survival alone is considered then no patients should choose radiotherapy

MOPP, Mustine, Vincristine, Procarbazine, Prednisolone; CS, Clinical stage: Chl Vpp, Chlorambucil, Vincristine, Procarbazine, Prednisolone; TNI, Total nodal irradiation; LAG, Lymphangiogram; Lap, Laparotomy; NS, Nodular sclerosis; AJC, American Joint Staging Committee; S, Surgery; N+, node positive; N/A, not applicable.

variant of Parkinson's Law, with the work expanding to fill the programs available? Ennui, rather than outright disagreement, may explain why decision analysis has not been enthusiastically adopted by oncologists. The advantages and disadvantages of decision analysis have been extensively described and debated [32-39]. Table 2 summarizes some of the arguments that have been put forward. Clearly, if the basic structure of the decision tree is unsound the analysis can have no real relevance for clinical practice. In the analysis of immediate versus delayed cystectomy following myocardial infarction, the failure to consider radiotherapy nullifies any conclusions concerning practical management [28]. The power of the analyses of management policies for Hodgkin's disease is eroded by their failure to consider the problem of second malignancies [12, 13, 15, 16]. A similar problem exists for studies in cancer

Table 2. A summary of the main advantages and disadvantages of decision analysis Advantages

Disadvantages

Explicit Data driven Formal and logical Didactically useful Identification of blind spots Shows where decision hinges Patients' preferences can be built in

Tree structure Assessment of probabilities Assessment of outcome Time consuming Results may be equivocal

of the head and neck. Tree structure need not be totally inclusive; it is not necessary to add the nodes 'hit by a bus' or 'shot by jealous lover' to every tree, but important consequences must not be neglected. Statistics apply to populations, not to individuals.

Decision Analysis in Oncology

If, as proponents of decision analysis would wish us to believe [35], the technique can be used at the bedside for making individual decisions about individual patients, then there is a major problem with choosing which probabilities to use. As Keynes has pointed out: 'there is no direct relation between the truth of a proposition and its probability' [40]. The problem with probabilities may manifest itself as the all or none phenomenon. The surgeon who performs prostatectomy on an HIV positive patient may, on average, risk losing one quality-adjusted life day [50]. But it is not the average that is important to her. If she contracts AIDS as a result of a needlestick then her whole life, not just one day, is in jeopardy. However, a major advantage of decision analysis is that, by forcing both processes and assumptions to be made explicit, it can draw attention to areas in which data are inadequate. For example, in clinical oncology the data most conspicuously lacking are those dealing with patients' perceptions of morbidity and values concerning outcome of treatment. These do not always accord with the assumptions of those caring for them. Many decision analyses have relied on fabricating the missing data or seeking the opinion of whoever is nearby, usually physicians or nurses, sometimes firemen [19]. There has been little attempt to involve patients directly in the process of making judgements about the relative disutilities of various side-effects of treatment. In only one of the studies were the values of actual patients incorporated into utility assessments [26]. It is not easy to assess patients' preferences accurately and reproducibly. Many factors will influence the results achieved [4143]. But the difficulty of the problem is not an excuse for ignoring it. There is also a genuine problem with the assessment of outcome. The simplest measure, the QALY, can be heavily criticized on ethical grounds [5]. By its very nature it overvalues treatment for younger patients and discriminates against palliative, as opposed to radical, approaches to treatment. The assessment of utility, and disutility, highlights some of the dehumanising aspects of decision analysis. Even the vocabulary reflects this; disutility is a coy periphrasis for pain, suffering and anguish. A calculus of suffering is not an attractive concept. As Ransohoff has pointed out, it is difficult to quantify the disutility experienced by a family who watch a beloved relative dying a painful and protracted death [38]. If it proves impossible accurately and fairly to measure utilities then decision analysis will have to confine itself to problems in which utility assessment does not, or can not, affect the conclusion. Decision analysis often demonstrates that a decision is finely balanced, is a 'toss up' [44]. Sophisticated analysis is not always required to identify such problems, since these are often already the subject of strenuous debate. It is no surprise to find that the decision whether or not to perform staging laparotomy in early stage Hodgkin's disease is finely balanced. Similarly, the critical influence of quality of life after treatment in the conservative management of sarcomas of the limb is not unexpected. Decision analysis can be very time consuming. In a study of the morbidity of therapy in Hodgkin's disease it took 4 to 6 hours to elicit the utility

311

functions from one, medically qualified, surrogate [13]. Simply obtaining and assessing the relevant literature for an individual problem can take several hours. The advantage is, of course, that the decision maker is forced to produce hard data to drive the analysis and is not able, as in informal decision making, to rely on opinion and prejudice. As long as the topic remains impregnated with jargon, oncologists are going to remain unaware or sceptical, or both. But recent articles from the Glasgow school and others show how even complex ideas can be presented simply and acceptably in general medical journals [45-47]. Others have outlined the criteria which should be satisfied for computer assisted decision systems to be acceptable to physicians [48]. One of the main points is that overt explanation in clear and concise terms, as well as manifest common sense, is essential for such approaches to prove generally acceptable.

CONCLUSION Decision analysis is no panacea. The mismatch between demands and resources for health care cannot be solved simply by improving the cost effectiveness of doctors' decisions. Nor is it a complete chimera; it is monstrous in only a few aspects. It offers less than its evangelists promise but more than its detractors would claim. Its use should be judiciously circumscribed when making decisions about the management of individual patients, but it can have a useful role in assessing policies for managing groups of patients. Its major role, for the present at least, is didactic. It can identify those factors to which close attention must be paid for the best decision to be made. Socrates claimed that it was the fact that he knew that he knew nothing that distinguished him from other men and earned him his reputation for wisdom. Decision analysis can teach a similar lesson.

Acknowledgements. I would like to thank Dr R. Knill-Jones for drawing my attention to several papers that had escaped my notice. I would like to thank the following individuals whose ideas and conversations have stimulated my interest in this area: Padraig Warde, Hilary Llewellyn-Thomas, James Till and William MacKillop.

References 1. Raiffa H. Decision analysis. Introductory lectures on choices under uncertainty, Reading, MA: Addison-Wesley, 1968. 2. Weinstein MC, Fineberg HV. Clinical decision analysis. Philadelphia: Saunders, 1980. 3. Henschke UK, Flehinger BJ. Decision theory in cancer therapy. Cancer 1967;20:1819-26 4. Baynes CF. I Ching, the Richard Wilhelm translation rendered into English [translation]. London: Routledge and Kegan Paul, 1951. 5. Harris J. QALYfying the value of life. J Med Ethics 1987;13:117-23. 6. Bayes T. an essay towards solving a problem in the doctrine of chances. Proc R Soc 1763;13:370-418. 7. Sackett DL, Haynes RB, Tugwell P. In: Sackett DL, Haynes

312

8.

9. 10. 11. 12. 13. 14.

15. 16. 17. 18. 19. 20.

21. 22. 23. 24.

25. 26. 27. 28.

A . J . Munro RB, Tugwell P, editors. Clinical epldemiology. A basic science for clinical medicine. Boston: Little Brown, 1985:59138. Metz CE, Tokars RP, Kronman HB, et al. Maximum likelihood estimation of dose-response parameters for therapeutic operating characteristic (TOC) analysis of carcinoma of the nasopharynx. Int J Radiat Oncol Biol Phys 1983;8:1185-92. Moore DH, Mendelsohn ML. Optimal treatment levels in cancer therapy. Cancer 1972;30:97-106. Andrews JR. Benefit, risk, and optimization by ROC analysis in cancer radiotherapy. Int J Radiat Oncol Biol Phys 1985;11:1557-62. Beck JR, Pauker SG. The Markov process in medical prognosis. Med Decis Making 1983;3:419-58. Corder MP, Ellwein LB. Comparison of treatments for symptomatic Hodgkin's lymphoma employing decision analysis. Am J Clin Oncol (CCT) 1984;6:33-8. Corder MP, Ellwein LB. A decision-analysis methodology for consideration of morbidity factors in clinical decision-making. Am J Clin Oncol (CCT) 1984:6:19-32. Leibenhaut MH, Hoppe RT, Efron B, et al. Prognostic indicators of laparotomy findings in clinical Stage l-II supradiaphragmatic Hodgkin's disease. J Clin Oncol 1989;7:81-91. Rutherford CJ, Desforges JF, Davies B, et al. The decision to perform staging laparotomy in symptomatic Hodgkin's disease. Br J Haematol i980;44:347-58. Fineberg HV. Decision trees: construction, uses, and limits. Bull Cancer 1980:67:395-404 Plante DA, Piccirillo JF, Sofferman RA. Decision analysis of treatment options in pyriform sinus carcinoma. Med Decis Making 1987;7:74-83. Stalpers L J, Verbeek AL, van Daal WA. Radiotherapy or surgery for T2NoM0 glottic carcinoma? A decision-analytic approach. Radiother Oncol 1989;14:209-17. McNeil B J, Weichselbaum R, Pauker SG. Speech and survival. Tradeoffs between quality and quantity of life in laryngeal cancer. N Engl J Med 1981;305:982-7. Stockwell RM, Barry M, Davidoff F. Managing thyroid abnormalities in adults exposed to upper body irradiation in childhood: a decision analysis. Should patients without palpable nodules be scanned and should those with scan defects be subjected to subtotal thyroidectomy? J Clin Endocrinol Metab 1984;58:804-12. SimesRJ. Treatment selection for cancer patients: application of statistical decision theory to the treatment of advanced ovarian cancer. J Chron Dis 1985;38:171-86. Twiggs LB,Potish RA. Decision theory analysis of the enterlc morbidity and surgical staging in the treatment of advanced cervical cancer. Am J Obstet Gynecol 1984;148:134-40. Levine ML, Drummond MF, Labelle RJ. Cost-effectiveness in the diagnosis and treatment of carcinoma of unknown primary origin. Can Med Assoc J 1985;133:977-87. Zwetsloot-Schonk JHM, Pliskin JS, Leer JWH. Analysis of diagnostic therapeutic strategies in prostate cancer: I. When should a pelvic lymphadenectomy for staging be performed? Theor Surg 1989;4:67-78. Moskowitz A J, Pauker SG A decision analytic approach to limb-sparing treatment for adult soft tissue and osteogenic sarcoma. Cancer Treat Syrup 1985;3:11-26. NcNeil B J, Weichselbaum R, Pauker SG. Fallacy of the fiveyear survival in lung cancer. N Engl J Med 1978;299:1397-401. Eckman MH, Robert N J, Parkinson DR, et al. EatonLambert syndrome and small cell lung cancer: side effects and certainty. Med Decis Making 1986;6:174-86. Eckman MH, NcNutt RA, Parkinson DR, et al. The timing of

29. 30. 31. 32. 33. 34 35. 36. 37. 38. 39. 40. 41. 42. 43

44. 45. 46.

47 48. 49.

50

radical cystectomy after recent myocardial infarction: waiting for Godot. Med Decis Making 1988;7:52-66. Roach PJ, Fleming C, Hagen MD, et al. Prostatic cancer in a patient with asymptomatic HIV infection: are some lives more equal than others? Med Decis Making 1988;8:132-44. Rubln P. Discussion: the choice and meaning of tolerance in radiotherapeutics. The Bayesian model. Front Radiat Ther Oncol 1972;6:499-511. Munro AJ. A graphical method for the analysis of decisions in clinical oncology. Clin Radiol 1986;37:263-6. Detsky AS, Redelmeier D, Abrams HB. What's wrong with decision analysis? Can the left brain influence the right? J Chron Dis 1987;40:831-6. Pauker SG, Kassirer JP. Marchiafava-Bignami disease among academicians in Toronto: can decision analysis help? J Chron Dis 1987;40'837-8. Albert DA. Decision theory in medicine: a review and critique. Milbank Memorial Fund Quarterly: Health and Society 1978;56:362-401. Pauker SG, Kassirer JP Decision analysis N Engl J Med 1987;316:250-8. Brown RV. Do managers find decision theory useful? Harvard Business Review 1970;48:78-89. Feinstein AR. Clinical biostatistics, XXXIX. The haze of Bayes, the aerial palaces of decision analysis, and the computerized ouija board. Clin Pharmacol Ther 1977;21:482-96. Ransohoff DF, Feinstein AR. Is decision analysis useful in clinical medicine? Yale J Biol Med 1976;49:165-8. Cebul RC. 'A look at the chief complaints' revisited. Current obstacles and opportumties for decision analysis. Med Decis Making 1984;4:271-83. Keynes JM. A treatise on probability. New York: Harper and Row, 1962:321-3. Llwellyn-Thomas H, Sutherland HJ, Tibshirani R, et al. The measurement of patients' values in medicine. Med Decis Making 1982;2:449-62. Llewellyn-Thomas H, Sutherland H J, Ciampi A, et al. The assessment of values in laryngeal cancer: reliability of measurement methods. J Chron Dis 1984;37:283-91. O'Connor AM, Boyd NF, Tritchler DL, et al. Eliclting preferences for alternative cancer drug treatments. The influence of framing, medium, and rater variables. Med Decis Making 1985;5:453-63. Kassirer JP, Pauker SG. The toss-up. N Engl J Med 1981 ;305:1467-9. Knill-Jones RP. Diagnostic systems as an aid to clinical decision making. Br Med J 1987;295:1392-6. Spiegelhalter D J, Knill-Jones RP. Statistical and knowledge based approaches to clinical decision-support systems with an application in gastroenterology. J R Stat Soc (A) 1984;147:3577. Seymour DG, Green M, Vaz FG. Making better decisions: construction of clinical scoring systems by the SpiegelhalterKmll-Jones approach. Br Med J 1990;300:223-6. Teach RL, Shortliffe EH. An analysis of physician attitudes regarding computer-based clinical consultation systems. Comput Biomed Res 1981;14:542-58. Munro AJ, Warde PR. The use of a Markov process to simulate and assess follow-up policies for patients with malignant disease: surveillance for Stage I non-seminomatous tumours of the testis. Med Decis Making 1991;11:131-9. Roach PJ, Fleming C, Hagen MD, Pauker SG. Prostatic cancer in a patient with asymptomatic HIV infection: are some lives more equal than others? Med Decis Making 1988;8:13244.

Received for pubhcation 1991 Accepted following revision May 1992