Cost-effectiveness of FDG-PET for staging non–small cell lung cancer: a decision analysis

ORIGINAL ARTICLES: GENERAL THORACIC

Cost-Effectiveness of FDG-PET for Staging Non–Small Cell Lung Cancer: A Decision Analysis Walter J. Scott, MD, James Shepherd, MS, and Sanjiv Sam Gambhir, MD, PhD Department of Surgery, Creighton University, Omaha, Nebraska; and The Crump Institute for Biological Imaging and Department of Molecular and Medical Pharmacology, Division of Nuclear Medicine, and Department of Biomathematics, University of California Los Angeles School of Medicine, Los Angeles, California

Background. Preliminary studies have shown that thoracic positron emission tomography (PET) is more accurate than thoracic computed tomography (CT) for the staging of non–small cell lung carcinoma. In the present study the cost-effectiveness, as measured by national Medicare reimbursed costs, and patient life expectancy are used to compare several thoracic PET-based strategies with a conventional thoracic CT-based strategy for preoperative staging. Methods. Five decision strategies for selection of potential surgical candidates were compared; thoracic CT alone or four different strategies that use thoracic CT plus thoracic PET. The various paths of each strategy are dependent on numerous variables that were determined from a review of the medical literature. Life expectancy was calculated using the declining exponential approximation of life expectancy and reduced on the basis of procedural morbidity and mortality. Costs were based on

national Medicare reimbursed costs. For all possible outcomes of each strategy, the expected cost and projected life expectancy were determined. The effects of changing one or more variables on the expected cost and life expectancy were studied using sensitivity analysis. Results. A strategy that uses PET only after a negative CT study is shown to be a cost-effective alternative to the CT-alone strategy ($25,286 per life-year saved). Conclusions. These results show through rigorous decision tree analysis the potential cost-effectiveness of using thoracic PET in the management of non–small cell lung carcinoma. Greater use of thoracic PET for non– small cell lung carcinoma staging is warranted, and further clinical trials should help to validate the analytic results predicted from this study.

C

Gambhir and colleagues [6] indicated that PET has a cost-effective role. The current study builds on and enhances the analysis of Gambhir and coworkers by investigating additional management strategies, precisely modeling the dependence between CT and PET scans, as well as updating the relative costs of procedures and changing to the more universal Medicare reimbursement values. The goals of the present study were to quantitatively model under what conditions PET could play a cost-effective role in the staging of NSCLC by more accurately determining the presence or absence of metastases to mediastinal (stage N2 or N3) lymph nodes.

omputed tomography (CT) of the chest is a standard test in the evaluation of a patient with lung cancer. Over the past decade and a half, numerous studies have evaluated the diagnostic accuracy of CT for the assessment of mediastinal lymph nodes. More recent studies have shown that positron emission tomography (PET) using the glucose analogue [18F]2-fluoro-2-deoxy-dglucose (FDG) is more accurate than CT alone for determining the presence or absence of mediastinal (stage N2 or N3) lymph node metastases in patients with non–small cell lung cancer (NSCLC) [1–5]. Rational decisions regarding the use of new technology, such as the use of FDG-PET in lung cancer staging, depend on an assessment of its effects under actual or expected conditions of use. The technique of decision analysis offers useful ways to organize information and evaluate strategies involved in staging the patient with lung cancer [6]. By including information on cost and life expectancy, decision analysis can relate costs and benefits of a lung cancer staging strategy. A previously published analysis of PET in the staging of NSCLC by Presented at the Thirty-fourth Annual Meeting of The Society of Thoracic Surgeons, New Orleans, LA, Jan 26 –28, 1998. Address reprint requests to Dr Gambhir, Crump Institute for Biological Imaging, UCLA School of Medicine, A-222B JLNRC, 700 Westwood Plaza, Box 951770, Los Angeles, CA 90095-1770 (e-mail: sgambhir@mednet. ucla.edu, [email protected]).

© 1998 by The Society of Thoracic Surgeons Published by Elsevier Science Inc

(Ann Thorac Surg 1998;66:1876 – 85) © 1998 by The Society of Thoracic Surgeons

Material and Methods Decision tree models were constructed with four competing strategies (one with thoracic CT alone and three that included both thoracic CT and thoracic PET). To each possible outcome of each strategy, estimated national reimbursed Medicare costs and patient life expectancy were assigned. The explicit probabilities of each outcome in the tree were obtained as a function of the variables shown in Table 1. These probabilities were computed using simple Bayesian analysis [7, 8]. Multiple decision trees were evaluated because there are several ways to incorporate PET into an NSCLC staging strategy. The medical literature was surveyed to obtain a mean 0003-4975/98/$19.00 PII S0003-4975(98)01055-8

Ann Thorac Surg 1998;66:1876 – 85

SCOTT ET AL COST-EFFECTIVENESS OF FDG-PET FOR NSCLC

Table 1. Baseline and Range of All Relevant Variables Used in Analysis Variable Prevalence (%) CT sensitivity (%) CT specificity (%) PET sensitivity, CT1 (%) PET sensitivity, CT2 (%) PET specificity, CT1 (%) PET specificity, CT2 (%) Mortality rate (%) PET CT Operation Biopsy Morbidity (y) Operation Biopsy Life expectancy (y) Surgical cure Unresectable disease Reimbursed costs ($) Thoracic CT Thoracic PET Biopsy Operation

Baseline 31 67 73 89 76 81 97

Range 28 –38 61–73 62– 86 80 –100 67–90 60 –100 83–100

0.0 0.0025 3.0 0.3

0 –1 0 –1 0 –20 0 –5

0.083 0.007

0 –1.0 0 – 0.1

7.0 1.0 378 2,000 4,360 18,500

CT 5 computed tomography; CT1 5 CT positive; negative; PET 5 positron emission tomography.

1877

Structure of Decision Trees Five strategies labeled A to E were modeled in this analysis. Simplified versions of strategies A to D are depicted in Figure 1. Strategy A is a baseline strategy that uses only thoracic CT for noninvasive preoperative staging, whereas strategies B to E include the use of a thoracic PET scan in different approaches. Strategy A and strategies similar to D and E were previously analyzed by Gambhir and coworkers [6]. We constructed these management algorithms according to several assumptions. All the algorithms assume that if a patient has a positive mediastinal lymph node biopsy (metastases to N2/N3 lymph nodes), he or she is not a candidate for thoracotomy and lung resection as sole therapy. As a result, the management algorithms

1–15 0.1–2 300 –7,000 1,200 –2,200 2,000 – 6,000 15,000 –25,000 CT2 5 CT

and range for all variables of interest. A comprehensive literature survey was used to arrive at the sensitivity and specificity of PET. Previously pooled data for the sensitivity and specificity of CT were used [6]. The literature was also surveyed to determine the prevalence of N2 or N3 metastases in patients undergoing staging for NSCLC, morbidity and mortality for all studies in each decision tree, as well as life expectancy for an otherwise healthy 64-year-old white man. Calculations of expected cost and life expectancy of competing strategies were calculated by summing the products of the probabilities and values (in terms of national Medicare reimbursed dollar cost and patient life expectancy) of the outcomes for each strategy. To compare each PET-based strategy with the CTalone strategy (strategy A), an incremental costeffectiveness ratio (ICER) was formed [(Costx 2 CostA)/ (LEx 2 LEA)], where LE represents life expectancy [8]. Finally, because the precise value of the various variables are not known, a sensitivity analysis was performed for each decision tree. This involved evaluating each tree over a particular variable’s range and determining the effect on the ICER. In addition, multiple variables that favored the winning strategy (strategy B) were shifted by 10% to understand the effects of simultaneously varying multiple variables. New software developed at the University of California Los Angeles [9] was used to construct and analyze each decision tree.

Fig 1. Decision tree showing the four strategies analyzed (A to D). Square represents a decision node; circles are chance nodes; and triangles are terminal nodes. (Bx 5 biopsy; CT 5 computed tomography; NSCLC 5 non–small cell lung carcinoma; PET 5 positron emission tomography.)

1878

SCOTT ET AL COST-EFFECTIVENESS OF FDG-PET FOR NSCLC

exclude a patient with preoperatively documented N2 or N3 metastases from undergoing surgical resection. The cost of subsequent therapies such as chemotherapy and radiation therapy were not included in this analysis. A patient diverted from surgical intervention incurs subsequent treatment costs similar to those of the patient found to have advanced stage disease at thoracotomy (terminal costs). Therefore, our model primarily accounts for the difference in treatment cost that is due to thoracotomy. We also assumed that based on the relatively high positive predictive value of biopsy, there will be negligibly few patients who will be falsely positive after biopsy [10]. In strategies A through D all patients have a biopsy to confirm lymph node involvement before being diverted from operation. Therefore, in these strategies no patients who can benefit from a thoracotomy are denied this intervention, whereas in strategy E some patients who are surgical candidates are mistakenly diverted from operation. For this reason strategy E is only treated in brief in this analysis. In strategy A those patients who are CT positive for contralateral or mediastinal lymph node involvement, or both, have a biopsy to confirm that the patient is not a surgical candidate. If the biopsy results are negative, the patient proceeds to thoracotomy. Patients who are CT negative proceed directly to thoracotomy. In strategy B those patients who are CT positive undergo a confirming biopsy, and if the biopsy results are negative, they proceed to the necessary operation. The CT-negative patients undergo thoracic PET. If the PET results are positive, the patient has a confirming biopsy. If the PET results are negative, the patient proceeds to operation. This strategy potentially reduces the number of thoracotomies performed in patients who have N2 or N3 lymph node metastasis and a false-negative CT compared with strategy A. In strategy C all patients undergo thoracic PET regardless of the CT result. The PET-positive patients proceed to biopsy, and the PET-negative patients proceed to thoracotomy. This strategy uses PET for the staging decisions and uses CT only for anatomic information. In strategy D all patients undergo thoracic PET regardless of the CT result. All CT-positive patients also undergo a confirming biopsy regardless of the PET result. The CT-negative patients with PET positivity undergo a confirming biopsy; those with PET negativity proceed directly to thoracotomy. This strategy uses the combined CT and PET information for staging. Strategy E (not shown in Fig 1) is similar to strategy D, except that patients who are positive on both CT and PET do not have a confirming biopsy but are treated without thoracotomy. Some of these patients will have falsepositive results on both CT and PET and will therefore be incorrectly diverted from a beneficial operation.

Survey of Medical Literature The prevalence of contralateral or mediastinal involvement, or both, in patients with NSCLC was estimated at 31% (range, 28% to 38%) on the basis of our review of the

Ann Thorac Surg 1998;66:1876 – 85

literature [11–15]. This value is not the prevalence of NSCLC, but the prevalence of N2 or N3 lymph node metastases in those patients with a histologic diagnosis of NSCLC who are being considered for thoracotomy. The prevalence baseline value and range are given in Table 1, along with values for all other variables used in this analysis. We used pooled sensitivity (67%) and specificity (73%) for thoracic CT based on five published studies [11–15] that used a patient-based analysis (Table 2), as has been previously done [6]. Using receiver operating characteristic (ROC) analysis and the mean threshold criteria [16], the sensitivity and specificity were estimated as 66% and 76%, respectively. Because of the small difference in the estimates, we used the pooled estimates, in keeping with the previous analysis [6]. A CT criterion of greater than or equal to 1 cm lymph node size in the short-axis diameter to represent lymph node metastasis was used in these studies. At the time of this writing, only five studies of thoracic PET (210 patients) contained sufficient information (presented data on a per-patient basis with CT scan diagnosis, defined N stage as “positive” if N2 or N3 lymph nodes contained metastases) to allow us to calculate the sensitivity and specificity of PET [1–5]. Because the sensitivity and specificity of PET are not totally independent of those of CT, we determined the sensitivity and specificity of PET after negative (76% and 97%, respectively) and positive CT results (89% and 81%, respectively) and used these values for PET according to whether the preceding CT results were negative (Table 3) or positive (Table 4). The results are from a pooled analysis of the five studies, although ROC curve and weighted mean analyses gave results within 1% agreement of the pooled results, similar to the comparisons given for CT previously. The reported mortality associated with surgical resection of lung cancer ranges from 2.4% to 10%. The largest study, by Ginsberg and colleagues [17], reported an overall mortality of 3.7% and a mortality of only 2.9% for thoracotomy. We chose a baseline surgical mortality of 3% (range, 0% to 20%). In addition, we subtracted 1 month (0.083 year; range, 0 to 1 year) for the morbidity associated with the recovery from thoracotomy, as others have done in decision analyses involving surgical intervention for lung cancer [18 –20]. The mortality and morbidity for biopsy procedures were derived from studies in the literature. These were discussed in detail in a previous paper [6]. For the present analysis we used a baseline mortality rate of 0.3% (range, 0% to 5%) for the biopsy procedure, as previously reported [18]. We estimated the average morbidity for anterior mediastinotomy to be approximately 0.028 year (10 days), one-third that for curative surgical resection. Because the morbidity associated with other biopsy procedures (transthoracic needle biopsy, transbronchial biopsy) is less significant, we used a baseline morbidity of 0.007 year (2.5 days; range, 0 to 0.1 year) for the biopsy procedure. The accuracy of biopsy was assumed to be 100%. The risk associated with CT is primarily attributable to

Ann Thorac Surg 1998;66:1876 – 85

SCOTT ET AL COST-EFFECTIVENESS OF FDG-PET FOR NSCLC

1879

Table 2. Individual and Combined Sensitivities and Specificities of Chest Computed Tomography for Staging of Non–Small Cell Lung Carcinoma (Based on Patient Analysis) First Author [ref no.]

Year

Gross [11] McLoud [12] Primack [15] White [14] Dillemans [13]

1988 1992 1994 1994 1994

No. of Patients

Total No. 5 number;

Prevalence (%)

Sensitivity (%)

Specificity (%)

39 143 159 94 569

11/39 (28) 42/143 (30) 60/159 (38) 28/94 (30) 174/569 (31)

8/11 (73) 27/42 (64) 38/60 (63) 17/28 (61) 120/174 (69)

23/28 (82) 63/101 (62) 85/99 (86) 50/66 (76) 280/395 (71)

1,004

315/1,004 (31)

210/31 (67)

501/689 (73)

ref 5 reference.

administration of intravenous contrast material. We previously chose a baseline mortality of 0.0025% (range, 0% to 1%) [6]. The risk associated with PET is assumed to be negligible because there have been no reports to date of reactions or complications from the injection of FDG. Baseline life expectancy was calculated using the declining exponential approximation of life expectancy method developed by Beck et al. [21]: Average life expectancy (years) 5 1/(ASR 1 DSR), where ASR is the age-, sex-, and race-specific annual mortality rate of the general population, and DSR is the additional average mortality rate attributable to the patient’s disease. The age-, sex-, and race-specific mortality rate for a healthy 64-year-old white man is 0.067 (range, 1 to 15 years). The disease-specific mortality rate for a 2.3-cm cancer (average diameter of T1N0M0 tumors) has been estimated to be 0.075 [21]. The combined annual mortality rate for our baseline patient is equal to 0.142, and his life expectancy would be the reciprocal of this sum, 7.0 years. We used a baseline of 7.0 years (range, 1 to 15 years). The life expectancy for unresectable lung cancer for patients with highly advanced disease (evident on chest radiography) has been shown to be 0.47 years according to surgical data and the declining exponential approximation of life expectancy method [19]. Although there are few data concerning the life expectancy of patients with mediastinal metastasis not evident on the chest radiograph, these patients are expected to live longer than those with metastasis evident on the chest radiograph. Therefore, we chose a slightly greater baseline value of 1.0 years (range, 0.1 to 2 years). For patients with falsepositive results on both CT and PET and are not operated

on, we chose a mean life expectancy of 2 years, as previously reported [6]. We assigned a value of 0.0 year for all death outcomes, as is usually done [19, 20]. We estimated the mean national Medicare reimbursed dollar costs (combined technical and professional charges) of thoracic CT, biopsy, and curative thoracotomy, as shown in Table 1. The Medicare reimbursement for diagnosis related group 75 (major thoracotomy) is based on several characteristics, such as the presence of the hospital in a large urban area or other area, the number of indigent patients cared for by that hospital, whether the hospital is a teaching hospital (including the ratio of residents and trainees to the number of hospital beds), regional wages, capital costs, and other important factors. Therefore, an average, national figure that is relevant to all possible practices is difficult to derive. We used a baseline reimbursement value of $18,500 derived from estimates of the highest and lowest Medicare reimbursement values ($14,000 to $21,000) for the relevant diagnosis related group, combined with estimates of Medicare reimbursement for the relevant professional services (Current Procedural Terminology codes). The biopsy costs can vary considerably depending on the exact type of biopsy performed. We assume that two of three of all biopsies are mediastinoscopies and the rest are mediastinotomies. Thoracic PET reimbursement for NSCLC was approved by the Health Care Financing Administration at the end of 1997, and the amount of reimbursement was set at $2,080 (global fee, including technical, professional, and tracer components) in June 1998. This value will vary according to many factors, including those mentioned previously regarding thora-

Table 3. Positron Emission Tomographic Accuracy: Negative Results on Computed Tomography First Author [ref no.]

Year

No. of Patients

TP

FP

FN

TN

Sensitivity

Specificity

Guhlman [1] Bury [2] Scott [3] Valk [4] Scott [5]

1997 1996 1996 1995 1994

16 34 18 47 22

3 4 3 5 1

0 0 0 0 3

0 0 0 4 1

13 30 15 38 17

1.000 1.000 1.000 0.556 0.500

1.000 1.000 1.000 1.000 0.850

137

16

3

5

113

0.762

0.974

Total FP 5 false positive;

TP 5 true positive;

other abbreviations as in Table 2.

1880

SCOTT ET AL COST-EFFECTIVENESS OF FDG-PET FOR NSCLC

Ann Thorac Surg 1998;66:1876 – 85

Table 4. Positron Emission Tomographic Accuracy: Positive Results on Computed Tomography First Author [ref no.]

Year

No. of Patients

TP

FP

FN

TN

Sensitivity

Specificity

Guhlman [1] Bury [2] Scott [3] Valk [4] Scott [5]

1997 1996 1996 1995 1994

16 16 9 29 3

10 11 6 14 1

0 1 0 4 0

2 2 0 1 0

4 2 3 10 2

0.833 0.846 1.000 0.933 1.000

1.000 0.667 1.000 0.714 1.000

73

42

5

5

21

0.894

0.808

Total Abbreviations as in Tables 2 and 3.

that generates an ICER of $25,286 per life-year saved. Table 5 (row 1) shows that strategy B is the most costeffective strategy if one is trying to maximize life expectancy while incurring the least additional cost. Strategy C would be the best strategy if increasing life expectancy were the sole consideration; however, the higher expected cost of strategy C gives an ICER of $70,889, greater than the acceptable limit of $50,000 per life-year saved. Table 5 also includes results for changing several of the variable values. Rows 2, 3, 5, and 6 show results for changing a single value. Rows 4 and 7 show results for simultaneously altering the sensitivity and specificity of PET and CT, respectively. Row 4 shows the results for using the same estimated values of PET sensitivity and specificity regardless of the CT results. This illustrates that modeling CT and PET as statistically independent tests leads to higher expected costs and lower life expectancy. Rows 8 and 9 of Table 5 are the analytic results for altering several values by 10% that penalize the alternative strategies. Row 8 decreases all PET sensitivity and specificity values by 10% of their baseline values (sensitivity and specificity of 80% and 73%, respectively, when CT results are positive and 69% and 87%, respectively, when CT results are negative). This analysis shows that strategy B remains the cost-effective alternative when PET accuracy is penalized. Row 9 of Table 5 penalizes PET sensitivity and specificity and increases the cost of

cotomy reimbursement. We used a mean PET national Medicare reimbursed cost of $2,000 (range, $1,200 to $2,200).

Results Shown in the first row of Table 5 are the results of cost, life expectancy, and ICER for baseline estimates of all variables for strategies A to D. The ICER compares the cost of additional life for each of strategies B, C, and D with that for strategy A. A negative ICER value (values shown in parenthesis) indicates a dominance of the alternative strategy over strategy A (both lower cost and higher life expectancy). Table 5 indicates that strategy B is the most cost-effective strategy based on a generally held criteria that society is willing to spend up to $50,000 for an additional year of life [22]. Strategy E (not shown in Fig 1) has the lowest expected cost ($16,095) but also has the lowest life expectancy (4.76 years). Because strategy E diverts patients from operation without a confirming biopsy, patients that have false positive results on both CT and PET are incorrectly diverted from operation. This reduces cost, but with a significant reduction in life expectancy. Because 3.5% of strategy E patients miss undergoing a beneficial thoracotomy, this strategy was not considered further. Strategy B is preferred to strategy A because strategy B yields higher life expectancy at an additional expense Table 5. Results for Baseline Values and Sensitivity Analysis Strategy A Row 1 2 3 4 5 6 7 8 9 a

Variable

Value

Baseline ... PET cost $1,600 PET cost $1,700 PET S/Sp 0.87/0.94 Operation $14,000 Operation $22,000 CT S/Sp 0.69/0.79 Penalize 10% S/Sp Penalize 10% all

Strategy B

Strategy C

Strategy D

Cost ($) LE (y) Cost ($) LE (y) ICER ($/y) Cost ($) LE (y) ICER ($/y) Cost ($) LE (y) ICER ($/y) 16,743 16,743 16,743 16,743 13,180 19,514 16,477 16,743 16,743

4.921 4.921 4.921 4.921 4.921 4.921 4.923 4.921 4.921

16,920 16,678 16,799 17,143 13,708 19,419 16,797 16,920 17,360

4.928 4.928 4.928 4.927 4.928 4.928 4.930 4.926 4.926

25,286 (9,286)a 8,000 66,667 75,429 (13,571)a 45,722 35,400 123,318

17,381 16,982 17,182 17,675 14,064 19,962 17,342 18,028 18,228

4.930 4.930 4.930 4.928 4.930 4.930 4.930 4.930 4.930

70,889 26,556 48,778 133,143 98,222 49,778 123,589 142,816 165,037

17,708 17,308 17,503 17,931 14,496 20,207 17,514 18,026 18,226

4.928 4.928 4.928 4.927 4.928 4.928 4.930 4.926 4.926

137,857 80,714 108,571 198,000 188,000 99,000 148,234 256,674 296,673

Numbers in parentheses are negative ICER values.

CT 5 computed tomographic; sensitivity/specificity.

ICER 5 incremental cost-effectiveness ratio;

LE 5 life expectancy;

PET 5 positron emission tomography;

S/Sp 5

Ann Thorac Surg 1998;66:1876 – 85

SCOTT ET AL COST-EFFECTIVENESS OF FDG-PET FOR NSCLC

1881

Comment

Fig 2. Incremental cost-effectiveness ratio (ICER) analysis over a range of positron emission tomography (PET) Medicare reimbursement values with CT plus PET strategies (B to D) compared with the baseline strategy A. Alternative strategies with a negative ICER dominate strategy A. Horizontal line marks $50,000 per life-year saved, the threshold value below which an alternative strategy is considered to be cost-effective.

PET by 10%. For this change none of the alternative strategies are more cost-effective than strategy A. Performing a sensitivity analysis of other variables in Table 1 did not have a significant impact on the baseline results. The most critical variable in this analysis is the ratio of the cost of thoracotomy to the cost of PET. Shown in Figure 2 is a plot of the effect of varying the cost of PET versus the ICER for strategies B to D compared with strategy A. This plot shows that strategy B remains cost-effective over the entire range of PET costs analyzed and that strategy C becomes cost-effective for PET costs below $1,800. Strategies B to D all have a higher life expectancy than strategy A, and life expectancy is not affected by changing PET costs. Therefore, the negative ICER for strategies B and C indicates that those strategies have a lower expected cost than strategy A and dominate strategy A. Figure 2 shows that strategy B dominates strategy A if PET were to cost less than $1,700. Figure 3 shows a plot of the effect of the prevalence of N2 or N3 lymph node metastasis on the alternative strategy ICERs. Strategy B is cost-effective for a prevalence greater than 0.28, and strategy B dominates strategy A for a prevalence of 0.35 or greater. Strategies C and D become cost-effective if the prevalence is greater than 0.40 and 0.43, respectively. Strategy B sends 10% fewer patients to surgical intervention than strategy A; 79% of patients undergo operation in strategy A compared with 71% in strategy B. This translates to a reduction of approximately 10,000 operations by using strategy B (based on 75% of the 170,000 cases of lung cancer being NSCLC). Strategy C sends 74% of patients to operation; this reduces the number of surgical procedures by 6.4% compared with strategy A. Strategy D results in the same number of operations as strategy B. The higher cost of strategy D is due to the addition of PET scans after a positive CT scan.

The results of the current investigation support the use of thoracic PET as an adjunct to thoracic CT for preoperative staging. A strategy that requires performance of a PET study only after negative CT results was shown to be most cost-effective compared with the current CT-only management strategy. Regardless of the exact option taken, a role for thoracic PET seems highly likely. Even under a wide range of assumptions, thoracic PET identifies a significant number of patients with mediastinal or contralateral metastases in whom initial surgical therapy would not be optimal. Some of the limitations of this decision analysis imposed by our assumptions and tree structure deserve mention. These analyses assume that PET is readily available and that extra days in the hospital are not required while patients are waiting for this study. Furthermore, this analysis supports the fact that technologies such as PET, which may be more expensive, can be more cost-effective because of their improved accuracy and they are negligible to no risk. The current work assumes that biopsy has 100% accuracy. Although biopsy is very accurate, it will produce some false-negative results in some patients. Such patients would be sent for operation when in fact they are not surgical candidates. This error will increase costs for both the CT and CT plus PET strategies. Therefore the results are not expected to be sensitive to small deviations in accuracy of 100% for biopsy. Studies to explore the effects of inaccuracies of biopsy are currently underway. We also assume for the purposes of the present analysis that the benefit of surgical intervention has not been proved for most patients with N2 or N3 metastases from NSCLC and that these patients will not be referred for operation. Most clinicians would agree that surgical resection and adjuvant therapy are not optimal therapy for patients with preoperatively documented N2 disease (although patients with lymph node metastases to levels

Fig 3. Incremental cost-effectiveness ratio (ICER) analysis over a range of prevalence values for metastasis with CT plus PET strategies (B to D) compared with the baseline strategy A. Horizontal line as in Figure 2.

1882

SCOTT ET AL COST-EFFECTIVENESS OF FDG-PET FOR NSCLC

5 and 6 [aortopulmonary window] from a left upper lobe lung cancer were shown by Patterson and colleagues [23] to have up to a 45% 5-year survival rate after lobectomy and lymph node resection alone). The benefit, if any, of surgical intervention in the multimodality treatment of patients with stage III NSCLC is currently being investigated in Thoracic Intergroup Trial 1039. That study randomizes patients with biopsy-proved T1 to T3 N2 NSCLC to receive induction radiation therapy and concurrent induction chemotherapy followed by operation or to the same induction therapy with uninterrupted radiation therapy and with two additional cycles of chemotherapy. Also, some patients will be operable even though they have a distant metastasis (eg, single brain metastasis). This category of patients is not explicitly accounted for in the current analysis. These patients are very small in number and are therefore not expected to alter the results of this study significantly. Also the current analysis assumes that all patients are candidates for surgical procedure. There will be a small number of patients who will not be surgical candidates for various medical reasons. The present analysis used estimated national Medicare reimbursed costs to compare the costs of various procedures. An optimal approach would look at the true costs of each procedure and model those costs explicitly. Indirect costs such as patient income loss would also be directly modeled. Such cost analyses are beyond the scope of the current work, but many cost-effectiveness analyses have used national Medicare reimbursed costs as an attempt to standardize costs across various procedures. In actuality, it is the relative ratio of surgical costs to PET costs that control the cost-savings component. If surgical procedures can be performed at overall lower costs, then PET would also have to decrease in costs to continue to have a cost-effective role. The present work did not directly account for costs associated with bone scans, whole-body screening, or plain films. These studies will vary significantly on an individual by individual basis. This is not expected to alter the outcome of the present work because most patients would not require this workup, and both the CT and CT plus PET strategies would be affected equally. Additionally, the use of whole-body FDG-PET (which was not modeled in this work) would potentially identify more patients with advanced disease than are currently being identified by a comprehensive clinical evaluation including chest CT and other appropriate studies (bone scan or head or abdominal CT). The cost of whole-body FDG-PET would be somewhat more than thoracic PET, but it may also prove to be cost-effective in the long term. The current study has also used life expectancy as an outcome measure to model effectiveness. Future studies that use quality-adjusted life-years may prove to be a more useful outcome measure to model effectiveness. It is assumed that including quality of life will favor the alternative strategies because surgical procedures have at least a short term negative impact on quality of life. The previous analysis of NSCLC staging by Gambhir and colleagues [6] indicated that a strategy similar to

Ann Thorac Surg 1998;66:1876 – 85

strategy D in the current analysis was dominant over strategy A. Strategy D differs from the previous analysis in several aspects previously mentioned. Modeling the dependence of CT and PET resulted in a more costeffective strategy D; however, using the current Medicare reimbursement values for costs has substantially reduced the ratio of surgical costs to PET costs, with the result that strategy D now has higher expected costs than strategy A. A related analysis has shown that thoracic PET has a cost-effective role in diagnosing solitary pulmonary nodules [24]; future analysis will merge the solitary primary nodule and NSCLC analysis to develop a single cohesive understanding of overall treatment strategies for lung cancer. In conclusion, the present study quantitatively showed the cost-effectiveness of using a PET-based strategy in the management of patients with NSCLC. It was shown that several different CT plus PET strategies result in a greater life expectancy than a CT-only strategy. A strategy in which a thoracic PET is only performed after negative CT results was found to have the lowest costs of all the PET-based strategies. Furthermore, even with the uncertainty in various variables, the cost-effectiveness of the CT plus PET strategy was shown over a large range. The present study supports the wider use of PET in managing NSCLC as a significant cost-effective tool. This work was partially supported by funding from the Laubisch Foundation and the Ahmanson Foundation, with grants awarded to Dr. Gambhir.

References 1. Guhlmann A, Storck M, Kotzerke J, Moog F, SunderPlassman L, Reske SN. Lymph node staging in non-small cell lung cancer: evaluation by [18F] FDG positron emission tomography (PET). Thorax 1997;52:438– 41. 2. Bury T, Paulus P, Dowlati A, et al. Staging of the mediastinum: value of positron emission tomography imaging in non-small cell lung cancer. Eur Respir J 1996;9:2560– 4. 3. Scott WJ, Gobar LS, Terry JD, Dewan NA, Sutherland JJ. Mediastinal lymph node staging of non-small cell lung cancer: a prospective comparison of computed tomography and positron emission tomography. J Thorac Cardiovasc Surg 1996;111:642– 8. 4. Valk PE, Pounds TR, Hopkins DM, et al. Staging non-small cell lung cancer by whole-body positron emission tomographic imaging. Ann Thorac Surg 1995;60:1573– 81. 5. Scott WJ, Schwabe JL, Gupta NC, et al. Positron emission tomography of lung tumors and mediastinal lymph nodes using [18F]fluorodeoxyglucose. Ann Thorac Surg 1994;58: 698 –703. 6. Gambhir SS, Hoh CK, Phelps ME, Madar I, Maddahi J. Decision tree sensitivity analysis for cost-effectiveness of FDG-PET in the staging and management of non-small cell carcinoma. J Nucl Med 1996;37:1428–36. 7. Pauker SG, Kassirer JP. Decision analysis. In: Bailar JC III, Mosteller F, eds. Medical uses of statistics. Boston, MA: NEJM Books, 1992:159– 80. 8. Weinstein M, Fineberg H. Clinical decision analysis. Philadelphia: WB Saunders, 1980. 9. Gambhir SS, Gupta P, Shepherd JE, et al. Medical decision analysis tool: a new physician friendly decision tree tool. MD Computing 1998;15:40– 8. 10. Luke WP, Pearson FG, Todd TRJ, et al. Prospective evaluation of mediastinoscopy for assessment of carcinoma of the lung. J Thorac Cardiovasc Surg 1986;91:53– 60.

Ann Thorac Surg 1998;66:1876 – 85

11. Gross B, Glazer G, Orringer M, Spizary D, Flint A. Bronchogenic carcinoma metastatic to normal-sized lymph nodes: frequency and significance. Radiology 1988;166:71– 4. 12. McLoud T, Bourgouin P, Greenberg R. Bronchogenic carcinoma: analysis of staging in the mediastinum with CT by correlative lymph node mapping and sampling. Radiology 1992;183:319–23. 13. Dillemans B, Deneffe G, Verschakelen J, Decramer M. Value of computed tomography and mediastinoscopy in preoperative evaluation of mediastinal nodes in non-small cell lung cancer. Eur J Cardiothorac Surg 1994;8:37– 42. 14. White P, Adams H, Crane M, Butchart E. Preoperative staging of carcinoma of the bronchus: can computed tomographic scanning reliably identify stage III tumors? Thorax 1994;49:951–7. 15. Primack S, Lee K, Logan P. Bronchogenic carcinoma: utility of CT in the evaluation of patients with suspected lesions. Radiology 1994;193:795– 800. 16. Moses LE, Shapiro D. Combining independent studies of a diagnostic test into a summary ROC curve: data analytic approaches and some additional considerations. Stat Med 1993;12:1293–316. 17. Ginsberg R, Hill I, Eagan R, et al. Modern thirty-day oper-

SCOTT ET AL COST-EFFECTIVENESS OF FDG-PET FOR NSCLC

18. 19.

20.

21.

22.

23.

24.

1883

ative mortality for surgical resection in lung cancer. J Thorac Cardiovasc Surg 1983;86:654– 8. Black WC, Armstrong P, Daniel T. Cost-effectiveness of chest CT in T1N0M0 lung cancer. Radiology 1988;167:373– 8. Moroff S, Pauker S. What to do when the patient outlives the literature, or DEALE-ing with a full deck. Med Decis Making 1983;3:313–38. Cummings S, Illington G, Richard R. Managing solitary pulmonary nodules: the choice of strategy is a “close call.” Am Rev Respir Dis 1986;134:453– 60. Beck J, Pauker S, Gottlieb J. A convenient approximation of life expectancy (the “DEALE”). II. Use in medical decisionmaking. Am J Med 1982;73:883–97. Garber AM, Phelps CE. Economic foundations of costeffectiveness analysis: working papers series. Natl Bureau Econ Res 1992;4164:1–34. Patterson GA, Piazza D, Pearson FG, et al: Significance of metastatic disease in subaortic lymph node. Ann Thorac Surg 1987;43:155– 8. Gambhir SS, Shepherd JE, Shah BD, et al. Analytical decision model for the cost-effective management of solitary pulmonary nodules. J Clin Oncol 1998;16:2113–25.

DISCUSSION DR THOMAS W. RICE (Cleveland, OH): Doctor Anderson, Dr Pairolero, members, and guests. I would like to congratulate Dr Scott and his coworkers for developing this interesting model analyzing the effects of positron emission tomographic (PET) scanning on the treatment of non–small cell lung cancer. The analysis considers cost and life expectancy for various investigation and treatment strategies and uses the ratio of cost to life expectancy, which Dr Scott and colleagues call the incremental cost-effectiveness ratio, to compare the strategies. This ratio is crucial to the analysis of this work. In reviewing the manuscript, I find three important findings. First, sensitivity analysis reveals that only a small number of variables studied influence the decision analysis. Thus, errors in assumptions and estimates for these insensitive variables are a minor problem. However, errors in the few sensitive variables may be crucial to this analysis. Second, the ratio of costs and not the actual costs was important in the sensitivity analysis. Thus, controversy about true costs, reported costs, or charges is irrelevant to the inferences from this study so long as the ratio of costs are reasonable. Third, all decision strategies were equally efficacious. The extreme differences were at most 3.3 days in a nearly 5-year life expectancy. Thus, an unstated but crucial inference from this study is that survival is insensitive to the sequence of diagnostic tests studied; therefore, the cost– benefit analysis is dependent completely on the differential costs and not the efficacy. This leads to four questions and one comment. First, because life expectancy differences are tiny and these constitute the denominator of the incremental cost-effectiveness ratio, small differences in the denominator cause astronomic differences in this ratio. In the sensitivity analysis, in which the cost of surgical intervention and PET are varied by 10% in each of the strategies, this ratio varies by more than half a million from a cost saving of minus $81,000 per year to a cost of $525,000 per year. So my first question is, Because cost totally dominates the study, would not the elimination of life expectancy and a comparison of only absolute differences of cost be a more stable end point? This analysis uses the standard cost– benefit decision tree simulation method. The disturbing feature of this methodology is the use of point estimates of variables generated from a

meta-analysis of the literature. An illustration from this study is the selection of the sensitivity of PET in the assessment of mediastinal nodes in non–small cell lung cancer. In 10 studies, the sensitivity varies from 67% to 100%, with a mean of 87%. Because PET is not independent of computed tomography (CT), Dr Scott and colleagues choose the sensitivity of PET to be 76% after a negative CT scan and 88% after a positive CT scan. The analysis does not take into account the inherent variability of these data but only picks a point. My second question is, Have your statisticians considered a formal measure of the uncertainty of these point estimates and incorporated this into the analysis [1]? This study assumes that CT is always required in the treatment strategies of patients with non–small cell lung cancer. Interestingly, the cost of the CT scan is chosen at $378, very similar to the cost savings of a PET-based strategy of $398. My third question is, Have Dr Scott and associates considered the strategy of no CT and the use of PET alone? My fourth and final question is, Because the discrimination of N1 from N2 nodes by PET scan is difficult and better strategies for the evaluation of mediastinal nodes exist, the importance of PET may indeed be in its assessment of distant metastatic sites. Have Dr Scott and colleagues considered the inclusion of distant site assessment in this model? And finally my comment. Because this model depends on very small cost differences alone to draw its conclusions, and because messages from these models can sometimes be taken and suddenly be translated into national regulations and reimbursement policies, the lack of measures of uncertainty by statisticians is a serious omission that we as surgeons cannot tolerate. DR KEITH S. NAUNHEIM (St. Louis, MO): I rise to congratulate Walter Scott not only on his report, which as Tom Rice points out, may have some mathematical shortcomings, but also on the spirit of the report. Once again we have heard at the business meeting the kind of political and economic pressure the Society and our discipline is under. This is relatively—sorry, Walter— dry research. I mean I felt like I wanted a beer the entire time he was talking. But it is critically important to the Society that this kind of research be carried on. Outcomes research is one of the

1884

SCOTT ET AL COST-EFFECTIVENESS OF FDG-PET FOR NSCLC

key phrases in Congress, and these studies are the kind of things they are looking for. We need what appears to be dry and bland research. Such studies provide good statistics and predictive numbers for outcomes. This research must be undertaken if we are ever going to be competitive for health care resources. Make no mistake about it, we are vying for a limited bundle of money and competing with primary practice, preventive medicine, mammograms, Papanicolaou smears, and the like. If we can do cost-utility analysis that proves that thoracic surgery is indeed cost-effective and provides as much quality of life and number of quality-of-life-years as, say, Papanicolaou smears or mammograms, then we will continue to be a viable specialty. And it is very important for our patients that they continue to have access to our care. DR SCOTT: I would like to thank Dr Rice and Dr Naunheim for their thoughtful comments and analysis. With regard to Dr Rice’s first question regarding life expectancy, we felt it was important to show that incorporating PET into an algorithm did not result in any decrease in life expectancy. My colleagues and I were uncomfortable with excluding someone from operation on the basis of a positive PET scan, or presuming that on the basis of a negative PET scan that operation could proceed, and that is the reason that we left life expectancy in. As far as the sensitivity of PET and the values that we chose for our analysis, we are reanalyzing the data with different sensitivities and actually different ranges of sensitivities. That is one of the important strengths of this method of quantitative analysis. With regard to using PET without CT, I think CT is important for the anatomic information it provides. Nuclear medicine scans provide metabolic images with little anatomic detail.

Ann Thorac Surg 1998;66:1876 – 85

There has been a significant amount of interest in using PET to identify sites of distant metastases. At this time, adequate data are available for evaluating the role of PET in the assessment of mediastinal lymph nodes. There are much fewer data available regarding the sensitivity and specificity of PET for the detection of liver or bone metastases. Therefore, I cannot comment on the role of PET in that setting. Funding for a clinical trial that would compare PET with clinical evaluation, including abdominal and head CT and bone scans for the preoperative detection of distant metastases in patients with otherwise operable non–small cell lung cancer, has been requested from the National Cancer Institute by the American College of Surgeons Oncology Group. In response to Dr Rice’s comment about a lack of statistical analysis in our model, I would say that in decision tree analysis sensitivity analysis is the traditional method of handling the uncertainty of estimated values for the variables in the model. Our approach here, which is the standard approach, is to first find the range of reported values through a literature search. We then use the weighted mean, or pooled results for the sensitivity and specificity of diagnostics, as the baseline values in the model. Finally we perform sensitivity analysis for each variable over the entire range of values reported in the literature. This is analogous to taking the minimum and maximum reported values for a given variable as the confidence interval for that variable. Finally, a statistical study of variance of the results is definable but requires an intractably large calculation that is not practical to solve, even using modern computers. This calculation also would require multiple unsupported assumptions about each variable being estimated, thus reducing the confidence in the resulting estimate of the distribution for the expected values.

INVITED COMMENTARY The use of decision analysis is increasing in the surgical literature [1]. Because this method of research may be unfamiliar to clinicians, it is worthwhile to review the technique of decision analysis before commenting on this study. Decision analysis is useful when it is desirable to quantitatively compare alternative treatment strategies. As pointed out by Berkmeyer and Welch in their article, “A Reader’s Guide to Surgical Decision Analysis” [2], there are four basic components to a decision analysis. The first is the decision model, which specifies the alternative strategies and the outcomes associated with each strategy. Second, probabilities for each of these outcomes are estimated from the best available literature and assigned to each decision point in the model. Third, a value such as cost, patient preference, life expectancy, or quality adjusted life expectancy is assigned to each possible outcome. These values are labeled utilities. Fourth, an analysis is performed to calculate the expected value of each strategy. This is accomplished by multiplying the outcome utilities by the probabilities of each occurrence. By calculating the expected value, a favored strategy is identified. A well-performed decision analysis includes a sensitivity analysis, which varies each of the baseline probabilities over a plausible range. In this issue of The Annals of Thoracic Surgery, Scott et al performed a decision analysis of the cost-effectiveness of © 1998 by The Society of Thoracic Surgeons Published by Elsevier Science Inc

FDG-PET for staging non–small cell lung cancer. The model asks whether the addition of the FDG-PET scan to the current preoperative evaluation of the mediastinum in patients with lung cancer is cost-effective. The authors identify all relevent treatment strategies in Figure 1. Do the strategies make sense to the surgeon who may wish to order this extra test? The answer is clearly yes. The researchers use all the possible combinations of CT with or without PET scan to delineate benign from malignant lymphadenopathy before surgery. It is also apparent that Scott and coworkers included the important outcomes possible for each strategy. Next we turn to the probabilities for each of the outcomes. The authors of this study painstakingly reviewed the best available literature on the subject, and the baseline probabilities for each of these outcomes are shown in Table 1. The same table shows the range over which those baseline estimates were analyzed in the model. They seem reasonable, with the possible exception of the specificity of CT scan, which may be higher in patients with T1N0M0 lesions [3]. A higher specificity for CT could make PET scans less cost-efficacious. Satisfied with both the decision model and the baseline probabilities, we turn to the results. In this study, the favored strategy is one that uses PET only after a negative CT of the mediastinum, as compared to a strategy that uses CT scan alone. If the PET scan is 0003-4975/98/$19.00 PII S0003-4975(98)01055-8