Evaluation of an objective plan-evaluation model in the three dimensional treatment of nonsmall cell lung cancer

1n1. J. Radiation

Oncology

Biol.

Phys., Vol. 34. No. 2. pp. 469-474. 1996 Copyright 0 1996 Elscvier Science Inc. Printed in the USA. All rights reserved

03hO-30

I h/96 $ I s.00

+ .wJ

ELSEVIER

0360-3016(95)02115-9

0 Technical Innovations and Notes EVALUATION OF AN OBJECTIVE DIMENSIONAL TREATMENT MARY

V. GRAHAM,

M.D.,* NILESH ROBERT E. DRZYMALA,

PLAN-EVALUATION MODEL IN THE THREE OF NONSMALL CELL LUNG CANCER L. JAIN, M.S.,+* MICHAEL G. KAHN, M.D., PH.D.* AND JAMES A. FWRDY, PH.D.*

PH.D.,+$

*Radiation Oncology Center, Mallinckrodt Institute of Radiology and Section of Medical Informatics, Department of Internal Medicine, Washington University School of Medicine, St. Louis, MO, *Department of Computer Science, Washington University, St. Louis, MO Purpose: Evaluation of three dimensional (3D) radiotherapy plans is diflicult because it requires the review of vast amounts of data. Selecting the optimal plan from a set of competing plans involves making tradeoffs among the doses delivered to the target volumes and normal tissues. The purpose of thii study was to test an objective plan-evaluation model and evaluate its clinical usefulness in 3D treatment planning for nonsmall cell lung cancer. Methods and Materials: Twenty patients with inoperable nonsmall cell lung cancer treated with definitive radiotherapy were studied using full 3D techniques for treatment design and implementation. For each patient, the evaluator (the treating radiation oncologist) initially ranked three plans using room-view dosesurface displays and dose-volume histograms, and identified the issues that needed to be improved. The three plans were then ranked by the objective plan-evaluation model. Afisure of merit (FOM) was computed for each plan by combining the numerical score (utility in decision-theoretic terms) for each clinical issue. The r&i&y was computed from a probability of occurrence of the issue and a physician-specific weight indicating its clinical relevance. The FOM was used to rank the competing plans for a patient, and the r&f&y was used to identify issues that needed to be improved. These were compared with the initial evaluations of the physician and discrepancies were analyzed. The issues identified in the best treatment plan were then used to attempt further manual optimization of this plan. Results: For the 20 patients (60 plans) in the study, the final plan ranking produced by the plan-evaluation model bad an initial 73% agreement with the ranking provided by the evaluator. After discrepant cases were reviewed by the physician, the model was usually judged more objective or “correct.” In most cases the model was also able to correctly identify the issues that needed improvement in each plan. Subsequent replanning conGrmed that further manual plan optimization could be achieved in 17 patients. Conclusion: The objective plan-evaluation model was able to rank lung cancer radiotherapy plans from best to worst. It was useful in improving plans and may be useful to physicians in defining goals for patients based on the ability to effectively and safely treat their tumors. Lung cancer, Three dimensional radiation therapy, Treatment plan-evaluation,

Decision theory.

number of beams, and this has greatly increased the number of plausible treatment plans. It is impossible to generate or consider all the plausible plans in the course of manual treatment plan optimization. Therefore, numerous investigators have recently focused on the automatic optimization of 3D RT plans (12- 15,20,23). Score functions are desirable to compare alternate treatment plans. Constraint-based optimization systems may have two or more plans that meet the constraints. To select the best one, a score function is needed ( 12, 15). Various score functions

INTRODUCTION

With the rapid improvement in computer technology in terms of computation speed and high-resolution graphics, three dimensional (3D) radiation treatment planning (RTP) may become the standard of practice in radiation therapy (11, 18). Three-dimensional RTP allows the unrestricted use of multiple noncoplanar beams to conform the high prescription dose volume to the shape of the tumor volume. Such conformal plans usually need a large

Reprint requests to: Mary V. Graham, M.D., Radiation Oncology Center, Washington University School of Medicine, 4939 Children’s Place, Ste. 5500, St. Louis, MO 63110. Acknowledgements-This work is supported in part by an American Cancer Society Clinical Oncology Career Develop-

ment Award, National Library of Medicine Grant S-R29LMO5387, National Cancer Institute Contract NOl-CM-97564, and a U.S. Biosciences Industrial Grant. Accepted for publication 22 August 1995. 469

470

I. 3. Radiation Oncology 0 Biology 0 Physics

have been used including biological dose-response models and physical dose-volume limits, but few have developed a single figure of merit (FOM) because of the need for systematic weighting of the different scores for the target, nontarget tissue, and specific dose limiting organs. In addition, few of the published reports describe the clinical validation of their score functions (19, 22). Our research focuses on the use of decision theory to develop an objective plan-evaluation model to serve as a score function for the manual and automatic optimization of 3D RT plans (8,9). The purpose of this study is to clinically validate the recommendations of our objective plan-evaluation model, and to demonstrate its usefulness as a tool in the manual optimization of 3D RT plans. METHODS

AND MATERIALS

Twenty patients with inoperable nonsmall cell lung cancer who were treated with definitive radiation therapy were selected for the study. These patients were randomly chosen from patients treated in the last 2 years using full 3D techniques for treatment design and implementation at the Radiation Oncology Center of Mallinckrodt Institute of Radiology. Table 1 shows the clinical characteristics of the tumor for each patient. Two target volumes were outlined for each patient-Planning Target Volume Two (PTV2), which was the gross tumor plus a margin for treatment setup reproducibility and movement; and Planning Target Volume One (PTVl), which included PTV2 and electively irradiated nodal areas plus margin. Nine normal tissues were outlined for each patient-ipsilateral lung, contralateral lung, spinal cord, esophagus, heart, ipsilateral brachial plexus, contralateral brachial plexus, liver, and connective tissue (which included all soft tissues in the treatment field not otherwise contoured or labeled as another organ). Table 1. Tumor characteristics Tumor location Right upperlobe Left upperlobe (including lingula) fight lower lobe Left lower lobe

7 5 2 6

T and N stage Tl T2 T3 T4 NO Nl N2 N3 Overall stage 1* lI* IIIA IIIB * Medically inoperable.

3 9 3 5 5 1 12 2 3 1 11 5

Volume 34, Number 2, 1996

For each patient, three plans were chosen including the plan that was used to treat the patient. Heterogeneity corrections were applied to the dose calculations for all the plans used in the study. For each patient, the evaluator (the treating radiation oncologist) was asked to evaluate these three treatment plans in three steps. Subjective evaluation

The first step was the subjective evaluation. The evaluator examined the beam arrangements and target coverage using Beam’s Eye View (BEV) and Room View displays available on our 3D treatment planning system (16), and the dose-volume histograms (DVHs) using the Graphical Plan Evaluation Tool developed as part of National Cancer Institute’s Radiotherapy Treatment Planning Tools Contract (2). The DVHs were computed using a sampling resolution that is user defined, resulting in resampling the rectilinear dose matrix, which can be set to any resolution between 1 to 10 rmn. We currently use 1 mm sampling resolution for generating DVHs in our clinical practice. Based on these parameters, the evaluator ranked the three plans from best to worst and identified up to five issues in each plan whose dose distributions needed to be changed to improve that plan. The FOM tool identified issues independent of the physician evaluator and did so as the three attributes with the lower utilities. There was no threshold number by which issues were identified. Objective evaluation

In this step, the three plans were evaluated using Jain’s decision-theoretic objective plan-evaluation model (8, 9). In this multiattribute decision model, attributes represent clinical issues such as noneradication of the tumor and radiation-induced damage to normal tissues in the treatment field, For each attribute, a utility from 0 (bad) to 1 (good) was computed. The utility measures how well the associated tissue is performing in the treatment plan, and is a combination of the likelihood of occurrence of the attribute and its clinical relevance. For attribute i: uti&k = 1 - probability, x weight,

(Eq. 1)

In Eq. 1, probability, is the likelihood of occurrence of attribute i, and weight, is its clinical relevance. The probabilities for tumor noneradication were computed using the Tumor Control Probability (TCP) model by Goitein (4). Table 2 contains the data used for the TCP model. The probabilities for normal tissue damage were computed using the Normal Tissue Complication Probability (NTCP) model developed during National Cancer Institute’s Photon Treatment Planning Contract (1). The radiobiological effectiveness data needed for the model were derived from the Photon Contract data, and a dose per fraction of 2 Gy was used. It is known that the theoretical basis for the determination of NTCP information still remains to be clinically validated. Two institutions have

Evaluation of an objective plan-evaluation model Table 2. Data used for calculating the TCP

Type of data

PTv2

PTVl

Probability that the volume is not malignant

10%

90%

Prescription dose TCP at prescription dose Gamma slope

70 Gy 60% 3

50 Gy 90% 3

TCP = Tumor control probability.

PTV2 = Planning target volume two. PTVl = Planning target volume one.

0

M. V.

GRAHAM

et al.

471

ments were planned to decrease the dose to the heart. Sometimes it was difficult to find new beam arrangements without causing increased dose to organs that previously had not been designated as having low utility. In this case, the original standard beam arrangements would be used and dose escalation would be performed up to the point where normal organs began to have dropping utilities. A replan was considered successful if the newly designed treatment plan was better in the opinion of the evaluator and had a higher FOM,,. RESULTS

tested the NTCP model with clinical results in the area of pneumonitis (6, 21). The purpose of this article was not, however, to test the NTCP model. For the lungs, the computed NTCP values seemed inappropriately high. To adjust for the inaccuracies of the NTCP calculations and still test the validity of the plan-evaluation model, the DVHs of the lungs were rank-ordered based on the lowest to highest doses delivered to the lungs as evaluated on the DVHs by the “area under the curve.” Thus, it was not a single maximum dose or threshold dose, but rather a more global figure incorporating both these statistics. The rank ordering process was blinded and in no way influenced by the pneumonitis complications. Estimated probabilities of complications were then made based on the rank order of the organ doses and used as the NTCP in the model tested. The weights were elicited using the Level of Enthusiasm methodology (9). All weight were kept constant for all plan evaluations. The overall score of the plan, i.e., its $gure of merit (FOM,,), is obtained by multiplying the utilities of all the attributes. The subscript Pr denotes that the FOM is based on probability values. Therefore, our score function is: FOM,,.

n I

(1 - probabilityi

X weight,)

(Eq. 2)

The plans were ranked based on their FOM,, values and compared to the physician’s subjective evaluation. The attributes with the lowest utility values were chosen as the issues to compare with those identified by the physician as needing improvement. Replanning (manual optimization) The final step was to determine whether our objective plan-evaluation model could be used as an aid in the optimization of radiation treatment plans. This was called the replan step, and its purpose was to improve the dose distribution in the attributes having the lowest utilities in the plan identified as the best by the FOM values. In this replan step new beam arrangements were explored to reduce the dose to the organs that had low utilities. For example, if a heart utility was very low and causing the overall score to be low, the new beam arrange-

Plan ranking The initial overall agreement in the ranking of radiotherapy plans between the evaluator and the FOM tool was 44 out of 60 (73%). There were circumstances of disagreement in 8 of the 20 patients, and these were further analyzed. In two of the eight patients, the differences between the three plans were so small that it was difficult for the physician to distinguish one plan as being superior to another. This was also reflected in the FOM,,. values, which were either identical or differed by < 0.01. In one of the eight patients, the physician was very conservative in one issue of normal tissue dose (the risk of brachialplexopathy). The physician’s conservatism was not reflected by the tool’s utility for that issue. On reflection, the physician stated that she would have discussed the higher possibility of a complication with the patient and jointly decided whether they wanted to risk this in the hope of achieving tumor control. The FOM tool identified the probability of brachialplexopathy as 10-15s. In the majority of the disagreements between the physician and the FOM tool (the remaining five out of eight), it appeared that the physician was not as objective as the tool. For example, the review of the DVHs of the plans that were under dispute clearly showed higher volumes of normal tissues being irradiated or higher total doses to those normal tissues, which should have resulted in the physician ranking one plan as being worse than the other. After reviewing these five cases, the physician decided that the FOM ranking was probably correct. The final FOM,, for the 20 best plans (one for each patient) ranged from 0.14 to 0.81, with a rather normal distribution around the middle range scores of 0.40 to 0.60 (Fig. 1). There were four patients who had very low FOM,, values (less than 0.30). Identi$cation of issues needing improvement In the 60 plans, the issues most frequently identified by either the physician or the tool as problems were: ipsilateral lung (57 times), heart (37 times), esophagus (29 times), PTV2 (21 times), contralateral lung (18 times), and spinal cord (5 times) (Table 3). In the first four of these issues, there was a good agreement with the physician and the tool that either the target was not being adequately treated by the prescription dose or that there

I. J. Radiation

412

0 Biology l Physics

Oncology

Distribution

of

Volume

34, Number

2, 1996

conservatism in the physician’s viewpoint, which, based on clinical experience, was not accurately reflected by NTCP predictions. Replan step

0.00-0.10-0.2~-0.30-0.40-0.50-0.60-0.70-0.80-0.900.09 0.19 0.29 0.39 0.49 0.59 0.69 0.79 0.89 1.00 FOM,, Scores

Fig. 1. Cumulative patients.

histograms

of the final FOM,, for all 20

was a high probability of a normal tissue complication. This agreement ranged from 69 to 95% (Table 3, column 2). When the physician identified these four issues as a potential source of concern, but the FOM tool did not (Table 3, column 3), the NTCPs were low (5 5%). In the circumstances where the physician did not identify these tissues as a potential source of complication but the FOM tool did (Table 3, column 4), it appeared that the FOM tool identified real potential issues that the physician missed based on DVH evaluation. In these cases, the NTCP calculations were 2 15%. In the contralateral lung and spinal cord, there was not a good agreement between the physician and the FOM tool. The physician was much more apt than the FOM tool to identify these two organs as a potential issue (Table 3, column 3). This reflects a Table 3. Issues needing improvement identified Physician and FOM tool

Ipsilateral lung (57) Heart (37) Esophagus

(2%

P-w2 (21) Contralateral lune (18) Spinaicord (5)

by

Correctly agreed with physician

Physician+ FOM-

49(86%) 28 (76%)

1 (2%) 7(19%)

7 (12%) 2(5%)

20 (69%) 20(95%)

6(21%) 1 (5%)

3 (10%) 0

10 (56%) 1 (20%)

7 (39%) 4(80%)

1 (5%) 0

An improved replan was generated for 17 of the 20 patients (85%). Of these 17 patients, 11 (65%) had considerable improvement in their plan with an increase in FOM,, of at least 0.05 (Fig. 2). The evaluator felt that every plan with an improved FOM was preferred. Most of the plans could be improved in some manner, regardless of whether their initial FOM,, was high or low. Despite this, with a manual attempt at optimization, the patients with the lowest scores still maintained those lower scores, while the patients with the higher scores improved further. In the patients with the lower scores, the tumors were quite large or were in intimate proximity to vital organs. In 6 of 17 patients the FOM,, improved because of ability to increase the amount of dose to the PTV2 (while still maintaining acceptable low doses to the normal organs). In 5 of 17 patients the FOM,, improved because of decreased dose to normal organs. In 6 of 17 patients the improved FOM,, was the result of both increased dose delivery to the PTV2 and decreased dose to the normal organs. DISCUSSION Traditional methods for evaluating radiotherapy plans have been subjective and dependent on the clinician’s experience and intuition. The FOM tool has been developed as a quantitative plan-evaluation tool. This tool can be used to compare NTCPs, TCPs, or other models to predict tumor control and normal tissue complications. One of the strengths of the FOM tool is that it allows clinicians to modify the overall scores based on clinical experience and clinical preferences. Nonetheless, it maintains an objective score function that may be useful to

PhysicianFOM+

Histogram of Original FOM,, Score and Additional Score Achieved with Replan Step q

Additional

Score after Replan

0.9 0.8 0.7 score

;:; 0.4 0.3 0.2

Physician+: Physician identified this issue as a problem. FOM+: FOM tool identified this issue as a problem. Physician-: Physician did not identify this issue as a problem. FOM-: FOM tool did not identify this issue as a problem. PTV2 = Planning target volume two (gross disease plus margin for uncertainty). FOM = Figure of merit.

0.1 0 1

3

5

7

9 Patient

11

13

15

17

19

Number

Fig. 2. Cumulative histogram of replan FOM,, after attempt to improve the final FOM,, with amount of improvement depicted in hatched bars. No improved plan for patients 10, 18, and 20.

Evaluation of an objective plan-evaluation model

the physician for consistently evaluating numerous plans for one patient and/or various plans for a number of patients. This methodology could also be used as a score function for optimization of treatment plans. An additional strength of the tool was found in the replanning phase where the tool identified new potential issues when plans were changed. For example, if a plan was originally deemed poor because of an inability to deliver adequate dose to the planning target, the tool correctly identified new issues such as lung or heart receiving excessive doses as target dose was increased. Complications of various surrounding organs emerged as potential issues as the dose to PTV2 was increased. Therefore, the tool appeared flexible in its identification of appropriate issues as the circumstances of a plan changed. Our current evaluation of the model uses existing NTCP and TCP models (1, 4); however, it is well recognized that these biological indices have major limitations and need to be further correlated with clinical data (5, 15, 17). A strong word of caution is required. We do not endorse any existing model, as we ourselves are not convinced of the clinical validity of the present generation of models. However, we believe that plans cannot be satisfactorily evaluated in a quantitative manner by looking at DVHs alone. Thus, we are reporting on a developing methodology that can be used for objective radiation treatment plan evaluation. We believe that better TCP and NTCP models will become available in the future and that they will easily be incorporated into our model. In the meantime, if the TCP and NTCP values seem unreasonable, our tool allows the evaluator to override the calculated probabilities and insert her own subjective probabilities based on her clinical experience. Despite some initial discrepancies between the physician identification of issues and the FOM tool identification of issues (Table 3), upon careful reanalysis by the evaluator, it was felt that the tool had actually been more objective. This suggests that DVHs are subject to multiple interpretations and that an individual may have difficulty interpreting the large amount of data objectively. For example, when a physician is given one set of DVHs to evaluate it is much simpler and less likely that information will be missed as opposed to when 10 or 12 DVHs were presented. The magnitude of the data made it much more likely that information will be missed and not included in the assessment of the plan. By having an objective score applied to each issue the model prevents this situation from happening, and this is the main reason the tool was more objective than the physician. It is desirable for a score function to incorporate the clinician’s experience and preferences. Therefore, the weight parameter of our model is important. Different clinicians will disagree on the importance of various issues; the tool accommodates for this. The weight parameters are determined based on the clinician’s individual preferences in evaluating plans and applied in a uniform fashion. The weight number remained constant for this

0

M. V.

GRAHAM

er al.

473

work. For example, based on the Photon Contract work of normal tissue partial volume irradiation threshold doses, the weight assigned to the complication in the ipsilateral lung in this article was 0.5 (3). However, weight assignment needs further research and is currently being studied by our group in efforts to make it valid and an accurate reflection of physician preferences (7). The model uses a multiplicative combining function for computing the FOM, where a lower utility diminished the value of the FOM. For example, if the NTCP was reduced from 25 to 20%, the utility of the corresponding tissue increased from 0.75 to 0.80, whereas a change in NTCP from 7 to 2% resulted in a utility change from 0.93 to 0.98 (assuming a weight of 1). Thus, the same magnitude of change in probability led to different changes in utility-depending on the value of the probability. If the plan evaluator believed that two outcomes are equally adverse, then he/she should be willing to undertake a large increase in the chance of the less common event to compensate for a small reduction in the chance of a more common event. It is not clear whether this truly describes physician behavior. Physicians do not always accept large increases in complication probabilities for small reductions in tumor recurrence, as evidenced by the fact that complications are a far less cormnon event and tumor recurrence a much more common and apparently accepted event. The results of our own studies point to the fact that the physician generally preferred plans with low complication probabilities even when they were perhaps not as effective in terms of tumor control. As mentioned, this was the case in the original choice of avoiding brachialplexopathy. Generally, physicians are loath to induce complications that are historically rare, whereas more common events, i.e., pneumonitis and/or tumor recurrence are more readily accepted. In the future with more objective score functions we may find physician preferences and practices undergo a change. If clinical occurrences can be adequately modeled then the risk of tumor recurrence or normal tissue complications will be more objectively delineated and clinical decisions, in fact, may change. In general, cancer to many patients does not carry the overall death sentence that it may have in the past, and more patients expect to have tumor control. If they are given reasonable estimates of their risk of tumor control vs. their risk of complications, they, with their physicians, will be able to make better decisions. The spinal cord utility also correlated poorly with the physician’s judgments. For this model the weight assigned was 1 (highest concern). Thus, the NTCPs are probably inaccurate because they are based on a relative lack of data. It will be difficult to improve this or correlate it with human outcomes because of the catastrophic nature of the complication. At the time the model was tested the lung volumes were separated by ipsilateral and contralateral lung volumes. We now have evidence to suggest that NTCP prob-

414

I. J. Radiation Oncology0 Biology 0 Physics

abilities correlate better with total lung volume (21). This is probably why the contralateral lung utility correlates so poorly with the physician’s evaluation.

CONCLUSIONS This article evaluates the use of the FOM tool for the clinical use of plan evaluation in the 3D treatment of nonsmall cell lung cancer. The strengths of the tool are that it is a quantitative plan evaluation tool and provides an objective plan evaluation. The FOM,,. agreed well with the physician’s ranking and identification of issues for the radiotherapy plans. The tool may be a suitable score function for automated plan optimization algorithms.

Volume 34, Number 2, 1996 Such algorithms may guide clinicians to better their treatment plans we well as more clearly define goals of therapy for patients. However, decision-point threshold scores require further clinical correlations to define the appropriate levels for decision making. Additional studies as to how a physician arrives at weight values for various issues and whether these are variable for different patients in currently being studied. Further correlations with clinical outcomes and objective score functions are needed, and this article’s presentations should be viewed as preliminary. An alternate FOMDS (Figure of Merit Dose Statistics) model based on dose statistics (minimum, maximum, and mean doses) is currently being developed and evaluated (10). Further correlation with clinical outcomes of lung cancer and other tumor sites is ongoing.

REFERENCES 1. Butman, C.; Kutcher, G. J.; Emami, B.; Goitein, M. Fitting of normal tissue tolerance data to an analytic function. Int. J. Radiat. Oncol. Biol. Phys. 21:123-135; 1991. 2. Drzymala, R. E.; Holman, M. D.; Yan, D.; Harms, W. B.; Jain, N. L.; Kahn, M. G.; Emami, B.; Purdy, J. A. Integrated software tools for the evaluation of radiotherapy treatment plans. Int. J. Radiat. Oncol. Biol. Phys. (accepted). 3. Emami, B.; Lyman, J.; Brown, A.; Coia, L.; Goitein, M.; Munzenrider, J. E.; Shank, B.; Solin, L. J.; Wesson, M. Tolerance of normal tissue to therapeutic irradiation. Int. J. Radiat. Oncol. Biol. Phys. 21:109-122; 1991. 4. Goitein, M. The probability of controlling an inhomogeneously irradiated tumor. In: Evaluation of treatment planning for particle beam radiotherapy. Radiotherapy Development Branch; Radiation Research Program; Division of Cancer Treatment, National Cancer Institute, Bethesda; September, I987. 5. Goiten, M. The comparison of treatment plans. Semin. Radiat. Oncol. 2:246-256; 1992. 6. Graham, M. V.; DrzymaIa, R. E.; Jain, N. L.; Purdy, J. A. Confirmation of dose-volume histograms and normal tissue complication probability calculations to predict pulmonary complications after radiotherapy for lung cancer. Int. J. Radiat. Oncol. Biol. Phys. 3O(Suppl 1):198; 1994. 7. Jain, N. L.; Kahn, M. G. A methodology for reconciliation of inconsistencies in physicians’ preferences through clinical use of decision-support systems. Med. Decis. Making 12:349; 1992. 8. Jain, N. L.; Kahn, M. G. Ranking radiotherapy treatment plans using decision-analytic and heuristic techniques. Comp. Biomed. Res. 25374-383; 1992. 9. Jain, N. L.; Kahn, M. G.; Drzymala, R. E.; Emami, B.; Purdy, J. A. Objective evaluation of 3-D radiation treatment plans: A decision-analytic tool incorporating treatment preferences of radiation oncologists. Int. J. Radiat. Oncol. Biol. Phys. 26:321-333; 1993. 10. Jain, N. L.; Kahn, M. G.; Graham, M. V.; Purdy, J. A. 3D conformal radiation therapy. V. Decision-theoretic evaluation of radiation treatment plans. XIth International Conference on the Use of Computers in Radiation Therapy. Manchester, UK; 1994:8-9. 11. Kutcher, G. J.; Leibel, S. A.; Mohan, R.; Harrison, L. B.; Armstrong, J. G.; Zelefsky, M. F.; LoSasso, T. J.; Burman, C. M.; Mageras, G. S.; Chui, C.-S.; Brewster, L. J.; Masterson, M. E.; Ling, C. C.; Fuks, Z. Advances in precision

12. 13. 14.

15.

16.

17. 18. 19.

20. 21.

22.

23.

treatment: Some aspects of 3D conformal radiation therapy. Front. Radiat. Ther. Oncol. 27:209-226; 1993. Langer, M.; Brown, R.; Kijewski, P.; Ha, C. The reliability of dose optimization under dose-volume limits. Int. J. Radiat. Oncol. Biol. Phys. 26:529-538; 1993. Mageras, G. S.; Mohan, R. Application of fast simulated annealing to optimization of conformal radiation treatments. Med. Phys. 20:639-647; 1993. Mohan, R.; Mageras, G. S.; Baldwin, B.; Brewster, L. J.; Kutcher, G. J.; Leibel, S.; Burman, C. M.; Ling, C. C.; Fuks, Z. Clinically relevant optimization of 3-D conformal treatments. Med. Phys. 19:933-944; 1992. Niemierko, A.; Urie, M.; Goitein, M. Optimization of 3D radiation therapy with both physical and biological end points and constraints. Int. J. Radiat. Oncol. Biol. Phys. 23:99- 108; 1992. Purdy, J. A.; Harms, W. B.; Matthews, J. M.; Drzymala, R.; Emami, B.; Simpson, J. R.; Manolis, J.; Rosenberger, F. U. Advances in 3-dimensional radiation treatment planning systems: Room-view display with real time interactivity. Int. J. Radiat. Oncol. Biol. Phys. 27:933-944; 1993. Rosen, I. I.; Morrill, S. M.; Lane, R. G. Optimized dynamic rotation with wedges. Med. Phys. 19:971-977; 1992. Rosenman, J.; Chaney, E. L.; Sailer, S.; Sherouse, G. W.; Tepper, J. E. Recent advances in radiotherapy treatment planning. Cancer Invest. 9:465-481; 1991. Shalev, S.; Viggars, D.; Carey, M.; Hahn, P. The objective evaluation of alternative treatment plans: II. Score functions. Int. J. Radiat. Oncol. Biol. Phys. 20:1067-1073; 1991. Starkschall, G.; Eifel, P. J. An interactive beam-weight optimization tool for three-dimensional radiotherapy treatment planning. Med. Phys. 19: 155- 163; 1992. Ten Haken, R. K.; Mattel, M. K.; Hazuka, M. B.; Kessler, M. L.; Turrisi, A. T. NTCP lung model parameter adjustment based on clinical 3-D dose-volume data. Int. J. Radiat. Oncol. Biol. Phys. 27(Suppl 1):239; 1993. Viggars, D. A.; Shalev, S.; Stewart, M.; Hahn, P. The objective evaluation of alternative treatment plans III: The quantitative analysis of dose volume histograms. Int. J. Radiat. Oncol. Biol. Phys. 22:419-427; 1992. Webb, S. Optimization by simulated annealing of threedimensional, conformal treatment planning for radiation fields defined by a multileaf collimator: II. Inclusion of two-dimensional modulation of the x-ray intensity. Phys. Med. Biol. 37:1689-1704; 1992.