- Email: [email protected]
Computer-assisted decision making in portal verification-optimization of a neural network approach
1. J. Radiation Oncology • Biology • Physics
156
Volume 42, Number 1 Supplement, 1998
63 COMPUTER-ASSISTED DECISION MAKING IN PORTAL VERIFICATION - OPTIMIZATION OF A NEURAL NETWORK APPROACH Konrad Leszczynski, Randall Bissett, Daniel Provost, Scott Cosby and Susan Boyko Northeastern Ontario Regional Cancer Centre, 41 Ramsey Lake Road, Sudbury, Ontario, Canada, P3E 5J1 Purpose / Obiective: On-line portal verification of radiation therapy is becoming increasingly important with the introduction of eonformal and threc-dimensional irradiation techniques, which rely on high geometric accuracy in treatment delivery. With the large amount of portal image data and stringent time constraints, the conventional scheme by which the portal verification decision as to the set-up acceptability is made (by a qualified radiation oncologist) is no longer sustainable, On the other hand, our previous studies indicated that delegation of the portal decision making task to other (non-oncologist) professionals would significantly affect the result. Therefore, the objective of this study was to develop, optimize and evaluate on clinical data, an artificial intelligence decision making tool, based on the artificial neural network (ANN) approach, that would approximate, as close as possible, portal verification assessments made by a radiation oncologist expert. Materials & Methods: 328 electronic portal images of tangential breast irradiations were included in the study. The portal images were registered (aligned) with the corresponding simulation radiographs using anatomical features, so that the prescribed and treated radiation field boundaries could be brought into a common reference fi'ame and displayed on a background of the simulation image. These synthetic representations of field placement during treatment were evaluated by a radiation oncologist expert, who rated the treatment set-up acceptability on a scale from 0 to 10. Scores below 5 indicated various degrees of unacceptable treatment set-up. Translational and rotational errors in the placement of different radiation field boundary segments (anterior, posterior, superior and inferior field borders) were quantified using a previously developed automated algorithm (segmentedchamfermatching). The values of the field placement errors formed seven-dimensional feature vectors, which represented each of the 328 treatments. The feature vectors were used as inputs to a three-layer, feed-forward ANN, the output of which was meant to reproduce the expert's ratings. The neural network was trained on the oncologist's ratings using a back-propagation algorithm. The evaluation of the performance of the ANN was carried out by repartitioning the entire data set into training and testing subsets, and by comparing the ANN's ratings with those of the expert. Results: A high degree of agreement between the oncologist and the ANN ratings was observed. The finear correlation coefficient between them was 0.76. The average discrepancy in ratings was -0.07 (the expert's scores were slightly lower, on average) and the standard deviation of the discrepancies was 1.02. Comparisons were also carried out from the portal decision making point of view. Using a decision threshold equal to 5 for both sets of ratings, the ANN classifier was capable of detecting 56% of the portals classified as unacceptable by the oncologist (true positive fraction, TPF), and only 2% of the portals acceptable to the oneologist were mis-classified as "unacceptable" by the ANN (false positive fraction, FPF). Setting the decision threshold to 6 raised the TPF to 80% and the FPF to 12%. By optimizing the ANN training process for accuracy in ratings of "unacceptable" portals, it was possible to raise the TPF to 100% with only a moderate increase of FPF, to 6.4%. The effect of the decision threshold in the optimized ANN was also examined using the Receiver Operating Characteristic (ROC) methodology. Conclusion: An automated portal image classifier based on the Artificial Neural Network approach exhibited excellent agreement with the radiation onculogist expert. After optimization, it was capable of detecting all portals flagged as "unacceptable" by the oncologist. At the same time, the rate of false alarms stayed at an acceptably low level. These results indicate the feasibility of using the ANN portal image classifier as an automated assistant to the radiation therapist, recommending an appropriate decision as to the acceptability or otherwise of a given treatment set-up depicted in a portal image - thereby relieving the radiation onculogist of the burden of managing the large number of on-line portal images.
64 ENHANCEMENT OF EI.E('IFR()NIC PORTAL iMAGES BY INFORMATION MATCHING WITh PRE-TRICATMENT DATA AND B't' AN ASSOCIATIVENEURAL INETWORK G, Krell, B. Michaclis, O. Gademann Otto-von-Guerickc-University Magdeburg. Germany Purpose/Objective: The electronic portal or megaw)ltage imaging technique is an important tool lor the clinician to verify the position of the patient during treatment in radiotherapy. The electronic portal images (EPI) are produced by projecting the body interior within the field of the treatment beam onto a screen and capturing this image by a camera. Due Io the high energy of the treatment beam the unprocessed EPI is poor in quality. The eft~:ct of conventional enhancement techniques is limited. Therelbre, additional infi~rmation from images captured in pro-treatment is used as a-priori knowledge to reach a considerably higher standard of EPI analysis. Materials and Methods: The main idea of the approach consists in the fusion of the dynamic inlormation in the EPI with a-priori knowledge obtained by the simulator image (SI). The simulator image is always taken for target localization during treatment planning and is usually a film image but also other data sources could be used such as CT. Fig. t illustrates the two steps of the algorithm. Firstly, a specially structured artificial neural network that wc call modified associative memory is trained with information in Ihc simulator image. Possible variations of the organ posilions and their representations are included in the training data set. The images are subdivided in small regions and a sophisticated search algorithm is applied to find corresponding features, in the second step. the recall by the EPI at the correct position follows. Results: As an example, the Simulator image (Fig, 2b) was stored in the associative memory including variations• An alignment procedure resulted in the displacement map Fig, 2c. In Fig. 2d the associative m~mory was recalled by the corresponding data in the EPI. Conclusion: The proposed approach leads to a higher restoration quality tot the EPI than conventional solutions without inclusion of protreatment data. Features can be tracked to detect deviations of the desired patient position. Variations in shape and position are considered.
r
,I
dynamic aa[a)
I
q
: , I ModtfredAss°clat'veMem°ry
I
L. _
, I
,
tmage j~ : porlal(EPI,
\ x~'/f
:~--
i
I
t f~Mufes [ ' ,mpoftaB--.7
']
t.zzv method 1or
]2' : *sTr~'an;?vge~;nmeerY
"
~L
;i
--
x (a-priori
data from
, I Simulator i
. I
•
~i - "
J
,
~ \ '
Recaltoft~o ]~\l ~Assoclatlve Memory
:
t d
/~ assoc~a e image J
i
Fig. l . Functionality of the Modified Associative Memory
(a)
(b)
(c)
(d)
INMI
Irtg. 2. Marked regions of interest in EPI (a), simulator image (b), warlx:d simulator image with displacement map (c), recall of the associative memory (associated image) by the aligned (warped) EPI (d)
156
Volume 42, Number 1 Supplement, 1998
63 COMPUTER-ASSISTED DECISION MAKING IN PORTAL VERIFICATION - OPTIMIZATION OF A NEURAL NETWORK APPROACH Konrad Leszczynski, Randall Bissett, Daniel Provost, Scott Cosby and Susan Boyko Northeastern Ontario Regional Cancer Centre, 41 Ramsey Lake Road, Sudbury, Ontario, Canada, P3E 5J1 Purpose / Obiective: On-line portal verification of radiation therapy is becoming increasingly important with the introduction of eonformal and threc-dimensional irradiation techniques, which rely on high geometric accuracy in treatment delivery. With the large amount of portal image data and stringent time constraints, the conventional scheme by which the portal verification decision as to the set-up acceptability is made (by a qualified radiation oncologist) is no longer sustainable, On the other hand, our previous studies indicated that delegation of the portal decision making task to other (non-oncologist) professionals would significantly affect the result. Therefore, the objective of this study was to develop, optimize and evaluate on clinical data, an artificial intelligence decision making tool, based on the artificial neural network (ANN) approach, that would approximate, as close as possible, portal verification assessments made by a radiation oncologist expert. Materials & Methods: 328 electronic portal images of tangential breast irradiations were included in the study. The portal images were registered (aligned) with the corresponding simulation radiographs using anatomical features, so that the prescribed and treated radiation field boundaries could be brought into a common reference fi'ame and displayed on a background of the simulation image. These synthetic representations of field placement during treatment were evaluated by a radiation oncologist expert, who rated the treatment set-up acceptability on a scale from 0 to 10. Scores below 5 indicated various degrees of unacceptable treatment set-up. Translational and rotational errors in the placement of different radiation field boundary segments (anterior, posterior, superior and inferior field borders) were quantified using a previously developed automated algorithm (segmentedchamfermatching). The values of the field placement errors formed seven-dimensional feature vectors, which represented each of the 328 treatments. The feature vectors were used as inputs to a three-layer, feed-forward ANN, the output of which was meant to reproduce the expert's ratings. The neural network was trained on the oncologist's ratings using a back-propagation algorithm. The evaluation of the performance of the ANN was carried out by repartitioning the entire data set into training and testing subsets, and by comparing the ANN's ratings with those of the expert. Results: A high degree of agreement between the oncologist and the ANN ratings was observed. The finear correlation coefficient between them was 0.76. The average discrepancy in ratings was -0.07 (the expert's scores were slightly lower, on average) and the standard deviation of the discrepancies was 1.02. Comparisons were also carried out from the portal decision making point of view. Using a decision threshold equal to 5 for both sets of ratings, the ANN classifier was capable of detecting 56% of the portals classified as unacceptable by the oncologist (true positive fraction, TPF), and only 2% of the portals acceptable to the oneologist were mis-classified as "unacceptable" by the ANN (false positive fraction, FPF). Setting the decision threshold to 6 raised the TPF to 80% and the FPF to 12%. By optimizing the ANN training process for accuracy in ratings of "unacceptable" portals, it was possible to raise the TPF to 100% with only a moderate increase of FPF, to 6.4%. The effect of the decision threshold in the optimized ANN was also examined using the Receiver Operating Characteristic (ROC) methodology. Conclusion: An automated portal image classifier based on the Artificial Neural Network approach exhibited excellent agreement with the radiation onculogist expert. After optimization, it was capable of detecting all portals flagged as "unacceptable" by the oncologist. At the same time, the rate of false alarms stayed at an acceptably low level. These results indicate the feasibility of using the ANN portal image classifier as an automated assistant to the radiation therapist, recommending an appropriate decision as to the acceptability or otherwise of a given treatment set-up depicted in a portal image - thereby relieving the radiation onculogist of the burden of managing the large number of on-line portal images.
64 ENHANCEMENT OF EI.E('IFR()NIC PORTAL iMAGES BY INFORMATION MATCHING WITh PRE-TRICATMENT DATA AND B't' AN ASSOCIATIVENEURAL INETWORK G, Krell, B. Michaclis, O. Gademann Otto-von-Guerickc-University Magdeburg. Germany Purpose/Objective: The electronic portal or megaw)ltage imaging technique is an important tool lor the clinician to verify the position of the patient during treatment in radiotherapy. The electronic portal images (EPI) are produced by projecting the body interior within the field of the treatment beam onto a screen and capturing this image by a camera. Due Io the high energy of the treatment beam the unprocessed EPI is poor in quality. The eft~:ct of conventional enhancement techniques is limited. Therelbre, additional infi~rmation from images captured in pro-treatment is used as a-priori knowledge to reach a considerably higher standard of EPI analysis. Materials and Methods: The main idea of the approach consists in the fusion of the dynamic inlormation in the EPI with a-priori knowledge obtained by the simulator image (SI). The simulator image is always taken for target localization during treatment planning and is usually a film image but also other data sources could be used such as CT. Fig. t illustrates the two steps of the algorithm. Firstly, a specially structured artificial neural network that wc call modified associative memory is trained with information in Ihc simulator image. Possible variations of the organ posilions and their representations are included in the training data set. The images are subdivided in small regions and a sophisticated search algorithm is applied to find corresponding features, in the second step. the recall by the EPI at the correct position follows. Results: As an example, the Simulator image (Fig, 2b) was stored in the associative memory including variations• An alignment procedure resulted in the displacement map Fig, 2c. In Fig. 2d the associative m~mory was recalled by the corresponding data in the EPI. Conclusion: The proposed approach leads to a higher restoration quality tot the EPI than conventional solutions without inclusion of protreatment data. Features can be tracked to detect deviations of the desired patient position. Variations in shape and position are considered.
r
,I
dynamic aa[a)
I
q
: , I ModtfredAss°clat'veMem°ry
I
L. _
, I
,
tmage j~ : porlal(EPI,
\ x~'/f
:~--
i
I
t f~Mufes [ ' ,mpoftaB--.7
']
t.zzv method 1or
]2' : *sTr~'an;?vge~;nmeerY
"
~L
;i
--
x (a-priori
data from
, I Simulator i
. I
•
~i - "
J
,
~ \ '
Recaltoft~o ]~\l ~Assoclatlve Memory
:
t d
/~ assoc~a e image J
i
Fig. l . Functionality of the Modified Associative Memory
(a)
(b)
(c)
(d)
INMI
Irtg. 2. Marked regions of interest in EPI (a), simulator image (b), warlx:d simulator image with displacement map (c), recall of the associative memory (associated image) by the aligned (warped) EPI (d)