- Email: [email protected]
The electronic view box: A software tool for radiation therapy treatment verification
Int.
.I. Radiation
Oncology
Pergamon
Biol.
Phys., Vol. 31. No. I, pp. 135-142. 1995 Copyright 0 I994 Else&r Science Ltd Printed in the USA. All rights reserved 0360-3016/95 $9.50 + .oO
0360-3016(94)EOO3300-9
l
Technical Innovations and Notes THE
ELECTRONIC
VIEW THERAPY
BOX: A SOFTWARE TOOL TREATMENT VERIFICATION
FOR RADIATION
WALTER R. BOSCH, D.Sc., DANIEL A. Low, PH.D., RUSSELL L. GERBER, M.S., JEFF M. MICHALSKI, M.D., MARY V. GRAHAM, M.D., CARLOSA.PEREZ, M.D., WILLIAM B. HARMS, B.S. ANDJAMES A. PURDY, PH.D. Radiation Oncology Center, Mallinckrodt Institute of Radiology,WashingtonUniversity Schoolof Medicine, St. Louis, MO Purpose: We have developed a software tool for interactively verifying treatment plan implementation. The Electronic ax (EVB) tool copies the paradigm of current practice but does so electronically. A portal image (online portal image or digitized port film) is displayed side by side with a prescription image (digitized simulator film or digitally reconstructed radiograph). The user can measure distances between features in prescription and portal images and “write” on the display, either to approve the image or to indicate required corrective actions. The EVB tool also provides several features not available in conventional verification practice using a light box. Methods and Materials: The EVB tool has been written in ANSI C using the X window system. The tool makes use of the Virtual Machine Platform and Foundation Library specifications of the NCI-sponsored Radiation Therapy Planning Tools Collaborative Working Group for portability into an arbitrary treatment planning system that conforms to these specifications. The present EVB tool is based on an earlier Verification Image Review tool, but with a substantial redesign of the user interface. A graphical user interface prototyping system was used in iteratively refining the tool layout to allow rapid modifications of the interface in response to user comments. Results: Features of the EVB tool include 1) hierarchical selection of digital portal images based on physician name, patient name, and field identifier; 2) side-by-side presentation of prescription and portal images at equal magnification and orientation, and with independent grayscale controls; 3) “trace” facility for outlining anatomical structures; 4) “ruler” facility for measuring distances; 5) zoomed display of corresponding regions in both images; 6) image contrast enhancement; and 7) communication of portal image evaluation results (approval, block modification, repeat image acquisition, etc.). Conclusion: The EVB tool facilitates the rapid comparison of prescription and portal images and permits electronic communication of corrections in port shape and positioning. Treatment
verification,
Image display, Electronic
portal imaging, Radiation
INTRODUCTION
therapy.
Emerging three-dimensional (3D) conformal radiation therapy techniques have underscored the importance of accurately and precisely positioning patients for treatment. Although these methods promise to achieve improved local control by increasing dose to tumor volumes, they must also shrink field margins to avoid normal-tissue complications. The smaller fields and higher doses of conformal therapy make errors in field placement more
costly becausethe dose to portions of the tumor volume are reduced and the dose to adjacent normal structures are increased to a greater extent than in conventional treatment (4). Verification of patient setup is especially important in conformal therapy because it provides a means of quality assurance for treatment delivery. Additionally, verification of patient position permits evaluation of patient setup and immobilization techniques. Comparison of portal imagesacquired during treatment with prescription imagesobtained in simulation remains
Presentedat the 35th annualmeetingof the American Society for Therapeutic Radiology and Oncology, New Orleans, LA, October 12, 1993. Reprint requeststo: Walter R. Bosch, D.Sc., Mallinckrodt Institute of Radiology, WashingtonUniversity Schoolof Medicine, RadiationOncology Center, Box 8224,5 10S. Kingshighway Blvd., St. Louis, MO 63110. Acknowledgements-Thiswork wassupportedin part by NC1 contract NOl-CM-97564. An earliersetof treatmentverification softwaretools wasdevelopedat WashingtonUniversity aspart
of the NC1 Radiation Treatment PlanningTools contract. Development of thesetools, which demonstratedthe concept of treatment verification usingside-by-sidecomparisonof digital simulation and portal images,wascarried out by a group of physicistsand physiciansincluding Drs. Di Yan, John Wong, Karen Halverson, and Martin Weinhous. Although the Electronic View Box representsa substantialredesignof the user interface, many algorithmsand techniquesusedin the original tool have beenretained. Acceptedfor publication 26 May 1994. 135
136
I. J. Radiation Oncology 0 Biology 0 Physics
the primary method for verifying patient setup and positioning of target volumes. Recent advances in electronic portal imaging systems have made it possible to acquire daily images of each treatment field. To be useful for verifying patient setup, however, these images must be reviewed and analyzed. Efficient software tools are needed to manage this large volume of digital image data, to analyze held placement, and to communicate the results of this analysis. The Electronic View Box (EVB) is a software tool for interactive verification of treatment plan implementation which was developed using work from our online imaging studies (1). The purpose of the EVB tool is to support rapid evaluation and comparison of digital portal images offline by mimicking the conventional practice of reviewing simulation and port films on a view box, and by providing additional capabilities that enhance the verification process. The EVB tool allows a digital portal image (online portal image or digitized port film) to be examined side by side with a prescription image (digitized simulator film or digital radiograph) on a computer display screen. The user can measure distances between on-screen features in
Volume 31. Number 1, 1995
prescription and portal images. The tool supports annotation of portal images to document evaluation of images and to communicate required corrective actions. The EVB tool also provides for the interactive alignment of images, their display at equal magnification, and the control of image contrast, features that are not available in conventional verification practice using a light box. METHODS
AND MATERIALS
The EVB tool has been written in ANSI C using the X window system. The tool makes use of the Virtual Machine Platform and Foundation Library specifications of the NCI-sponsored Radiation Therapy Planning Tools Collaborative Working Group (University of North Carolina; University of Washington, Seattle; and Washington University, St. Louis) for portability into an arbitrary treatment planning system that conforms to these specifications (2, 3, 5, 6). The EVB tool requires an Xl 1 display server with a screen resolution of at least 1152 X 900 pixels ( 1280 X 1024 pixels preferred) capable of displaying eight-bit
Electronic ViewBox .I * I I,ltc,.‘~J. L‘ Mnllinkrodt Radian
Institute of Radiology ancology Ccntcr
Fig. 1. The Electronic View Box patient data selector screen presents hierarchical lists of physician names, patient names, and treatment fields. Selecting a physician causes the names of the physician’s patients to be displayed; selecting a patient causes the patient’s treatment fields to he displayed.
The
electronic view box 0
W. R. BOSCH
et al.
Fig. 2. The Electronic View Box image comparison screen displays a pair of images. The prescription image for a selected treatment port is displayed automatically on the left side of this screen. An image selector menu appears on the right side of the screen and is replaced by the selected portal image. A sliding grayscale control appears below each image. At the bottom of the screen are a dialog/help window (lower left), a ruler display (lower center), and a command menu (lower right).
pseudocolor images. The 256-entry colormap of the X display server is divided into four 64-entry submaps to achieve rapid control over image contrast and to provide an erasable overlay color for templates, graticules, rulers, tracing, and annotation. Submap 0 is used for text display and for backgrounds and buttons in dialog windows. Submaps I and 2 define the grayscales used to display the left and right images, respectively. Submap 3 contains the color used to display erasable overlays in both image windows. A graphical user interface prototyping system was used in iteratively refining the tool layout to allow rapid modifications of the interface in response to user comments. The very large effort required to implement changes in X 11-based application programs makes it costly to construct such programs when the design requirements of the interface are not completely specified. On the other hand, it is difficult to specify an interface design completely without some way of seeing its form and behavior. This ‘Sun Microsystems, Inc.
dilemma has motivated investment in prototyping techniques for developing the EVB user interface. The ability to easily and rapidly construct a mock-up user interface has proven invaluable. Using the Developer’s
GuideTM prototyping
software,’
a mode1 user interface for the EVB tool was developed, tested, and modified. Although no images were displayed by the model, a realistic simulation of responses to button presses and menu selections was achieved. This model has been used to explore alternative layouts for displays and controls and as a vehicle for discussions with physicians and physicists concerning design preferences and requirements. RESULTS The EVB tool user interface provides the basic functions of the earlier Verification Image Review (VIR) and Cu-
mulative Verification
Image Analysis tools (7), including
138
1. .I. Radiation Oncology 0 Biology 0 Physics
presentation of aligned images at equal magnification. zoomed display of corresponding regions of both images, and image contrast enhancement. The Electronic View Box also includes several new features that are not present in the earlier treatment verification tools. Patient data selector screen Digital portal images are selected using hierarchical lists of physician name, patient name, and treatment field (Fig. 1). When a physician is selected, a list of all the patients in the care of the selected physician for which images are available is displayed. Similarly, when a patient is selected, a list of treatment fields for which images are present is displayed. The numbers of “pending” images, that is, those that have not yet been evaluated, for each patient and each treatment field are also displayed in the corresponding lists. Side-by-side image comparison screen When a treatment field has been selected and confirmed, the image selector screen is replaced by the side-
Volume 31. Number I. 1995
by-side image comparison screen (Fig. 2). The prescription image (simulation film or digitally reconstructed radiograph) is displayed automatically on the left side of the view box. Prescription and portal images for the treatment field are then selected from a list of images on the right side of the view box. For each image in the list, an identifying name, the date, and time of acquisition, as well as its evaluation status are displayed. The evaluation status may be “new” (image has not yet been viewed), “pending” (image has been viewed, but not fully evaluated), or “done” (evaluation complete). When an image has been selected and confirmed, this list of images is replaced by the selected image. Dual, independent controls for setting the grayscale window and level parameters of the right and left images are located below the image displays. Independent control of image contrast parameters is particularly important in comparing images acquired from differing sources, for example, digitized films and electronic portal imaging devices. A dialog/help window (left), a ruler display (center),
Fig. 3. Alignment of prescription and portal images on the Electronic View Box is performed by defining a template of some feature such as the port edge in the prescription image (a). Each portal image is aligned to the prescription image by matching this template to the corresponding feature in the portal image (b). The mouse buttons are used to translate, rotate, and scale the template until it matches the portal image (c). When the template has been aligned, the portal image is redrawn so that the aligned features occupy corresponding positions of both images
(4.
The electronicviewbox 0 W. R. BOSCH and a command menu (right) are located below the grayscale controls along the bottom of the EVB image comparison panel. The dialog window presents information and control buttons that are specific to the current EVB mode or function. The ruler display contains a pair of scrolling lists of distances measured by the ruler function on the right and left image windows. The command menu presents buttons for invoking the various EVB functions, as well as for changing the image or treatment field to be examined and for exiting the EVB program. Image alignment Simulation and portal images are aligned interactively on the Electronic View Box by matching an overlay template, defined on the simulation image, to the corresponding feature of the portal image. A template of the reference feature (usually the port edge) is defined on the simulation image by drawing with the mouse (Fig. 3a). In general, this step is performed only once; the template is saved for reuse with all images of this treatment field. Each portal image is aligned to the simulation image by matching the template to the corresponding feature (port
er nl.
139
edge) in the portal image (Fig. 3b and c). The three mouse buttons are used to position, scale, and rotate the overlay template. After the template has been matched to the port outline, the transformation required to bring the template into alignment with the portal image is used to redraw the portal image so that the aligned features occupy corresponding pixels of both images (Fig. 3d). The “trace” function can then be used to trace corresponding points of both images. Distance measurement The RULER function of the Electronic View Box facilitates measurement of distances in simulation and portal images. The DEFINE GRID function is used to calibrate image magnification by matching an overlay grid to a graticule in the simulation image. Once the image magnification has been calibrated, the RULER function is used to measure distances by clicking and dragging with the mouse. A label is printed near the starting point of the measurement interval. Measured distances in each image window are displayed on one of two scrolling lists in the ruler window (Fig. 4).
Fig. 4. The Electronic View Box ruler function allows distance measurement in prescription and portal images. Measurement intervals are specified on the images by clicking and dragging with the mouse. Measured distances are displayed in one of two lists in the ruler window.
140
1. J. Radiation Oncology 0 Biology 0 Physics
Image zoom A zoom function provides additional magnification of corresponding portions of both simulation and portal images. The user specifies the area to be zoomed by pressing the left mouse button at the center of this area. As the mouse is dragged outward, a rectangular overlay follows the mouse cursor to indicate the selected area. When the mouse button is released, the rectangular overlay remains fixed. If the user then clicks inside the rectangle, the selected area of each image is magnified to fill the image window. If the user clicks outside the rectangle, the zoom function is aborted. The current version of the EVB tool permits zooming of corresponding portions of both images, but does not allow measurement, tracing, or annotation of zoomed images. Image evahation In the conventional practice of reviewing films on a light box, physicians often write on simulation and port films using a grease pencil. The Electronic View Box provides a similar facility for annotating portal images. The evaluate function presents four options (Fig. 5): the user may (a) mark the image “approved,” (b) prescribe cor-
Volume 31. Number 1. 1995
rections to the shape (“trim” or “fill”) or placemenf (“shift” or “rotate”) of the port, (c) indicate that the portal image is unusable because of poor image quality, improper patient setup, or the need for a “double exposure,” or(d) add a text annotation to the image. DISCUSSION
The Electronic View Box (EVB) tool is a software tool for interactively verifying treatment plan implementation. The EVB tool copies the traditional side-by-side image comparison paradigm, but the digital representation of images by this tool permits much greater flexibility in image storage and communication, the control of image grayscale and magnification, and the interactive alignment and comparison of images. A comparison of several features of the EVB tool with their conventional counterparts is presented in Table 1. The emergence of electronic portal imaging (EPI) systems has generated demand for software tools that can be used to display and compare electronic portal images. In response to this demand, several manufacturers of EPI systems are developing software for portal image review.
Fig. 5. The Electronic View Box provides a facility for annotating portal images. The user may approve the image, prescribe changes in the shape or position of the port, request that additional portal images be acquired, or add a text message to the image.
The electronic view box 0 W. R. BOSCH
et al.
141
Table 1. Summary of electronic view box functions with their conventional counterparts Feature
Conventional practice
Electronic view box
Display
Side-by-side films on lightbox
Side-by-side digital images on computer display
Image media
Simuiation and portal films
Grayscale control
Very limited (“hot light”)
Alignment
Manual positioning; film
Digitized simulation and portal films, on-line images, computed radiographs Independent window and level control for each image Interactive alignment using template matching
Magnification
None
Images displayed at equal magnification
control Zoom
None
Zoom-in to corresponding regions of both images
Tracing
Trace through overlaid film
Trace corresponding pixels of aligned images at equal magnification
overlay
with greasepencil (manual alignment, no magnification control) Distance measurement
Ruler + hand calculation
Click + drag (after calibration);
Annotation
Mark on film with grease
Draw with mouse to specify trim, fill, shift, and rotation of port; type message on keyboard. Annotation and requests for corrective action are
multiple measurements displayed.
pencil
communicated via computer network.
Often, this software is closely linked to the manufacturer’s imaging system and is difficult to use with computers or imaging systems other than those supplied by the manufacturer. An important distinguishing feature of the EVB tool is its portability. The EVB tool can be run without modification on any system that satisfies the requirements of the virtual machine platform specification and for which a Foundation Library exists to translate between file formats used by the treatment planning software and NC1 data formats (3, 5). In addition to improving the process of treatment verification by facilitating the evaluation of portal images, the EVB tool also permits the use of digital images in the context of networked communications. Not only is this important for the evaluation of images acquired electronically, but it means that images and the results of their evaluation may be communicated rapidly and integrated into the overall planning, treatment, and verification system. In fact, the role of the EVB tool is that of a user interface for a radiation therapy picture archiving and communication system. The EVB tool is based on an earlier treatment verification tool: the verification image review (VIR) tool (7). Experience gained from experimental use of this treatment verification tool at Washington University made it apparent that although the user interface of this tool is adequate to support its academic use in the review of portal image data, the interface is too cumbersome to be effective for daily clinical use in treatment verification. This evaluation prompted the decision to redesign the user interface and to change the name of the tool to Electronic View
Box (EVB) to more accurately reflect its role in the clinical review of portal images. Most of the algorithms and methods underlying VIR have been used without major modification in the new EVB tool. Several changes have been made, however, in the display of images and the functions provided for control of these displays. First, the view box paradigm has been emphasized by the use of a pair of side-by-side images each having the aspect ratio of a 14 X 17 inch film. Second, the layout and labeling of controls has been redesigned to make their meanings clearer and to simplify accessto frequently used functions. Finally, several core functions have been enhanced including tracing of corresponding points in both prescription and portal images, and measuring the size of features in these images. The EVB tool provides independent control over window and level parameters of the grayscales used in the left and right image windows. A “sliderbar” control was developed to permit the user to set these parameters: the center of the sliderbar can be “dragged” with the mouse to set the grayscale level and the ends of the sliderbar can be “dragged” with the mouse to set the width of the grayscale window. Four 64-entry colormaps are used to achieve rapid, independent control over the contrast of the two side-byside images and to provide an erasable overlay color for templates, graticules, rulers, tracing, and annotation. The six-bit grayscale colormaps used in this approach appear to be adequate for image display when the grayscale window is relatively wide. With a narrow grayscale window, however, only a few colormap entries are actually used
142
1.J. Radiation Oncology 0 Biology0 Physics
to represent the range of intensities between black and white, and intensity “contours” become quite noticeable as bands on the image. An “expand” function is provided to temporarily remap image pixel values to use the entire 64 grayscale steps to represent the range of intensities from black to white. Because remapping the image is a timeconsuming process, no further adjustment of the grayscale is possible in expanded mode. Except when executing functions that draw new images on the display screen, the EVB tool responds quite rapidly to user input, typically, within 1 s. When the user selects a new simulation or portal image, however, the response is somewhat slower. Tool response times are strongly dependent on the speed of the computing hardware and network interconnections used. For the workstations and X-terminals that we have tested, response times for drawing new images range between 4 and 12 s. The development of the EVB tool has been instructive in helping to identify requirements for implementing electronic tools for clinical radiation treatment verification. To be useful for prescribing corrective action based on portal image evaluation, the results of this evaluation must be communicated to those responsible for patient setup and block modifications. It has become clear that
Volume 31, Number 1, 1995
additional tools are needed to provide the infrastructure for storing and displaying the evaluation results generated by the EVB tool. The EVB software generates data files that describe these results, but other software is needed to examine these files and communicate their content to the appropriate personnel. Work continues at out institution on the integration of the EVB tool into a system for electronic treatment verification.
CONCLUSIONS
The Electronic View Box is one of several software tools being developed as part of an NC1 sponsored research contract to develop radiation treatment planning software tools (2, 3, 5,6). It is hoped that the features described in this report will be useful for others developing similar software for offline treatment verification image analysis. Additional features to support the cumulative analysis of portal images over the course of treatment are planned for a second version of the EVB software. These include functions for determining port position with respect to patient anatomy and for extracting port position and coverage statistics.
REFERENCES 1. Bosch, W. R.; Low, D. A.; Gerber, R. L.; Graham, M. V.;
Michalski, J. M.; Perez, C. A.; Purdy, J. A. An electronic ViewBox tool for radiation therapy treatment verification (Abstr.). Med. Phys. 20:897; 1993. 2. Drzymala, R. E.; Holman, M. D.; Yan, D.; Harms, W. B.; Jain, N. L.; Kahn, M. G.; Emami, B.; Purdy, J. A. Integrated
software tools for the evaluation of radiotherapy treatment plans. Int. J. Radiat. Oncol. Biol. Phys. 30:909-919; 1994. 3. Kalet, I. J.; Chaney, E. L.; Purdy, J. A.; Zink, S. Radiotherapy treatment planning tools: A collaborative project of the National Cancer Institute (Abstr.). Med. Phys. 17: 529; 1990. 4. Michalski, J. M.; Wong, J. W.; Bosch, W. R.; Yan, D.; Cheng, A.; Gerber, R. L.; Graham, M. V.; Low, D.; Vali-
centi, R. K.; Piephoff, J. V. An evaluation of two methods of anatomical alignment of radiotherapy portal images. Int. J. Radiat. Oncol. Biol. Phys. 27:1199-1206; 1993. Purdy, J. A.; Kalet, I. J.; Chaney, E. L.; Zink, S. Radiotherapy treatment planning tools: A collaborative project of the National Cancer Institute (Abstr.). Med. Phys. 18: 666; 1991. Purdy, J. A.; Austin-Seymour, M. M.; Leibel, S. A.; Rosen-
man, J. G. 3-D radiotherapy treatment planning tools (Panel XII). Int. J. Radiat. Oncol. Biol. Phys. 24:68; 1992. Wong, J.; Yan, D.; Harms, W. B.; Purdy, J. A. Software tools for radiation therapy treatment verification (Abstr.). Med. Phys. 18:6 11; 199 1.
.I. Radiation
Oncology
Pergamon
Biol.
Phys., Vol. 31. No. I, pp. 135-142. 1995 Copyright 0 I994 Else&r Science Ltd Printed in the USA. All rights reserved 0360-3016/95 $9.50 + .oO
0360-3016(94)EOO3300-9
l
Technical Innovations and Notes THE
ELECTRONIC
VIEW THERAPY
BOX: A SOFTWARE TOOL TREATMENT VERIFICATION
FOR RADIATION
WALTER R. BOSCH, D.Sc., DANIEL A. Low, PH.D., RUSSELL L. GERBER, M.S., JEFF M. MICHALSKI, M.D., MARY V. GRAHAM, M.D., CARLOSA.PEREZ, M.D., WILLIAM B. HARMS, B.S. ANDJAMES A. PURDY, PH.D. Radiation Oncology Center, Mallinckrodt Institute of Radiology,WashingtonUniversity Schoolof Medicine, St. Louis, MO Purpose: We have developed a software tool for interactively verifying treatment plan implementation. The Electronic ax (EVB) tool copies the paradigm of current practice but does so electronically. A portal image (online portal image or digitized port film) is displayed side by side with a prescription image (digitized simulator film or digitally reconstructed radiograph). The user can measure distances between features in prescription and portal images and “write” on the display, either to approve the image or to indicate required corrective actions. The EVB tool also provides several features not available in conventional verification practice using a light box. Methods and Materials: The EVB tool has been written in ANSI C using the X window system. The tool makes use of the Virtual Machine Platform and Foundation Library specifications of the NCI-sponsored Radiation Therapy Planning Tools Collaborative Working Group for portability into an arbitrary treatment planning system that conforms to these specifications. The present EVB tool is based on an earlier Verification Image Review tool, but with a substantial redesign of the user interface. A graphical user interface prototyping system was used in iteratively refining the tool layout to allow rapid modifications of the interface in response to user comments. Results: Features of the EVB tool include 1) hierarchical selection of digital portal images based on physician name, patient name, and field identifier; 2) side-by-side presentation of prescription and portal images at equal magnification and orientation, and with independent grayscale controls; 3) “trace” facility for outlining anatomical structures; 4) “ruler” facility for measuring distances; 5) zoomed display of corresponding regions in both images; 6) image contrast enhancement; and 7) communication of portal image evaluation results (approval, block modification, repeat image acquisition, etc.). Conclusion: The EVB tool facilitates the rapid comparison of prescription and portal images and permits electronic communication of corrections in port shape and positioning. Treatment
verification,
Image display, Electronic
portal imaging, Radiation
INTRODUCTION
therapy.
Emerging three-dimensional (3D) conformal radiation therapy techniques have underscored the importance of accurately and precisely positioning patients for treatment. Although these methods promise to achieve improved local control by increasing dose to tumor volumes, they must also shrink field margins to avoid normal-tissue complications. The smaller fields and higher doses of conformal therapy make errors in field placement more
costly becausethe dose to portions of the tumor volume are reduced and the dose to adjacent normal structures are increased to a greater extent than in conventional treatment (4). Verification of patient setup is especially important in conformal therapy because it provides a means of quality assurance for treatment delivery. Additionally, verification of patient position permits evaluation of patient setup and immobilization techniques. Comparison of portal imagesacquired during treatment with prescription imagesobtained in simulation remains
Presentedat the 35th annualmeetingof the American Society for Therapeutic Radiology and Oncology, New Orleans, LA, October 12, 1993. Reprint requeststo: Walter R. Bosch, D.Sc., Mallinckrodt Institute of Radiology, WashingtonUniversity Schoolof Medicine, RadiationOncology Center, Box 8224,5 10S. Kingshighway Blvd., St. Louis, MO 63110. Acknowledgements-Thiswork wassupportedin part by NC1 contract NOl-CM-97564. An earliersetof treatmentverification softwaretools wasdevelopedat WashingtonUniversity aspart
of the NC1 Radiation Treatment PlanningTools contract. Development of thesetools, which demonstratedthe concept of treatment verification usingside-by-sidecomparisonof digital simulation and portal images,wascarried out by a group of physicistsand physiciansincluding Drs. Di Yan, John Wong, Karen Halverson, and Martin Weinhous. Although the Electronic View Box representsa substantialredesignof the user interface, many algorithmsand techniquesusedin the original tool have beenretained. Acceptedfor publication 26 May 1994. 135
136
I. J. Radiation Oncology 0 Biology 0 Physics
the primary method for verifying patient setup and positioning of target volumes. Recent advances in electronic portal imaging systems have made it possible to acquire daily images of each treatment field. To be useful for verifying patient setup, however, these images must be reviewed and analyzed. Efficient software tools are needed to manage this large volume of digital image data, to analyze held placement, and to communicate the results of this analysis. The Electronic View Box (EVB) is a software tool for interactive verification of treatment plan implementation which was developed using work from our online imaging studies (1). The purpose of the EVB tool is to support rapid evaluation and comparison of digital portal images offline by mimicking the conventional practice of reviewing simulation and port films on a view box, and by providing additional capabilities that enhance the verification process. The EVB tool allows a digital portal image (online portal image or digitized port film) to be examined side by side with a prescription image (digitized simulator film or digital radiograph) on a computer display screen. The user can measure distances between on-screen features in
Volume 31. Number 1, 1995
prescription and portal images. The tool supports annotation of portal images to document evaluation of images and to communicate required corrective actions. The EVB tool also provides for the interactive alignment of images, their display at equal magnification, and the control of image contrast, features that are not available in conventional verification practice using a light box. METHODS
AND MATERIALS
The EVB tool has been written in ANSI C using the X window system. The tool makes use of the Virtual Machine Platform and Foundation Library specifications of the NCI-sponsored Radiation Therapy Planning Tools Collaborative Working Group (University of North Carolina; University of Washington, Seattle; and Washington University, St. Louis) for portability into an arbitrary treatment planning system that conforms to these specifications (2, 3, 5, 6). The EVB tool requires an Xl 1 display server with a screen resolution of at least 1152 X 900 pixels ( 1280 X 1024 pixels preferred) capable of displaying eight-bit
Electronic ViewBox .I * I I,ltc,.‘~J. L‘ Mnllinkrodt Radian
Institute of Radiology ancology Ccntcr
Fig. 1. The Electronic View Box patient data selector screen presents hierarchical lists of physician names, patient names, and treatment fields. Selecting a physician causes the names of the physician’s patients to be displayed; selecting a patient causes the patient’s treatment fields to he displayed.
The
electronic view box 0
W. R. BOSCH
et al.
Fig. 2. The Electronic View Box image comparison screen displays a pair of images. The prescription image for a selected treatment port is displayed automatically on the left side of this screen. An image selector menu appears on the right side of the screen and is replaced by the selected portal image. A sliding grayscale control appears below each image. At the bottom of the screen are a dialog/help window (lower left), a ruler display (lower center), and a command menu (lower right).
pseudocolor images. The 256-entry colormap of the X display server is divided into four 64-entry submaps to achieve rapid control over image contrast and to provide an erasable overlay color for templates, graticules, rulers, tracing, and annotation. Submap 0 is used for text display and for backgrounds and buttons in dialog windows. Submaps I and 2 define the grayscales used to display the left and right images, respectively. Submap 3 contains the color used to display erasable overlays in both image windows. A graphical user interface prototyping system was used in iteratively refining the tool layout to allow rapid modifications of the interface in response to user comments. The very large effort required to implement changes in X 11-based application programs makes it costly to construct such programs when the design requirements of the interface are not completely specified. On the other hand, it is difficult to specify an interface design completely without some way of seeing its form and behavior. This ‘Sun Microsystems, Inc.
dilemma has motivated investment in prototyping techniques for developing the EVB user interface. The ability to easily and rapidly construct a mock-up user interface has proven invaluable. Using the Developer’s
GuideTM prototyping
software,’
a mode1 user interface for the EVB tool was developed, tested, and modified. Although no images were displayed by the model, a realistic simulation of responses to button presses and menu selections was achieved. This model has been used to explore alternative layouts for displays and controls and as a vehicle for discussions with physicians and physicists concerning design preferences and requirements. RESULTS The EVB tool user interface provides the basic functions of the earlier Verification Image Review (VIR) and Cu-
mulative Verification
Image Analysis tools (7), including
138
1. .I. Radiation Oncology 0 Biology 0 Physics
presentation of aligned images at equal magnification. zoomed display of corresponding regions of both images, and image contrast enhancement. The Electronic View Box also includes several new features that are not present in the earlier treatment verification tools. Patient data selector screen Digital portal images are selected using hierarchical lists of physician name, patient name, and treatment field (Fig. 1). When a physician is selected, a list of all the patients in the care of the selected physician for which images are available is displayed. Similarly, when a patient is selected, a list of treatment fields for which images are present is displayed. The numbers of “pending” images, that is, those that have not yet been evaluated, for each patient and each treatment field are also displayed in the corresponding lists. Side-by-side image comparison screen When a treatment field has been selected and confirmed, the image selector screen is replaced by the side-
Volume 31. Number I. 1995
by-side image comparison screen (Fig. 2). The prescription image (simulation film or digitally reconstructed radiograph) is displayed automatically on the left side of the view box. Prescription and portal images for the treatment field are then selected from a list of images on the right side of the view box. For each image in the list, an identifying name, the date, and time of acquisition, as well as its evaluation status are displayed. The evaluation status may be “new” (image has not yet been viewed), “pending” (image has been viewed, but not fully evaluated), or “done” (evaluation complete). When an image has been selected and confirmed, this list of images is replaced by the selected image. Dual, independent controls for setting the grayscale window and level parameters of the right and left images are located below the image displays. Independent control of image contrast parameters is particularly important in comparing images acquired from differing sources, for example, digitized films and electronic portal imaging devices. A dialog/help window (left), a ruler display (center),
Fig. 3. Alignment of prescription and portal images on the Electronic View Box is performed by defining a template of some feature such as the port edge in the prescription image (a). Each portal image is aligned to the prescription image by matching this template to the corresponding feature in the portal image (b). The mouse buttons are used to translate, rotate, and scale the template until it matches the portal image (c). When the template has been aligned, the portal image is redrawn so that the aligned features occupy corresponding positions of both images
(4.
The electronicviewbox 0 W. R. BOSCH and a command menu (right) are located below the grayscale controls along the bottom of the EVB image comparison panel. The dialog window presents information and control buttons that are specific to the current EVB mode or function. The ruler display contains a pair of scrolling lists of distances measured by the ruler function on the right and left image windows. The command menu presents buttons for invoking the various EVB functions, as well as for changing the image or treatment field to be examined and for exiting the EVB program. Image alignment Simulation and portal images are aligned interactively on the Electronic View Box by matching an overlay template, defined on the simulation image, to the corresponding feature of the portal image. A template of the reference feature (usually the port edge) is defined on the simulation image by drawing with the mouse (Fig. 3a). In general, this step is performed only once; the template is saved for reuse with all images of this treatment field. Each portal image is aligned to the simulation image by matching the template to the corresponding feature (port
er nl.
139
edge) in the portal image (Fig. 3b and c). The three mouse buttons are used to position, scale, and rotate the overlay template. After the template has been matched to the port outline, the transformation required to bring the template into alignment with the portal image is used to redraw the portal image so that the aligned features occupy corresponding pixels of both images (Fig. 3d). The “trace” function can then be used to trace corresponding points of both images. Distance measurement The RULER function of the Electronic View Box facilitates measurement of distances in simulation and portal images. The DEFINE GRID function is used to calibrate image magnification by matching an overlay grid to a graticule in the simulation image. Once the image magnification has been calibrated, the RULER function is used to measure distances by clicking and dragging with the mouse. A label is printed near the starting point of the measurement interval. Measured distances in each image window are displayed on one of two scrolling lists in the ruler window (Fig. 4).
Fig. 4. The Electronic View Box ruler function allows distance measurement in prescription and portal images. Measurement intervals are specified on the images by clicking and dragging with the mouse. Measured distances are displayed in one of two lists in the ruler window.
140
1. J. Radiation Oncology 0 Biology 0 Physics
Image zoom A zoom function provides additional magnification of corresponding portions of both simulation and portal images. The user specifies the area to be zoomed by pressing the left mouse button at the center of this area. As the mouse is dragged outward, a rectangular overlay follows the mouse cursor to indicate the selected area. When the mouse button is released, the rectangular overlay remains fixed. If the user then clicks inside the rectangle, the selected area of each image is magnified to fill the image window. If the user clicks outside the rectangle, the zoom function is aborted. The current version of the EVB tool permits zooming of corresponding portions of both images, but does not allow measurement, tracing, or annotation of zoomed images. Image evahation In the conventional practice of reviewing films on a light box, physicians often write on simulation and port films using a grease pencil. The Electronic View Box provides a similar facility for annotating portal images. The evaluate function presents four options (Fig. 5): the user may (a) mark the image “approved,” (b) prescribe cor-
Volume 31. Number 1. 1995
rections to the shape (“trim” or “fill”) or placemenf (“shift” or “rotate”) of the port, (c) indicate that the portal image is unusable because of poor image quality, improper patient setup, or the need for a “double exposure,” or(d) add a text annotation to the image. DISCUSSION
The Electronic View Box (EVB) tool is a software tool for interactively verifying treatment plan implementation. The EVB tool copies the traditional side-by-side image comparison paradigm, but the digital representation of images by this tool permits much greater flexibility in image storage and communication, the control of image grayscale and magnification, and the interactive alignment and comparison of images. A comparison of several features of the EVB tool with their conventional counterparts is presented in Table 1. The emergence of electronic portal imaging (EPI) systems has generated demand for software tools that can be used to display and compare electronic portal images. In response to this demand, several manufacturers of EPI systems are developing software for portal image review.
Fig. 5. The Electronic View Box provides a facility for annotating portal images. The user may approve the image, prescribe changes in the shape or position of the port, request that additional portal images be acquired, or add a text message to the image.
The electronic view box 0 W. R. BOSCH
et al.
141
Table 1. Summary of electronic view box functions with their conventional counterparts Feature
Conventional practice
Electronic view box
Display
Side-by-side films on lightbox
Side-by-side digital images on computer display
Image media
Simuiation and portal films
Grayscale control
Very limited (“hot light”)
Alignment
Manual positioning; film
Digitized simulation and portal films, on-line images, computed radiographs Independent window and level control for each image Interactive alignment using template matching
Magnification
None
Images displayed at equal magnification
control Zoom
None
Zoom-in to corresponding regions of both images
Tracing
Trace through overlaid film
Trace corresponding pixels of aligned images at equal magnification
overlay
with greasepencil (manual alignment, no magnification control) Distance measurement
Ruler + hand calculation
Click + drag (after calibration);
Annotation
Mark on film with grease
Draw with mouse to specify trim, fill, shift, and rotation of port; type message on keyboard. Annotation and requests for corrective action are
multiple measurements displayed.
pencil
communicated via computer network.
Often, this software is closely linked to the manufacturer’s imaging system and is difficult to use with computers or imaging systems other than those supplied by the manufacturer. An important distinguishing feature of the EVB tool is its portability. The EVB tool can be run without modification on any system that satisfies the requirements of the virtual machine platform specification and for which a Foundation Library exists to translate between file formats used by the treatment planning software and NC1 data formats (3, 5). In addition to improving the process of treatment verification by facilitating the evaluation of portal images, the EVB tool also permits the use of digital images in the context of networked communications. Not only is this important for the evaluation of images acquired electronically, but it means that images and the results of their evaluation may be communicated rapidly and integrated into the overall planning, treatment, and verification system. In fact, the role of the EVB tool is that of a user interface for a radiation therapy picture archiving and communication system. The EVB tool is based on an earlier treatment verification tool: the verification image review (VIR) tool (7). Experience gained from experimental use of this treatment verification tool at Washington University made it apparent that although the user interface of this tool is adequate to support its academic use in the review of portal image data, the interface is too cumbersome to be effective for daily clinical use in treatment verification. This evaluation prompted the decision to redesign the user interface and to change the name of the tool to Electronic View
Box (EVB) to more accurately reflect its role in the clinical review of portal images. Most of the algorithms and methods underlying VIR have been used without major modification in the new EVB tool. Several changes have been made, however, in the display of images and the functions provided for control of these displays. First, the view box paradigm has been emphasized by the use of a pair of side-by-side images each having the aspect ratio of a 14 X 17 inch film. Second, the layout and labeling of controls has been redesigned to make their meanings clearer and to simplify accessto frequently used functions. Finally, several core functions have been enhanced including tracing of corresponding points in both prescription and portal images, and measuring the size of features in these images. The EVB tool provides independent control over window and level parameters of the grayscales used in the left and right image windows. A “sliderbar” control was developed to permit the user to set these parameters: the center of the sliderbar can be “dragged” with the mouse to set the grayscale level and the ends of the sliderbar can be “dragged” with the mouse to set the width of the grayscale window. Four 64-entry colormaps are used to achieve rapid, independent control over the contrast of the two side-byside images and to provide an erasable overlay color for templates, graticules, rulers, tracing, and annotation. The six-bit grayscale colormaps used in this approach appear to be adequate for image display when the grayscale window is relatively wide. With a narrow grayscale window, however, only a few colormap entries are actually used
142
1.J. Radiation Oncology 0 Biology0 Physics
to represent the range of intensities between black and white, and intensity “contours” become quite noticeable as bands on the image. An “expand” function is provided to temporarily remap image pixel values to use the entire 64 grayscale steps to represent the range of intensities from black to white. Because remapping the image is a timeconsuming process, no further adjustment of the grayscale is possible in expanded mode. Except when executing functions that draw new images on the display screen, the EVB tool responds quite rapidly to user input, typically, within 1 s. When the user selects a new simulation or portal image, however, the response is somewhat slower. Tool response times are strongly dependent on the speed of the computing hardware and network interconnections used. For the workstations and X-terminals that we have tested, response times for drawing new images range between 4 and 12 s. The development of the EVB tool has been instructive in helping to identify requirements for implementing electronic tools for clinical radiation treatment verification. To be useful for prescribing corrective action based on portal image evaluation, the results of this evaluation must be communicated to those responsible for patient setup and block modifications. It has become clear that
Volume 31, Number 1, 1995
additional tools are needed to provide the infrastructure for storing and displaying the evaluation results generated by the EVB tool. The EVB software generates data files that describe these results, but other software is needed to examine these files and communicate their content to the appropriate personnel. Work continues at out institution on the integration of the EVB tool into a system for electronic treatment verification.
CONCLUSIONS
The Electronic View Box is one of several software tools being developed as part of an NC1 sponsored research contract to develop radiation treatment planning software tools (2, 3, 5,6). It is hoped that the features described in this report will be useful for others developing similar software for offline treatment verification image analysis. Additional features to support the cumulative analysis of portal images over the course of treatment are planned for a second version of the EVB software. These include functions for determining port position with respect to patient anatomy and for extracting port position and coverage statistics.
REFERENCES 1. Bosch, W. R.; Low, D. A.; Gerber, R. L.; Graham, M. V.;
Michalski, J. M.; Perez, C. A.; Purdy, J. A. An electronic ViewBox tool for radiation therapy treatment verification (Abstr.). Med. Phys. 20:897; 1993. 2. Drzymala, R. E.; Holman, M. D.; Yan, D.; Harms, W. B.; Jain, N. L.; Kahn, M. G.; Emami, B.; Purdy, J. A. Integrated
software tools for the evaluation of radiotherapy treatment plans. Int. J. Radiat. Oncol. Biol. Phys. 30:909-919; 1994. 3. Kalet, I. J.; Chaney, E. L.; Purdy, J. A.; Zink, S. Radiotherapy treatment planning tools: A collaborative project of the National Cancer Institute (Abstr.). Med. Phys. 17: 529; 1990. 4. Michalski, J. M.; Wong, J. W.; Bosch, W. R.; Yan, D.; Cheng, A.; Gerber, R. L.; Graham, M. V.; Low, D.; Vali-
centi, R. K.; Piephoff, J. V. An evaluation of two methods of anatomical alignment of radiotherapy portal images. Int. J. Radiat. Oncol. Biol. Phys. 27:1199-1206; 1993. Purdy, J. A.; Kalet, I. J.; Chaney, E. L.; Zink, S. Radiotherapy treatment planning tools: A collaborative project of the National Cancer Institute (Abstr.). Med. Phys. 18: 666; 1991. Purdy, J. A.; Austin-Seymour, M. M.; Leibel, S. A.; Rosen-
man, J. G. 3-D radiotherapy treatment planning tools (Panel XII). Int. J. Radiat. Oncol. Biol. Phys. 24:68; 1992. Wong, J.; Yan, D.; Harms, W. B.; Purdy, J. A. Software tools for radiation therapy treatment verification (Abstr.). Med. Phys. 18:6 11; 199 1.