- Email: [email protected]
Application of optical photogrammetry in radiation oncology: HDR surface mold brachytherapy
Brachytherapy
-
(2019)
-
Application of optical photogrammetry in radiation oncology: HDR surface mold brachytherapy Michael J.J. Douglass*, Alexandre M. Carac¸a Santos School of Physical Sciences, University of Adelaide, South Australia, Australia Department of Medical Physics, Royal Adelaide Hospital, South Australia, Australia
ABSTRACT
PURPOSE: We propose a novel method of designing surface mold brachytherapy applicators using optical photogrammetry. The accuracy of this technique for the purpose of 3D-printing surface mold brachytherapy applicators is investigated. METHODS AND MATERIALS: Photogrammetry was used to generate a 3D model of a patient’s right arm. The geometric accuracy of the model was evaluated against CT in terms of volume, surface area, and the Hausdorff distance. A surface mold applicator was then 3D printed using this reconstructed model. The accuracy was evaluated by analyzing the displacement and air-gap volumes between the applicator and plaster cast on a CT image. This technique was subsequently applied to generate a 3D-printed applicator of the author’s hand directly, as a proof of principle, using only photographic images. RESULTS: The volume and surface area of the model were within 0.1% and 2.6% of the CTobtained values, respectively. Using the Hausdorff distance metric, it was determined that 93% of the visible vertices present in the CT-derived model had a matching vertex on the photogrammetry-derived model within 1 mm, indicating a high level of similarity. The maximum displacement between the plaster cast of the patient’s arm and the photo-derived 3D-printed applicator was 1.2 mm with a total air-gap volume of approximately 0.05 cm3. CONCLUSIONS: Photogrammetry has been applied to the task of generating 3D-printed brachytherapy surface mold applicators. The current work demonstrates the feasibility and accuracy of this technique and how it may be incorporated into a 3D-printing brachytherapy workflow. Crown Copyright Ó 2019 Published by Elsevier Inc. on behalf of American Brachytherapy Society. All rights reserved.
Keywords:
Photogrammetry; Brachytherapy; Surface mold; Blender; Slicer 3D; AliceVision; Bolus
1. Introduction High-dose-rate (HDR) surface mold brachytherapy is a radiotherapy modality used in the treatment of skin lesions. Owing to the rapid dose falloff of HDR sources, customized surface molds have been shown to produce highly conformal dose distributions on irregular surfaces [1].
Received 25 February 2019; received in revised form 23 April 2019; accepted 22 May 2019. Conflict of interest: The authors have no conflicts of interest to disclose. Financial disclosure: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. * Corresponding author. School of Physical Sciences, University of Adelaide, SA 5005, Australia. E-mail address: [email protected] (M.J.J. Douglass).
The major steps involved in developing a surface-type brachytherapy treatment in our center have been adopted from the GEC-ESTRO guidelines [2,3]. This procedure typically requires as many as three staff (radiation therapists and physicists) at any given time. The total preparation time for each customized surface mold in our hospital typically takes less than five staff hours with more complex sites taking up to seven staff hours to complete. This process includes construction of the plaster cast, construction of the wax mold, and placement of the catheters. Recent reports have evaluated the use of plastic 3Dprinted brachytherapy applicators from a CT image of the patient’s treatment site [2,4e9]. These studies found that the 3D-printed molds outperformed manual wax applicators in terms of dosimetric accuracy, dosimetric quality, and consistency. Furthermore, 3D-printed molds are reported to take significantly less staff preparation time [4,10].
1538-4721/$ - see front matter Crown Copyright Ó 2019 Published by Elsevier Inc. on behalf of American Brachytherapy Society. All rights reserved. https://doi.org/10.1016/j.brachy.2019.05.006
2
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
In the current work, we explore the feasibility of using optical photogrammetry to improve the efficiency and accuracy of 3D-printingebased HDR brachytherapy surface treatments. Photogrammetry has been previously applied in radiotherapy for patient setup verification tasks [11e 14]. Most of this research pertained to identification of patient external contours to verify patient position or to monitor breathing cycles. Photogrammetry is generally described as a method of ascertaining measurements (e.g., distance) between objects using multiple photographs. In the context of the current work, photogrammetry was used to construct a 3D model of an object using multiple digital photos taken from multiple positions and angles around the said object. Modern photogrammetry software achieves this by analyzing the photos and identifies common features between them. The result is generally a textured (i.e., the 3D model is overlaid with a colored texture map) 3D model of the photographed object. Photogrammetry is used for a variety of applications, including surveying, agriculture, and archeology, and by visual effects artists for movies and television. There are several potential advantages of using photogrammetry for surface mold treatments. Most significantly, in previously published 3D-printed applicator workflows, two CT images are typically required; one to construct the 3D-printed applicator from the external contour of the patient’s skin and another to design a plan in the treatment planning system. In the proposed workflow, one of these two CT images can potentially be replaced with photogrammetry, thus reducing the radiation dose to the patient. The following are other advantages: Avoid metal artifacts present in CT imaging. Because the photogrammetric reconstruction (PR) includes texture information of the patient’s skin, planning target volume (PTV) delineation can be performed by the radiation oncologist on the virtual ‘‘phantom’’ without the need for metal marker wires. However, PR has several limitations in the context of the proposed workflow: No absolute definition of scale in the reconstructed model (must be established indirectly). A requirement for an accurate reconstruction is good ambient (diffuse) lighting conditions during the photography stage. Photogrammetry relies on identification of common features in photos to reconstruct the camera position, which can be error prone under certain conditions. This may produce artifacts in the reconstructed model. One of the major dosimetric considerations of surface mold brachytherapy from the clinical perspective of our center is the conformity of the applicator to the patient’s skin. Unexpected air gaps between the applicator and the patient’s skin have undesirable consequences on the
-
(2019)
-
dosimetric quality of the treatment plan. Air gaps result in an increased effective distance between the source and the PTV surface, which reduces treatment plan quality because it (1) reduces the slope of the dose falloff beyond the PTV (due to inverse square law), thus minimizing the physical benefits of brachytherapy, and (2) limits the ability to shape the prescription isodoses to a curved PTV surface. The quantification of ‘‘air gaps’’ was therefore an important metric in the present study. The following were the primary objectives of this study: Assess the reconstructive accuracy of photogrammetry on an ideal phantom representative of a typical patient treatment site and compare this against CT imaging. Propose and test a workflow for designing 3D-printed surface applicators using photogrammetry (rather than existing CT based workflows). 3D print an applicator prototype and analyze the conformity of the said applicator to the original ideal phantom. This was established through quantification of the maximum displacement between the applicator and phantom and the total volume of air gaps under the applicator. Demonstrate the feasibility of this technique on a realistic treatment site and provide a qualitative assessment in this preliminary study. The quality control, quality assurance, and dosimetric implications of 3D-printed applicators have already been investigated and established in previous studies [6] and are outside the scope of the current work.
2. Methods The current work consists of three major investigative stages. The first stage consists of an investigation of the reconstructive accuracy of the photogrammetry technique compared with CT imaging. The second stage investigates the overall feasibility of developing a 3D-printed surface applicator for brachytherapy from a PR. The third stage explores the feasibility of generating surface mold brachytherapy applicators directly from images of a patient’s treatment site without the need for CT imaging.
2.1. Photogrammetric reconstructionegeometric accuracy The goal of the first stage was to evaluate how accurately photogrammetry could reconstruct a surface target on a patient compared with CT imaging. A plaster cast of an arm from a patient previously treated with brachytherapy was used as the test object to assess the reconstruction accuracy, shown in Fig. 1. A conventional surface mold applicator made for this patient is shown in the same figure. The
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
Fig. 1. Conventional surface mold applicator composed of a white thermoplastic material designed to conform to the shape of the treatment site. The red dental wax is designed to hold the source catheters (which are connected to the high-dose-rate afterloader) in position at regularly spaced intervals relative to the thermoplastic mold. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)
plaster cast is an ‘‘ideal’’ object for this comparison for several reasons: It is made of a material that reflects mostly diffuse light (little specular reflection). Which has the potential to improve the photogrammetric reconstructive accuracy. The object will not move during imaging (compared with a real patient arm), which would potentially produce motion artifacts. Made of a material with approximately uniform density, which allows for a more accurate HU-based segmentation in the treatment planning system from CT images. An accurate segmentation of the patient’s skin surface is essential when designing a treatment applicator in a 3D-printingebased workflow. This segmentation will be used to evaluate the accuracy of the photo-derived reconstruction. One disadvantage of using the plaster cast as an analogue for a real patient arm is the roughness of the surface. Small scale bumps and deformities (which would not be present on a real patient arm) may not be reconstructed accurately using photogrammetry. Thirty 12-megapixel images (3024 4032 pixels) of the plaster cast were taken with an Apple iPhone eight camera. The photos were taken at approximately uniform altitude
-
(2019)
-
3
and azimuth angles (relative to the plaster cast and always above the horizontal) at a distance of approximately 20e 40 cm to image all sides of the plaster cast. Fixed exposure settings were used to ensure accuracy of the reconstruction and uniform lighting of the resultant surface textures. Because absolute scale cannot be defined directly from the photo reconstruction, it was necessary to place another object (of known dimensions) within the photographs. This was used to define absolute scale for the model. This object can be artificially removed from the reconstructed model once the scale has been defined. In the current work, the dimension of the table, upon which the plaster cast was imaged, was circular and had a well-defined diameter of 50 cm. This was used to define the scale of the reconstruction. The photographs were then imported (without additional processing) into the open source photogrammetric software ‘‘Meshroom’’ (version 2018.1.0) [15,16]. The reconstruction process was then performed with the default reconstruction parameters. The photo reconstruction of the plaster cast required approximately 30 min to complete using a computer with an Intel Core i7-7700HQ CPU, 32 GB of RAM, and an NVIDIA GeForce GTX 1060 graphics card (a CUDA enabled graphics card is a system requirement). Once the reconstruction was complete, the 3D model generated by Meshroom was exported as an .OBJ file (a standard 3D model format) with the associated texture maps. The model was imported into the open source 3D modeling and rendering software ‘‘Blender’’ (Blender Foundation, Amsterdam, The Netherlands) [17] for postprocessing. A metric scale in units of millimeters was selected in the preferences. The reconstructed model was scaled uniformly in three dimensions, so that the diameter of the table (in the reconstructed model) matched that of the actual physical dimensions (50 cm diameter). The reconstructed model generated by Meshroom contains a large amount of redundant data including a reconstruction of the surrounding surface, which the plaster cast was resting on (in this case, a table) and other potentially unwanted objects contained within the original photographs (the floor, surrounding walls, etc.). Using Blender, the unwanted vertices and faces from the reconstructed mesh were selected and deleted, leaving only the mesh of the plaster cast. This was done to reduce RAM and storage requirements. The plaster cast was then scanned on an Aquilion LB (Canon Medical Systems) CT scanner using a 120-kVp tube voltage and 1-mm slice thickness (resolution 0.1073 cm 0.1073 cm 0.1 cm). The DICOM images were then imported into the open source software ‘‘Slicer 3D’’ [18]. The plaster cast volume was extracted from the CT image in Slicer 3D using the HU thresholdebased segmentation feature. This volume was considered the ‘‘ground truth’’ volume of the plaster cast for subsequent comparisons.
4
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
The CT-based reconstruction model was then imported into ‘‘Blender’’ alongside the photo-based model for direct volume comparison. The calculated surface area of the CT model was calculated by (1) importing the model into Blender, (2) scaling the volume of the imported model to match the value defined by Slicer 3D, and (3) using the 3D-printing module in Blender to calculate the surface area. The surface area and volume of the photo-based reconstruction were calculated directly in Blender. The volume and surface area for each reconstruction were used as a metric for the similarity of the models. The CT-reconstructed (CTR) and photogrammetryreconstructed meshes were then imported into the open source package ‘‘MeshLab’’[19,20]. The meshes were automatically aligned in 3D space using the ‘‘align’’ feature. The Hausdorff distance was then calculated between the two different reconstructed models as a more robust method of comparing their similarity. The Hausdorff distance [21] (or metric) is a method of quantifying the similarity between two arbitrary meshes. It measures the degree to which each point/vertex in an arbitrary mesh corresponds to points/vertices of another mesh. The Hausdorff distance was sampled at 116,886 vertex positions (corresponding to the number of vertices of the CTR mesh). The vertices corresponding to the bottom surface of both models were excluded from the analysis. This was due to the photogrammetry method not being able to accurately reconstruct the bottom surface of an object (i.e., the surface in contact with the table). Photogrammetry relies on the camera being able to visually ‘‘see’’ what is being reconstructed. The Hausdorff distance data were exported to Excel and analyzed. The percentage of points in the photogrammetryderived model with a matching point in the CT-derived model within the spatial resolution of the CT scanner was obtained.
-
(2019)
-
Fig. 2. An axial slice of the plaster cast and 3D-printed arm applicator. The high density plaster cast is shown in white, the 3D-printed applicator is anterior to the plaster cast, and the yellow contour represents any detected air volumes within the red volume of interest. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)
The selected faces were then duplicated from the surface of the models and extruded vertically by 5 mm in the direction of the surface normals. This resulted in applicators with a uniform thickness of 5 mm. This distance is fully customizable and should be chosen to obtain an appropriate dose falloff from the HDR brachytherapy source and the most uniform dose distribution over the PTV region. The 3D models of the virtual surface mold applicators were exported as an .stl file (in metric units with a scale
3D-printed applicator from photogrammetry The second stage of the current work consisted of designing and 3D printing virtual surface mold applicators from the PR and CTR models. The PR and CTR models of the plaster cast were edited in Blender to produce two ‘‘virtual’’ surface mold applicators, which could be 3D printed. This was done to compare the accuracy of the traditional 3D-printing workflow with the proposed photogrammetry workflow. This was achieved by selecting a series of faces on the top surface of the plaster cast models in Blender, which encompassed a hypothetical PTV region plus a margin within which to place catheters for the HDR brachytherapy source. The region of the plaster cast with the greatest variation in curvature was selected to convincingly verify the overall process. This region also had several medium scale ‘‘bumps’’ with which to verify the reconstructive accuracy.
Fig. 3. The test treatment site (author’s right hand) placed on the calibration template (designed in the current work). Black pen marks on the hand indicate approximate location of treatment site. The calibration template contains a rectangular outline (black line) to define the scale of the 3D model and colored circles to assist with the photogrammetric reconstruction.
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
-
(2019)
-
5
Fig. 4. Left: A photograph taken of the plaster cast of the patient’s right arm. Right: A computer-generated (CG) rendering of the textured photogrammetric reconstruction (PR) model of the plaster cast.
of millimeters). The surface mold applicators were then 3D printed using acrylonitrile butadiene styrene using a Zortrax (Olsztyn, Poland) M200 3D printer. To verify the overall accuracy of the PR (and CTR) workflows and the resultant 3D-printed surface mold applicators, the 3D-printed applicator prototypes and plaster cast were CT scanned together. The CT image was imported into Varian (Palo Alto, CA) BrachyVision. The ruler tool proved unreliable in visually estimating the displacement of the applicator from the plaster cast as the windowing level can be arbitrarily defined. To objectively and quantitatively estimate the position and volume of any air gaps (indicating a less than ideal conformity of the applicator to the plaster cast), the air gaps were automatically segmented in BrachyVision. This was achieved using the following steps: The plaster cast was automatically segmented using the HU-based segmentation tool with an intensity range of 800e2000 HU. The 3D-printed applicators were segmented using the same tool but with an intensity range of 300 to 300 HU. Any air gaps beneath the applicators were detected using the HU-based segmenter using an intensity range of 1000 to 500 HU. The volume was limited using the volume of interest tool to cover the entire
applicator, the volume beneath the applicator, and the top surface of the plaster cast (as shown in Fig. 2). An air gap has been identified medially and anterior to the top surface of the plaster cast in the same figure. This region is shown in yellow. The 2D brush tool was then used to manually remove any contoured region (in the air gap contour) that lay above the 3D-printed applicator volume. The remaining volume is representative of any air volume between the applicator and plaster cast. This estimate of the air volume is limited by the spatial resolution of the CT scanner. The maximum vertical displacement of the applicator from the plaster cast was estimated using the ruler tool and measuring the maximum vertical extent of the contoured air gaps (thus eliminating the subjectivity of the measurement caused by an arbitrarily defined windowing level).
2.3. Virtual applicator design from photogrammetry of a patient In the final stage of the current work, the feasibility of using photogrammetry to design a surface mold applicator directly from images of the treatment site on a patient was
Fig. 5. Comparison of CT- and photo-based reconstruction of the plaster cast. Left (in each image)dphoto reconstruction. Right (in each image)dCT reconstruction.
6
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
Table 1 Comparison of the computed volume and surface areas for the reconstructed model and the CT of the plaster cast Reconstruction method
Volume (cm3)
Area (cm2)
Photogrammetry CT
536.9 537.1
536.3 522.2
Difference
L0.1%
D2.6%
investigated. In the current work, the author’s right hand was selected as the treatment site. First, a theoretical PTV region was delineated on the test subject’s hand using a marker pen, shown in Fig. 3. Twenty still images were taken at approximately uniformly spaced altitude and azimuth angles (relative to the origin of the target volume) at a distance of approximately 20e40 cm. Figure 3 also shows a calibration template containing a rectangular outline (black line) to define the scale of the 3D model and colored circles to assist with the photogrammetric reconstruction. The still images were then processed in Meshroom using a similar method to that outlined in Section 3D Printed Applicator from Photogrammetry. To design a virtual brachytherapy applicator, the PTV region on the PR model plus a margin suitable for attaching brachytherapy catheters was contoured and duplicated from the original model. The faces of this duplicated region were extruded by 6 mm in a direction normal to each face, so that the virtual applicator had a uniform thickness. Five cylindrical volumes of diameter 2 mm (6 French) with approximately equal lateral spacing were then extruded from the applicator. This allowed for five 6-French catheters to be
-
(2019)
-
inserted into the 3D-printed applicator and then connected to the afterloader during treatment. The 3D model of the surface mold applicator was exported as an .stl file (in metric units with a scale of millimeters). The surface mold applicator was once again 3D printed using acrylonitrile butadiene styrene using a Zortrax M200 3D printer. The accuracy of the reconstructed applicator was evaluated qualitatively by visual inspection. In the current work, the accuracy of the 3D-printed hand applicator could not be assessed quantitatively on CT because of the ethical requirements. As such, a qualitative assessment is described in the current work as a preliminary demonstration of the feasibility of this technique.
3. Results 3.1. Photogrammetric reconstructionegeometric accuracy Figure 4 shows a photo and a rendering of the reconstructed model of the plaster cast. A visual inspection of the two models side by side shows excellent agreement with only small variations in the small scale detail. A side-by-side comparison of the ‘‘scaled’’ photo (without textures) and CTR plaster casts are shown in Fig. 5. The model shown on the left in each subimage is the photogrammetry reconstructed (PR) model, and the CTR model is shown on the right. Although the overall dimensions of the two models are almost identical, the photoreconstructed model shows more of the small scale detail (compared with the smooth topology of the CTR model).
Fig. 6. Hausdorff distance. Percentage of matching points between CT and photogrammetry models vs. distance to matching vertex (mm).
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
Fig. 7. 3D visualization of the Hausdorff distance metric for the plaster cast phantom. Red and orange vertices indicate a high level of similarity, whereas blue and green vertices indicate a low level of similarity. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)
The ‘‘true’’ volume of the plaster cast was calculated from an HU-based segmentation in the CT reconstruction using Slicer 3D to be 537.1 cm3. The CT-based segmentation of the plaster cast was imported into Blender, scaled so that the volume matched the ‘‘true volume’’ and the surface area was calculated to be 522.2 cm2. When the postprocessed PR model was scaled to match the physical dimensions of the reference object, the volume was calculated to be 536.9 cm3 and the surface area was 536.3 cm2. These results are summarized in Table 1. The difference in volume between the CT- and photobased reconstruction was approximately 0.1%. The difference in surface area was 2.6%. This is not an unexpected result. A visual inspection of the PR mesh shows a much higher degree of complexity (in terms of visibility of small scale details) compared with the CTR mesh, shown in Fig. 5. This could occur for several reasons including (1) a limitation of the HU-based segmentation algorithm in
-
(2019)
-
7
Slicer, which ‘‘smooths’’ the CT mesh which would reduce the surface area, (2) the limited spatial resolution of the CT scanner compared with photogrammetry, or (3) the reconstruction technique produced by Meshroom creates detail in the model, which increases the surface area of the mesh. It is not clear how much of this extra detail is ‘‘real,’’ but it is apparent that using this method, the photo reconstruction is able to sample significantly more small-scale detail than the CT image. This results in an apparent higher resolution using the photo reconstruction method. The results of the Hausdorff distance calculation are graphed in Fig. 6. The graph shows the percentage of vertices in the CT-obtained model with a corresponding vertex in the photogrammetry model vs. the distance between these correspondences in units of millimeters. The results indicate that when the bottom surface of both meshes is excluded from the analysis (due to a limitation of the photogrammetry method), 93% of the vertices in the CT model have a corresponding vertex in the photogrammetry model within 1 mm (the slice thickness of the CT scan). This indicates that the CT- and photogrammetry-obtained models of the plaster cast are spatially similar. This is visualized in Fig. 7. This result indicates that a high level of agreement was obtained between CT and photogrammetry on the anterior surface of the plaster cast. Poorer agreement was achieved on the superior and inferior surfaces of the phantom. 3.2. 3D-printed applicator from photogrammetry An approximately square projection (when viewed from above) of faces was selected on the PR (and CTR) plaster cast model and extruded by 5 mm to generate the ‘‘virtual’’ surface mold applicator shown in Fig. 8. The 3D-printed versions of the photogrammetry and CT-based applicators are shown in Figs. 9 and 10, respectively. The 3D-printed surface applicators were positioned on the plaster cast using measurements from the virtual
Fig. 8. Virtual surface mold applicator extruded from the photo-reconstructed plaster cast of the patient’s right arm.
8
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
-
(2019)
-
Fig. 9. 3D-printed surface mold applicator generated from the photogrammetry technique.
applicator models. CT scans were then acquired of the plaster cast together with the individual applicators to assess the overall accuracy of the PR technique compared with the CT-based method. Figure 11 shows a transverse and sagittal CT slice of the photogrammetry- (left) and CT-based (right) applicators positioned on the plaster cast. Using Varian (Palo Alto, CA) BrachyVision, two small ‘‘air gaps’’ were identified (shown in Fig. 11) between the bottom surface of the photogrammetry applicator and the plaster cast. Using the ‘‘ruler’’ tool in BrachyVision, the maximum vertical extent of these air gaps were approximately 0.9 mm and 1.2 mm. The total volume of all air gaps detected beneath the photogrammetry-derived applicator was measured to be 0.05 cm3. All other regions of the applicator were in contact with the plaster cast to within the spatial resolution of the CT scanner (resolution 0.1073 cm 0.1073 cm 0.1 cm). The CT-based 3D-printed applicator showed significantly poorer conformity to the plaster cast surface than the photogrammetry-derived applicator. The total volume of the air gaps detected beneath this applicator was measured to be at least 0.3 cm3 (This volume was
calculated based on repeated segmentations of two different CT image acquisitions and repeated applicator positioning on the plaster cast [to minimize uncertainties due to setup]. The volume presented here is the lowest of the air gap volumes recorded.). In the interest of fair comparison, the surface area of the CT-based 3D-printed applicator was 143 cm2, whereas the applicator printed using photogrammetry was 129 cm2. Thus, assuming both methods produced equally conformal applicators, the volume beneath the CT-derived applicator should only be 11% (ratio of applicator surface areas) more than the photogrammetryderived applicator. However, in this case, the volume was larger by a factor of approximately 6. The largest ‘‘air gap’’ detected beneath the CT-derived applicator (shown in the top right corner of Fig. 11) had a maximum vertical extent of 2 mm. These results demonstrate the potential increase in reconstruction accuracy obtained using photogrammetry compared with CT for 3D-printed surface mold applicators. However, there are several factors that may have led to the reduced accuracy of the CT-derived applicator (compared with photogrammetry). First, the resolution of
Fig. 10. 3D-printed surface mold applicator generated from CT acquisition.
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
-
(2019)
-
9
Fig. 11. Visual comparison of air gaps beneath photogrammetry- (left) and CT-derived (right) surface mold applicators. Air gaps beneath the applicators were detected using Brachyvision’s HU thresholding tool and shown in cyan. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)
the CT-derived applicator was limited by the reconstructed resolution of the CT image. The resolution of the CT images used to generate the applicator was 1.073 mm 1.073 mm 1 mm. This relatively low resolution (compared with photogrammetry) resulted in a comparably ‘‘smoothed’’ (spatially averaged) applicator, which did not conform as well to the plaster surface. Second, the topology of the 3D-printed applicator was limited by the HU-based segmentation algorithm used to define the surface of the plaster cast. The segmentation process relies on an HU threshold being specified to delineate the plaster cast from the air surrounding it.
This volume of air gaps could be potentially minimized (and resultant conformity of the applicator to the patient’s skin increased) for both methods of reconstruction by 3D printing the applicators using a semiflexible material. Virtual applicator design from photogrammetry of a patient Figure 12 shows the 3D rendering of the virtual applicator, and Fig. 13 shows the printed applicator placed on the hand of our test subject. Without acquiring a CT image of the applicator on the treatment site, quantitative
Fig. 12. 3D renderings showing the process of producing a custom surface mold applicator for brachytherapy treatments. Top left: Contour the required faces on the 3D model of the treatment area. Top right and bottom: Duplicate the selected faces and extrude by the required thickness (as specified by dosimetric requirements). Bottom: Extrude 2-mm- (6-French) diameter inserts from the applicator to insert catheters.
10
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
-
(2019)
-
accuracy and efficiency of 3D-printed applicator workflows [6,10]. As these workflows typically require 2 separate CT scans of a patient, one key advantage of the proposed technique is the reduction in patient radiation dose. The reason for this is the CT image used to design the 3D applicator can be accomplished with photogrammetry. Second, as photogrammetry can generate a textured surface of the patient’s skin during the reconstruction, accurate PTV delineation can be accomplished without the use of marker wires during the planning CT. There are several sources of uncertainty and limitations in the photogrammetry process, which will need further investigation if this were to be used clinically. These include the following. Fig. 13. Photos of the 3D-printed surface mold applicator. The right and bottom left subfigures show a 6-French interstitial catheter inserted into the central extruded cylinder.
measurements of the accuracy of the reconstructed applicator were not possible. However, the 3D-printed applicator appeared to visually conform well to the test treatment site. Owing to the small size and low mass of the 3D-printed applicator, it was observed that there was not enough weight to hold the applicator in place. This could be corrected in future by printing extra bolus on top of the applicator or simply applying a separate bolus material during treatment. It was also found that the filament-style 3D printer could not print the cylindrical inserts with sufficient accuracy to allow the treatment catheters to be inserted the entire length of the cylinder. In the prototype, this was corrected by pushing a 2-mm-diameter drill piece to remove the extraneous plastic. In a clinical setting, this may not be practical as the catheter inserts are curved in general. Increasing the extruded cylinder diameter by a small offset (e.g., 0.2 mm) during the modeling stage may partially correct this problem. This limitation and correction have been previously reported in the study by Clarke [6]. The approximate time to complete the individual steps in this workflow are summarized in Table 2.
4. Discussion Photogrammetry has been demonstrated as a potential alternative to CT in the generation of 3D-printed surface mold applicators and has the potential to improve the Table 2 Approximate times to complete the individual steps required to manufacture a 3D-printed surface mold applicator using photogrammetry Procedure Photographing treatment site (approx. 30 photos) Photogrammetric reconstruction (with default parameters and using desktop PC) Applicator design in Blender 3D printing
Approximate time (minutes) 5 35 45 240
Defining scale in the reconstructed model The 3D models created using photogrammetry have arbitrary scale. To overcome this issue, a reference object of known length was placed alongside the object being ‘‘scanned’’ to define the scale of the scene. It has been demonstrated by the similarity between the volume and surface area measurements (in the results section), as well as the accuracy of the 3D-printed applicators, that the proposed workflow is technically feasible. However, for treatment sites with more complex curvature, higher levels of scaling accuracy may be required. Image acquisition During the experimentation process, it was discovered that, for optimal results from the reconstruction process, photos must be taken in a well-lit room with diffuse lighting. In the future, a dedicated imaging unit including a mounted camera and lighting may be required for optimal results. The effect of the distance from the object to the camera was not assessed in the current work. Previous works have investigated [22] the effect of imaging distance on photogrammetry point cloud generation and reported the error in reconstructive accuracy increased slowly up to a distance of 20 m from the object. However, the test object used in this study was significantly larger than the object used in the current work and contained high-contrast coded targets to assist with reconstruction. Fundamentally, the ability to resolve two closely spaced features on the test object is inversely proportional to the distance between the test object and camera as specified by the Rayleigh criterion. Hence, photos should be taken as close as possible to the object of interest while ensuring the entire object remains within the frame of the photo. Future work should include a comprehensive analysis and optimization of imaging parameters. Model complexity The Meshroom software used in the current work is capable of creating models of very high resolution. In the
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
examples presented in the current work, the reconstructed models contained in excess of 500,000 vertices. These models required between 30 and 45 min to reconstruct. The potentially unnecessary numbers of vertices made the reconstruction and postprocessing stage slower. Reduction in the number of vertices was performed in the current work using Blender’s ‘‘decimate function,’’ which reduced the number of vertices (in both examples) to approximately 100,000 without significant changes in the model’s topology. Future work could include an investigation of the optimal settings to use in the reconstruction process (including the optimal number of photos and camera to target distance) to obtain an acceptably precise model for brachytherapy surface applicators while reducing the reconstruction and postprocessing times. Automation
-
(2019)
-
11
Although both techniques demonstrated sufficient applicator conformity for 3D-printed brachytherapy applicators, the current work has demonstrated the potential gains in accuracy obtained using the photogrammetry technique. The new workflow was subsequently used to 3D print a prototype of a brachytherapy surface mold applicator from a hypothetical patient’s right hand. Brachytherapy catheters were incorporated into the 3D-printed applicator demonstrating the potential for an accurate and efficient treatment workflow. In future work, we intend to compare and contrast the brachytherapy treatment plans achievable using this technique with our traditional surface mold applicator workflow.
Acknowledgments
Although the individual steps in the current workflow are generally fast, shown in Table 2, the entire process could potentially be scripted to improve efficiency. This has been previously demonstrated in the study by Clarke [6].
The authors thank Mr. Morgan Hunter from ThincLab, University of Adelaide, for providing the 3D printing services and advice.
Conclusions
References
Optical photogrammetry has been applied to the task of HDR brachytherapy surface mold applicator design in the current work using open source software. To our knowledge, this is the first example of the use of optical photogrammetry for HDR brachytherapy surface applicator design. A comparison of the 3D phantom models generated using photogrammetry and conventional CT-based reconstruction has verified the geometric accuracy of the photogrammetry technique. The reconstructed model obtained using photogrammetry had a measured volume and surface area that were within 0.1% and 2.6% of the CTobtained values, respectively. The models were subsequently compared using the Hausdorff distance metric. Using this technique, it was determined that 93% of the visible vertices present in the CT-derived model had a matching vertex on the photogrammetry-derived model within 1 mm (the slice thickness of the CT scanner), indicating a high level of similarity. Surface mold applicator prototypes were then designed and 3D printed directly from the surface of the CT and optically reconstructed phantoms. In the case of the photogrammetry-derived applicator, a CT scan identified two small air gaps between the applicator and the phantom with a total volume of 0.05 cm3. In addition, the maximum displacement between the anterior surface of the phantom and posterior surface of the applicator was measured to be 1.2 mm. The same analysis was subsequently performed on a conventional CT-derived 3D-printed applicator. The total volume of air gaps beneath this applicator was measured to be 0.3 cm3 with a maximum vertical displacement of 2 mm.
[1] Kuncman q, Koz1owski S, Pietraszek A, et al. Highly conformal CT based surface mould brachytherapy for non-melanoma skin cancers of earlobe and nose. J Contemp Brachytherapy 2016;8:195e200. [2] Jones E-L, Tonino Baldion A, Thomas C, et al. Introduction of novel 3D-printed superficial applicators for high-dose-rate skin brachytherapy. Brachytherapy 2017;16:409e414. [3] Guinot JL, Rembielak A, Perez-Calatayud J, et al. GEC-ESTRO ACROP recommendations in skin brachytherapy. Radiother Oncol 2018;126:377e385. [4] Schumacher M, Lasso A, Cumming I, et al. 3D-printed surface mould applicator for high-dose-rate brachytherapy. In: SPIE medical imaging. Orlando, FL: SPIE; 2015. [5] Zhao Y, Moran K, Yewondwossen M, et al. Clinical applications of 3-dimensional printing in radiation therapy. Med Dosim 2017;42: 150e155. [6] Clarke SA. 3D printed surface applicators for high dose rate brachytherapy. Halifax, Nova Scotia: Dalhousie University; 2016. [7] Tyler LF, et al. Clinical implementation of 3D printing for brachytherapy: techniques and emerging applications. Brachytherapy 2016;15:166. [8] Ricotti R, Vavassori A, Bazani A, et al. 3D-printed applicators for high dose rate brachytherapy: dosimetric assessment at different infill percentage. Phys Med 2016;32:1698e1706. [9] Harris B, Nilsson S, Poole C. A feasibility study for using ABS plastic and a low-cost 3D printer for patient-specific brachytherapy mould design. Australas Phys Eng Sci Med 2015;38:399e412. [10] Arenas M, Sabater S, Sintas A, et al. Individualized 3D scanning and printing for non-melanoma skin cancer brachytherapy: a financial study for its integration into clinical workflow. J Contemp Brachytherapy 2017;9:270e276. [11] Rogus RD, Stern RL, Kubo HD. Accuracy of a photogrammetrybased patient positioning and monitoring system for radiation therapy. Med Phys 1999;26:721e728. [12] Calow R, et al. Photogrammetric measurement of patients in radiotherapy. ISPRS J Photogramm Remote Sens 2002;56:347e359. [13] Deantonio L, Masini L, Loi G, et al. Detection of setup uncertainties with 3D surface registration system for conformal radiotherapy of breast cancer. Rep Pract Oncol Radiother 2011;16:77e81.
12
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
[14] Liu W, Cheung Y, Sabouri P, et al. A continuous surface reconstruction method on point cloud captured from a 3D surface photogrammetry system. Med Phys 2015;42:6564e6571. [15] Moulon P, Monasse P, Marlet R. Adaptive structure from motion with a contrario model estimation. Berlin, Heidelberg: Springer Berlin Heidelberg; 2013. [16] Jancosek M, Pajdla T. Multi-view reconstruction preserving weaklysupported surfaces. CVPR 2011;2011. [17] Blender Online Community, Blender-a 3D modelling and rendering package. Blender Institute. Amsterdam: Blender Foundation. [18] Kikinis R, Pieper SD, Vosburgh KG. 3D slicer: a platform for subjectspecific image analysis, visualization, and clinical support intraoperative imaging and image-guided therapy. New York, NY: Springer; 2014.
-
(2019)
-
[19] Cignoni P, Callieri M, Corsini M, et al. MeshLab: an opensource mesh processing tool. In: Eurographics Italian chapter conference. Geneva, Switzerland: The Eurographics Association; 2008. [20] Cignoni P, Rocchini C, Scopigno R. Metro: measuring error on simplified surfaces. Computer Graphics Forum 1998;17:167e 174. [21] Zhang D, He F, Han S, Zou L, Wu Y, Chen Y. An efficient approach to directly compute the exact Hausdorff distance for 3D point sets. Integrated Comput Aid Eng 2017;24:261e277. [22] An Z. Accuracy assessment of 3D point cloud generated by photogrammetry from different distances. Houghton, MI: Michigan Technological University; 2017.
-
(2019)
-
Application of optical photogrammetry in radiation oncology: HDR surface mold brachytherapy Michael J.J. Douglass*, Alexandre M. Carac¸a Santos School of Physical Sciences, University of Adelaide, South Australia, Australia Department of Medical Physics, Royal Adelaide Hospital, South Australia, Australia
ABSTRACT
PURPOSE: We propose a novel method of designing surface mold brachytherapy applicators using optical photogrammetry. The accuracy of this technique for the purpose of 3D-printing surface mold brachytherapy applicators is investigated. METHODS AND MATERIALS: Photogrammetry was used to generate a 3D model of a patient’s right arm. The geometric accuracy of the model was evaluated against CT in terms of volume, surface area, and the Hausdorff distance. A surface mold applicator was then 3D printed using this reconstructed model. The accuracy was evaluated by analyzing the displacement and air-gap volumes between the applicator and plaster cast on a CT image. This technique was subsequently applied to generate a 3D-printed applicator of the author’s hand directly, as a proof of principle, using only photographic images. RESULTS: The volume and surface area of the model were within 0.1% and 2.6% of the CTobtained values, respectively. Using the Hausdorff distance metric, it was determined that 93% of the visible vertices present in the CT-derived model had a matching vertex on the photogrammetry-derived model within 1 mm, indicating a high level of similarity. The maximum displacement between the plaster cast of the patient’s arm and the photo-derived 3D-printed applicator was 1.2 mm with a total air-gap volume of approximately 0.05 cm3. CONCLUSIONS: Photogrammetry has been applied to the task of generating 3D-printed brachytherapy surface mold applicators. The current work demonstrates the feasibility and accuracy of this technique and how it may be incorporated into a 3D-printing brachytherapy workflow. Crown Copyright Ó 2019 Published by Elsevier Inc. on behalf of American Brachytherapy Society. All rights reserved.
Keywords:
Photogrammetry; Brachytherapy; Surface mold; Blender; Slicer 3D; AliceVision; Bolus
1. Introduction High-dose-rate (HDR) surface mold brachytherapy is a radiotherapy modality used in the treatment of skin lesions. Owing to the rapid dose falloff of HDR sources, customized surface molds have been shown to produce highly conformal dose distributions on irregular surfaces [1].
Received 25 February 2019; received in revised form 23 April 2019; accepted 22 May 2019. Conflict of interest: The authors have no conflicts of interest to disclose. Financial disclosure: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. * Corresponding author. School of Physical Sciences, University of Adelaide, SA 5005, Australia. E-mail address: [email protected] (M.J.J. Douglass).
The major steps involved in developing a surface-type brachytherapy treatment in our center have been adopted from the GEC-ESTRO guidelines [2,3]. This procedure typically requires as many as three staff (radiation therapists and physicists) at any given time. The total preparation time for each customized surface mold in our hospital typically takes less than five staff hours with more complex sites taking up to seven staff hours to complete. This process includes construction of the plaster cast, construction of the wax mold, and placement of the catheters. Recent reports have evaluated the use of plastic 3Dprinted brachytherapy applicators from a CT image of the patient’s treatment site [2,4e9]. These studies found that the 3D-printed molds outperformed manual wax applicators in terms of dosimetric accuracy, dosimetric quality, and consistency. Furthermore, 3D-printed molds are reported to take significantly less staff preparation time [4,10].
1538-4721/$ - see front matter Crown Copyright Ó 2019 Published by Elsevier Inc. on behalf of American Brachytherapy Society. All rights reserved. https://doi.org/10.1016/j.brachy.2019.05.006
2
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
In the current work, we explore the feasibility of using optical photogrammetry to improve the efficiency and accuracy of 3D-printingebased HDR brachytherapy surface treatments. Photogrammetry has been previously applied in radiotherapy for patient setup verification tasks [11e 14]. Most of this research pertained to identification of patient external contours to verify patient position or to monitor breathing cycles. Photogrammetry is generally described as a method of ascertaining measurements (e.g., distance) between objects using multiple photographs. In the context of the current work, photogrammetry was used to construct a 3D model of an object using multiple digital photos taken from multiple positions and angles around the said object. Modern photogrammetry software achieves this by analyzing the photos and identifies common features between them. The result is generally a textured (i.e., the 3D model is overlaid with a colored texture map) 3D model of the photographed object. Photogrammetry is used for a variety of applications, including surveying, agriculture, and archeology, and by visual effects artists for movies and television. There are several potential advantages of using photogrammetry for surface mold treatments. Most significantly, in previously published 3D-printed applicator workflows, two CT images are typically required; one to construct the 3D-printed applicator from the external contour of the patient’s skin and another to design a plan in the treatment planning system. In the proposed workflow, one of these two CT images can potentially be replaced with photogrammetry, thus reducing the radiation dose to the patient. The following are other advantages: Avoid metal artifacts present in CT imaging. Because the photogrammetric reconstruction (PR) includes texture information of the patient’s skin, planning target volume (PTV) delineation can be performed by the radiation oncologist on the virtual ‘‘phantom’’ without the need for metal marker wires. However, PR has several limitations in the context of the proposed workflow: No absolute definition of scale in the reconstructed model (must be established indirectly). A requirement for an accurate reconstruction is good ambient (diffuse) lighting conditions during the photography stage. Photogrammetry relies on identification of common features in photos to reconstruct the camera position, which can be error prone under certain conditions. This may produce artifacts in the reconstructed model. One of the major dosimetric considerations of surface mold brachytherapy from the clinical perspective of our center is the conformity of the applicator to the patient’s skin. Unexpected air gaps between the applicator and the patient’s skin have undesirable consequences on the
-
(2019)
-
dosimetric quality of the treatment plan. Air gaps result in an increased effective distance between the source and the PTV surface, which reduces treatment plan quality because it (1) reduces the slope of the dose falloff beyond the PTV (due to inverse square law), thus minimizing the physical benefits of brachytherapy, and (2) limits the ability to shape the prescription isodoses to a curved PTV surface. The quantification of ‘‘air gaps’’ was therefore an important metric in the present study. The following were the primary objectives of this study: Assess the reconstructive accuracy of photogrammetry on an ideal phantom representative of a typical patient treatment site and compare this against CT imaging. Propose and test a workflow for designing 3D-printed surface applicators using photogrammetry (rather than existing CT based workflows). 3D print an applicator prototype and analyze the conformity of the said applicator to the original ideal phantom. This was established through quantification of the maximum displacement between the applicator and phantom and the total volume of air gaps under the applicator. Demonstrate the feasibility of this technique on a realistic treatment site and provide a qualitative assessment in this preliminary study. The quality control, quality assurance, and dosimetric implications of 3D-printed applicators have already been investigated and established in previous studies [6] and are outside the scope of the current work.
2. Methods The current work consists of three major investigative stages. The first stage consists of an investigation of the reconstructive accuracy of the photogrammetry technique compared with CT imaging. The second stage investigates the overall feasibility of developing a 3D-printed surface applicator for brachytherapy from a PR. The third stage explores the feasibility of generating surface mold brachytherapy applicators directly from images of a patient’s treatment site without the need for CT imaging.
2.1. Photogrammetric reconstructionegeometric accuracy The goal of the first stage was to evaluate how accurately photogrammetry could reconstruct a surface target on a patient compared with CT imaging. A plaster cast of an arm from a patient previously treated with brachytherapy was used as the test object to assess the reconstruction accuracy, shown in Fig. 1. A conventional surface mold applicator made for this patient is shown in the same figure. The
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
Fig. 1. Conventional surface mold applicator composed of a white thermoplastic material designed to conform to the shape of the treatment site. The red dental wax is designed to hold the source catheters (which are connected to the high-dose-rate afterloader) in position at regularly spaced intervals relative to the thermoplastic mold. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)
plaster cast is an ‘‘ideal’’ object for this comparison for several reasons: It is made of a material that reflects mostly diffuse light (little specular reflection). Which has the potential to improve the photogrammetric reconstructive accuracy. The object will not move during imaging (compared with a real patient arm), which would potentially produce motion artifacts. Made of a material with approximately uniform density, which allows for a more accurate HU-based segmentation in the treatment planning system from CT images. An accurate segmentation of the patient’s skin surface is essential when designing a treatment applicator in a 3D-printingebased workflow. This segmentation will be used to evaluate the accuracy of the photo-derived reconstruction. One disadvantage of using the plaster cast as an analogue for a real patient arm is the roughness of the surface. Small scale bumps and deformities (which would not be present on a real patient arm) may not be reconstructed accurately using photogrammetry. Thirty 12-megapixel images (3024 4032 pixels) of the plaster cast were taken with an Apple iPhone eight camera. The photos were taken at approximately uniform altitude
-
(2019)
-
3
and azimuth angles (relative to the plaster cast and always above the horizontal) at a distance of approximately 20e 40 cm to image all sides of the plaster cast. Fixed exposure settings were used to ensure accuracy of the reconstruction and uniform lighting of the resultant surface textures. Because absolute scale cannot be defined directly from the photo reconstruction, it was necessary to place another object (of known dimensions) within the photographs. This was used to define absolute scale for the model. This object can be artificially removed from the reconstructed model once the scale has been defined. In the current work, the dimension of the table, upon which the plaster cast was imaged, was circular and had a well-defined diameter of 50 cm. This was used to define the scale of the reconstruction. The photographs were then imported (without additional processing) into the open source photogrammetric software ‘‘Meshroom’’ (version 2018.1.0) [15,16]. The reconstruction process was then performed with the default reconstruction parameters. The photo reconstruction of the plaster cast required approximately 30 min to complete using a computer with an Intel Core i7-7700HQ CPU, 32 GB of RAM, and an NVIDIA GeForce GTX 1060 graphics card (a CUDA enabled graphics card is a system requirement). Once the reconstruction was complete, the 3D model generated by Meshroom was exported as an .OBJ file (a standard 3D model format) with the associated texture maps. The model was imported into the open source 3D modeling and rendering software ‘‘Blender’’ (Blender Foundation, Amsterdam, The Netherlands) [17] for postprocessing. A metric scale in units of millimeters was selected in the preferences. The reconstructed model was scaled uniformly in three dimensions, so that the diameter of the table (in the reconstructed model) matched that of the actual physical dimensions (50 cm diameter). The reconstructed model generated by Meshroom contains a large amount of redundant data including a reconstruction of the surrounding surface, which the plaster cast was resting on (in this case, a table) and other potentially unwanted objects contained within the original photographs (the floor, surrounding walls, etc.). Using Blender, the unwanted vertices and faces from the reconstructed mesh were selected and deleted, leaving only the mesh of the plaster cast. This was done to reduce RAM and storage requirements. The plaster cast was then scanned on an Aquilion LB (Canon Medical Systems) CT scanner using a 120-kVp tube voltage and 1-mm slice thickness (resolution 0.1073 cm 0.1073 cm 0.1 cm). The DICOM images were then imported into the open source software ‘‘Slicer 3D’’ [18]. The plaster cast volume was extracted from the CT image in Slicer 3D using the HU thresholdebased segmentation feature. This volume was considered the ‘‘ground truth’’ volume of the plaster cast for subsequent comparisons.
4
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
The CT-based reconstruction model was then imported into ‘‘Blender’’ alongside the photo-based model for direct volume comparison. The calculated surface area of the CT model was calculated by (1) importing the model into Blender, (2) scaling the volume of the imported model to match the value defined by Slicer 3D, and (3) using the 3D-printing module in Blender to calculate the surface area. The surface area and volume of the photo-based reconstruction were calculated directly in Blender. The volume and surface area for each reconstruction were used as a metric for the similarity of the models. The CT-reconstructed (CTR) and photogrammetryreconstructed meshes were then imported into the open source package ‘‘MeshLab’’[19,20]. The meshes were automatically aligned in 3D space using the ‘‘align’’ feature. The Hausdorff distance was then calculated between the two different reconstructed models as a more robust method of comparing their similarity. The Hausdorff distance [21] (or metric) is a method of quantifying the similarity between two arbitrary meshes. It measures the degree to which each point/vertex in an arbitrary mesh corresponds to points/vertices of another mesh. The Hausdorff distance was sampled at 116,886 vertex positions (corresponding to the number of vertices of the CTR mesh). The vertices corresponding to the bottom surface of both models were excluded from the analysis. This was due to the photogrammetry method not being able to accurately reconstruct the bottom surface of an object (i.e., the surface in contact with the table). Photogrammetry relies on the camera being able to visually ‘‘see’’ what is being reconstructed. The Hausdorff distance data were exported to Excel and analyzed. The percentage of points in the photogrammetryderived model with a matching point in the CT-derived model within the spatial resolution of the CT scanner was obtained.
-
(2019)
-
Fig. 2. An axial slice of the plaster cast and 3D-printed arm applicator. The high density plaster cast is shown in white, the 3D-printed applicator is anterior to the plaster cast, and the yellow contour represents any detected air volumes within the red volume of interest. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)
The selected faces were then duplicated from the surface of the models and extruded vertically by 5 mm in the direction of the surface normals. This resulted in applicators with a uniform thickness of 5 mm. This distance is fully customizable and should be chosen to obtain an appropriate dose falloff from the HDR brachytherapy source and the most uniform dose distribution over the PTV region. The 3D models of the virtual surface mold applicators were exported as an .stl file (in metric units with a scale
3D-printed applicator from photogrammetry The second stage of the current work consisted of designing and 3D printing virtual surface mold applicators from the PR and CTR models. The PR and CTR models of the plaster cast were edited in Blender to produce two ‘‘virtual’’ surface mold applicators, which could be 3D printed. This was done to compare the accuracy of the traditional 3D-printing workflow with the proposed photogrammetry workflow. This was achieved by selecting a series of faces on the top surface of the plaster cast models in Blender, which encompassed a hypothetical PTV region plus a margin within which to place catheters for the HDR brachytherapy source. The region of the plaster cast with the greatest variation in curvature was selected to convincingly verify the overall process. This region also had several medium scale ‘‘bumps’’ with which to verify the reconstructive accuracy.
Fig. 3. The test treatment site (author’s right hand) placed on the calibration template (designed in the current work). Black pen marks on the hand indicate approximate location of treatment site. The calibration template contains a rectangular outline (black line) to define the scale of the 3D model and colored circles to assist with the photogrammetric reconstruction.
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
-
(2019)
-
5
Fig. 4. Left: A photograph taken of the plaster cast of the patient’s right arm. Right: A computer-generated (CG) rendering of the textured photogrammetric reconstruction (PR) model of the plaster cast.
of millimeters). The surface mold applicators were then 3D printed using acrylonitrile butadiene styrene using a Zortrax (Olsztyn, Poland) M200 3D printer. To verify the overall accuracy of the PR (and CTR) workflows and the resultant 3D-printed surface mold applicators, the 3D-printed applicator prototypes and plaster cast were CT scanned together. The CT image was imported into Varian (Palo Alto, CA) BrachyVision. The ruler tool proved unreliable in visually estimating the displacement of the applicator from the plaster cast as the windowing level can be arbitrarily defined. To objectively and quantitatively estimate the position and volume of any air gaps (indicating a less than ideal conformity of the applicator to the plaster cast), the air gaps were automatically segmented in BrachyVision. This was achieved using the following steps: The plaster cast was automatically segmented using the HU-based segmentation tool with an intensity range of 800e2000 HU. The 3D-printed applicators were segmented using the same tool but with an intensity range of 300 to 300 HU. Any air gaps beneath the applicators were detected using the HU-based segmenter using an intensity range of 1000 to 500 HU. The volume was limited using the volume of interest tool to cover the entire
applicator, the volume beneath the applicator, and the top surface of the plaster cast (as shown in Fig. 2). An air gap has been identified medially and anterior to the top surface of the plaster cast in the same figure. This region is shown in yellow. The 2D brush tool was then used to manually remove any contoured region (in the air gap contour) that lay above the 3D-printed applicator volume. The remaining volume is representative of any air volume between the applicator and plaster cast. This estimate of the air volume is limited by the spatial resolution of the CT scanner. The maximum vertical displacement of the applicator from the plaster cast was estimated using the ruler tool and measuring the maximum vertical extent of the contoured air gaps (thus eliminating the subjectivity of the measurement caused by an arbitrarily defined windowing level).
2.3. Virtual applicator design from photogrammetry of a patient In the final stage of the current work, the feasibility of using photogrammetry to design a surface mold applicator directly from images of the treatment site on a patient was
Fig. 5. Comparison of CT- and photo-based reconstruction of the plaster cast. Left (in each image)dphoto reconstruction. Right (in each image)dCT reconstruction.
6
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
Table 1 Comparison of the computed volume and surface areas for the reconstructed model and the CT of the plaster cast Reconstruction method
Volume (cm3)
Area (cm2)
Photogrammetry CT
536.9 537.1
536.3 522.2
Difference
L0.1%
D2.6%
investigated. In the current work, the author’s right hand was selected as the treatment site. First, a theoretical PTV region was delineated on the test subject’s hand using a marker pen, shown in Fig. 3. Twenty still images were taken at approximately uniformly spaced altitude and azimuth angles (relative to the origin of the target volume) at a distance of approximately 20e40 cm. Figure 3 also shows a calibration template containing a rectangular outline (black line) to define the scale of the 3D model and colored circles to assist with the photogrammetric reconstruction. The still images were then processed in Meshroom using a similar method to that outlined in Section 3D Printed Applicator from Photogrammetry. To design a virtual brachytherapy applicator, the PTV region on the PR model plus a margin suitable for attaching brachytherapy catheters was contoured and duplicated from the original model. The faces of this duplicated region were extruded by 6 mm in a direction normal to each face, so that the virtual applicator had a uniform thickness. Five cylindrical volumes of diameter 2 mm (6 French) with approximately equal lateral spacing were then extruded from the applicator. This allowed for five 6-French catheters to be
-
(2019)
-
inserted into the 3D-printed applicator and then connected to the afterloader during treatment. The 3D model of the surface mold applicator was exported as an .stl file (in metric units with a scale of millimeters). The surface mold applicator was once again 3D printed using acrylonitrile butadiene styrene using a Zortrax M200 3D printer. The accuracy of the reconstructed applicator was evaluated qualitatively by visual inspection. In the current work, the accuracy of the 3D-printed hand applicator could not be assessed quantitatively on CT because of the ethical requirements. As such, a qualitative assessment is described in the current work as a preliminary demonstration of the feasibility of this technique.
3. Results 3.1. Photogrammetric reconstructionegeometric accuracy Figure 4 shows a photo and a rendering of the reconstructed model of the plaster cast. A visual inspection of the two models side by side shows excellent agreement with only small variations in the small scale detail. A side-by-side comparison of the ‘‘scaled’’ photo (without textures) and CTR plaster casts are shown in Fig. 5. The model shown on the left in each subimage is the photogrammetry reconstructed (PR) model, and the CTR model is shown on the right. Although the overall dimensions of the two models are almost identical, the photoreconstructed model shows more of the small scale detail (compared with the smooth topology of the CTR model).
Fig. 6. Hausdorff distance. Percentage of matching points between CT and photogrammetry models vs. distance to matching vertex (mm).
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
Fig. 7. 3D visualization of the Hausdorff distance metric for the plaster cast phantom. Red and orange vertices indicate a high level of similarity, whereas blue and green vertices indicate a low level of similarity. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)
The ‘‘true’’ volume of the plaster cast was calculated from an HU-based segmentation in the CT reconstruction using Slicer 3D to be 537.1 cm3. The CT-based segmentation of the plaster cast was imported into Blender, scaled so that the volume matched the ‘‘true volume’’ and the surface area was calculated to be 522.2 cm2. When the postprocessed PR model was scaled to match the physical dimensions of the reference object, the volume was calculated to be 536.9 cm3 and the surface area was 536.3 cm2. These results are summarized in Table 1. The difference in volume between the CT- and photobased reconstruction was approximately 0.1%. The difference in surface area was 2.6%. This is not an unexpected result. A visual inspection of the PR mesh shows a much higher degree of complexity (in terms of visibility of small scale details) compared with the CTR mesh, shown in Fig. 5. This could occur for several reasons including (1) a limitation of the HU-based segmentation algorithm in
-
(2019)
-
7
Slicer, which ‘‘smooths’’ the CT mesh which would reduce the surface area, (2) the limited spatial resolution of the CT scanner compared with photogrammetry, or (3) the reconstruction technique produced by Meshroom creates detail in the model, which increases the surface area of the mesh. It is not clear how much of this extra detail is ‘‘real,’’ but it is apparent that using this method, the photo reconstruction is able to sample significantly more small-scale detail than the CT image. This results in an apparent higher resolution using the photo reconstruction method. The results of the Hausdorff distance calculation are graphed in Fig. 6. The graph shows the percentage of vertices in the CT-obtained model with a corresponding vertex in the photogrammetry model vs. the distance between these correspondences in units of millimeters. The results indicate that when the bottom surface of both meshes is excluded from the analysis (due to a limitation of the photogrammetry method), 93% of the vertices in the CT model have a corresponding vertex in the photogrammetry model within 1 mm (the slice thickness of the CT scan). This indicates that the CT- and photogrammetry-obtained models of the plaster cast are spatially similar. This is visualized in Fig. 7. This result indicates that a high level of agreement was obtained between CT and photogrammetry on the anterior surface of the plaster cast. Poorer agreement was achieved on the superior and inferior surfaces of the phantom. 3.2. 3D-printed applicator from photogrammetry An approximately square projection (when viewed from above) of faces was selected on the PR (and CTR) plaster cast model and extruded by 5 mm to generate the ‘‘virtual’’ surface mold applicator shown in Fig. 8. The 3D-printed versions of the photogrammetry and CT-based applicators are shown in Figs. 9 and 10, respectively. The 3D-printed surface applicators were positioned on the plaster cast using measurements from the virtual
Fig. 8. Virtual surface mold applicator extruded from the photo-reconstructed plaster cast of the patient’s right arm.
8
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
-
(2019)
-
Fig. 9. 3D-printed surface mold applicator generated from the photogrammetry technique.
applicator models. CT scans were then acquired of the plaster cast together with the individual applicators to assess the overall accuracy of the PR technique compared with the CT-based method. Figure 11 shows a transverse and sagittal CT slice of the photogrammetry- (left) and CT-based (right) applicators positioned on the plaster cast. Using Varian (Palo Alto, CA) BrachyVision, two small ‘‘air gaps’’ were identified (shown in Fig. 11) between the bottom surface of the photogrammetry applicator and the plaster cast. Using the ‘‘ruler’’ tool in BrachyVision, the maximum vertical extent of these air gaps were approximately 0.9 mm and 1.2 mm. The total volume of all air gaps detected beneath the photogrammetry-derived applicator was measured to be 0.05 cm3. All other regions of the applicator were in contact with the plaster cast to within the spatial resolution of the CT scanner (resolution 0.1073 cm 0.1073 cm 0.1 cm). The CT-based 3D-printed applicator showed significantly poorer conformity to the plaster cast surface than the photogrammetry-derived applicator. The total volume of the air gaps detected beneath this applicator was measured to be at least 0.3 cm3 (This volume was
calculated based on repeated segmentations of two different CT image acquisitions and repeated applicator positioning on the plaster cast [to minimize uncertainties due to setup]. The volume presented here is the lowest of the air gap volumes recorded.). In the interest of fair comparison, the surface area of the CT-based 3D-printed applicator was 143 cm2, whereas the applicator printed using photogrammetry was 129 cm2. Thus, assuming both methods produced equally conformal applicators, the volume beneath the CT-derived applicator should only be 11% (ratio of applicator surface areas) more than the photogrammetryderived applicator. However, in this case, the volume was larger by a factor of approximately 6. The largest ‘‘air gap’’ detected beneath the CT-derived applicator (shown in the top right corner of Fig. 11) had a maximum vertical extent of 2 mm. These results demonstrate the potential increase in reconstruction accuracy obtained using photogrammetry compared with CT for 3D-printed surface mold applicators. However, there are several factors that may have led to the reduced accuracy of the CT-derived applicator (compared with photogrammetry). First, the resolution of
Fig. 10. 3D-printed surface mold applicator generated from CT acquisition.
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
-
(2019)
-
9
Fig. 11. Visual comparison of air gaps beneath photogrammetry- (left) and CT-derived (right) surface mold applicators. Air gaps beneath the applicators were detected using Brachyvision’s HU thresholding tool and shown in cyan. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)
the CT-derived applicator was limited by the reconstructed resolution of the CT image. The resolution of the CT images used to generate the applicator was 1.073 mm 1.073 mm 1 mm. This relatively low resolution (compared with photogrammetry) resulted in a comparably ‘‘smoothed’’ (spatially averaged) applicator, which did not conform as well to the plaster surface. Second, the topology of the 3D-printed applicator was limited by the HU-based segmentation algorithm used to define the surface of the plaster cast. The segmentation process relies on an HU threshold being specified to delineate the plaster cast from the air surrounding it.
This volume of air gaps could be potentially minimized (and resultant conformity of the applicator to the patient’s skin increased) for both methods of reconstruction by 3D printing the applicators using a semiflexible material. Virtual applicator design from photogrammetry of a patient Figure 12 shows the 3D rendering of the virtual applicator, and Fig. 13 shows the printed applicator placed on the hand of our test subject. Without acquiring a CT image of the applicator on the treatment site, quantitative
Fig. 12. 3D renderings showing the process of producing a custom surface mold applicator for brachytherapy treatments. Top left: Contour the required faces on the 3D model of the treatment area. Top right and bottom: Duplicate the selected faces and extrude by the required thickness (as specified by dosimetric requirements). Bottom: Extrude 2-mm- (6-French) diameter inserts from the applicator to insert catheters.
10
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
-
(2019)
-
accuracy and efficiency of 3D-printed applicator workflows [6,10]. As these workflows typically require 2 separate CT scans of a patient, one key advantage of the proposed technique is the reduction in patient radiation dose. The reason for this is the CT image used to design the 3D applicator can be accomplished with photogrammetry. Second, as photogrammetry can generate a textured surface of the patient’s skin during the reconstruction, accurate PTV delineation can be accomplished without the use of marker wires during the planning CT. There are several sources of uncertainty and limitations in the photogrammetry process, which will need further investigation if this were to be used clinically. These include the following. Fig. 13. Photos of the 3D-printed surface mold applicator. The right and bottom left subfigures show a 6-French interstitial catheter inserted into the central extruded cylinder.
measurements of the accuracy of the reconstructed applicator were not possible. However, the 3D-printed applicator appeared to visually conform well to the test treatment site. Owing to the small size and low mass of the 3D-printed applicator, it was observed that there was not enough weight to hold the applicator in place. This could be corrected in future by printing extra bolus on top of the applicator or simply applying a separate bolus material during treatment. It was also found that the filament-style 3D printer could not print the cylindrical inserts with sufficient accuracy to allow the treatment catheters to be inserted the entire length of the cylinder. In the prototype, this was corrected by pushing a 2-mm-diameter drill piece to remove the extraneous plastic. In a clinical setting, this may not be practical as the catheter inserts are curved in general. Increasing the extruded cylinder diameter by a small offset (e.g., 0.2 mm) during the modeling stage may partially correct this problem. This limitation and correction have been previously reported in the study by Clarke [6]. The approximate time to complete the individual steps in this workflow are summarized in Table 2.
4. Discussion Photogrammetry has been demonstrated as a potential alternative to CT in the generation of 3D-printed surface mold applicators and has the potential to improve the Table 2 Approximate times to complete the individual steps required to manufacture a 3D-printed surface mold applicator using photogrammetry Procedure Photographing treatment site (approx. 30 photos) Photogrammetric reconstruction (with default parameters and using desktop PC) Applicator design in Blender 3D printing
Approximate time (minutes) 5 35 45 240
Defining scale in the reconstructed model The 3D models created using photogrammetry have arbitrary scale. To overcome this issue, a reference object of known length was placed alongside the object being ‘‘scanned’’ to define the scale of the scene. It has been demonstrated by the similarity between the volume and surface area measurements (in the results section), as well as the accuracy of the 3D-printed applicators, that the proposed workflow is technically feasible. However, for treatment sites with more complex curvature, higher levels of scaling accuracy may be required. Image acquisition During the experimentation process, it was discovered that, for optimal results from the reconstruction process, photos must be taken in a well-lit room with diffuse lighting. In the future, a dedicated imaging unit including a mounted camera and lighting may be required for optimal results. The effect of the distance from the object to the camera was not assessed in the current work. Previous works have investigated [22] the effect of imaging distance on photogrammetry point cloud generation and reported the error in reconstructive accuracy increased slowly up to a distance of 20 m from the object. However, the test object used in this study was significantly larger than the object used in the current work and contained high-contrast coded targets to assist with reconstruction. Fundamentally, the ability to resolve two closely spaced features on the test object is inversely proportional to the distance between the test object and camera as specified by the Rayleigh criterion. Hence, photos should be taken as close as possible to the object of interest while ensuring the entire object remains within the frame of the photo. Future work should include a comprehensive analysis and optimization of imaging parameters. Model complexity The Meshroom software used in the current work is capable of creating models of very high resolution. In the
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
examples presented in the current work, the reconstructed models contained in excess of 500,000 vertices. These models required between 30 and 45 min to reconstruct. The potentially unnecessary numbers of vertices made the reconstruction and postprocessing stage slower. Reduction in the number of vertices was performed in the current work using Blender’s ‘‘decimate function,’’ which reduced the number of vertices (in both examples) to approximately 100,000 without significant changes in the model’s topology. Future work could include an investigation of the optimal settings to use in the reconstruction process (including the optimal number of photos and camera to target distance) to obtain an acceptably precise model for brachytherapy surface applicators while reducing the reconstruction and postprocessing times. Automation
-
(2019)
-
11
Although both techniques demonstrated sufficient applicator conformity for 3D-printed brachytherapy applicators, the current work has demonstrated the potential gains in accuracy obtained using the photogrammetry technique. The new workflow was subsequently used to 3D print a prototype of a brachytherapy surface mold applicator from a hypothetical patient’s right hand. Brachytherapy catheters were incorporated into the 3D-printed applicator demonstrating the potential for an accurate and efficient treatment workflow. In future work, we intend to compare and contrast the brachytherapy treatment plans achievable using this technique with our traditional surface mold applicator workflow.
Acknowledgments
Although the individual steps in the current workflow are generally fast, shown in Table 2, the entire process could potentially be scripted to improve efficiency. This has been previously demonstrated in the study by Clarke [6].
The authors thank Mr. Morgan Hunter from ThincLab, University of Adelaide, for providing the 3D printing services and advice.
Conclusions
References
Optical photogrammetry has been applied to the task of HDR brachytherapy surface mold applicator design in the current work using open source software. To our knowledge, this is the first example of the use of optical photogrammetry for HDR brachytherapy surface applicator design. A comparison of the 3D phantom models generated using photogrammetry and conventional CT-based reconstruction has verified the geometric accuracy of the photogrammetry technique. The reconstructed model obtained using photogrammetry had a measured volume and surface area that were within 0.1% and 2.6% of the CTobtained values, respectively. The models were subsequently compared using the Hausdorff distance metric. Using this technique, it was determined that 93% of the visible vertices present in the CT-derived model had a matching vertex on the photogrammetry-derived model within 1 mm (the slice thickness of the CT scanner), indicating a high level of similarity. Surface mold applicator prototypes were then designed and 3D printed directly from the surface of the CT and optically reconstructed phantoms. In the case of the photogrammetry-derived applicator, a CT scan identified two small air gaps between the applicator and the phantom with a total volume of 0.05 cm3. In addition, the maximum displacement between the anterior surface of the phantom and posterior surface of the applicator was measured to be 1.2 mm. The same analysis was subsequently performed on a conventional CT-derived 3D-printed applicator. The total volume of air gaps beneath this applicator was measured to be 0.3 cm3 with a maximum vertical displacement of 2 mm.
[1] Kuncman q, Koz1owski S, Pietraszek A, et al. Highly conformal CT based surface mould brachytherapy for non-melanoma skin cancers of earlobe and nose. J Contemp Brachytherapy 2016;8:195e200. [2] Jones E-L, Tonino Baldion A, Thomas C, et al. Introduction of novel 3D-printed superficial applicators for high-dose-rate skin brachytherapy. Brachytherapy 2017;16:409e414. [3] Guinot JL, Rembielak A, Perez-Calatayud J, et al. GEC-ESTRO ACROP recommendations in skin brachytherapy. Radiother Oncol 2018;126:377e385. [4] Schumacher M, Lasso A, Cumming I, et al. 3D-printed surface mould applicator for high-dose-rate brachytherapy. In: SPIE medical imaging. Orlando, FL: SPIE; 2015. [5] Zhao Y, Moran K, Yewondwossen M, et al. Clinical applications of 3-dimensional printing in radiation therapy. Med Dosim 2017;42: 150e155. [6] Clarke SA. 3D printed surface applicators for high dose rate brachytherapy. Halifax, Nova Scotia: Dalhousie University; 2016. [7] Tyler LF, et al. Clinical implementation of 3D printing for brachytherapy: techniques and emerging applications. Brachytherapy 2016;15:166. [8] Ricotti R, Vavassori A, Bazani A, et al. 3D-printed applicators for high dose rate brachytherapy: dosimetric assessment at different infill percentage. Phys Med 2016;32:1698e1706. [9] Harris B, Nilsson S, Poole C. A feasibility study for using ABS plastic and a low-cost 3D printer for patient-specific brachytherapy mould design. Australas Phys Eng Sci Med 2015;38:399e412. [10] Arenas M, Sabater S, Sintas A, et al. Individualized 3D scanning and printing for non-melanoma skin cancer brachytherapy: a financial study for its integration into clinical workflow. J Contemp Brachytherapy 2017;9:270e276. [11] Rogus RD, Stern RL, Kubo HD. Accuracy of a photogrammetrybased patient positioning and monitoring system for radiation therapy. Med Phys 1999;26:721e728. [12] Calow R, et al. Photogrammetric measurement of patients in radiotherapy. ISPRS J Photogramm Remote Sens 2002;56:347e359. [13] Deantonio L, Masini L, Loi G, et al. Detection of setup uncertainties with 3D surface registration system for conformal radiotherapy of breast cancer. Rep Pract Oncol Radiother 2011;16:77e81.
12
M.J.J. Douglass, A.M. Carac¸a Santos / Brachytherapy
[14] Liu W, Cheung Y, Sabouri P, et al. A continuous surface reconstruction method on point cloud captured from a 3D surface photogrammetry system. Med Phys 2015;42:6564e6571. [15] Moulon P, Monasse P, Marlet R. Adaptive structure from motion with a contrario model estimation. Berlin, Heidelberg: Springer Berlin Heidelberg; 2013. [16] Jancosek M, Pajdla T. Multi-view reconstruction preserving weaklysupported surfaces. CVPR 2011;2011. [17] Blender Online Community, Blender-a 3D modelling and rendering package. Blender Institute. Amsterdam: Blender Foundation. [18] Kikinis R, Pieper SD, Vosburgh KG. 3D slicer: a platform for subjectspecific image analysis, visualization, and clinical support intraoperative imaging and image-guided therapy. New York, NY: Springer; 2014.
-
(2019)
-
[19] Cignoni P, Callieri M, Corsini M, et al. MeshLab: an opensource mesh processing tool. In: Eurographics Italian chapter conference. Geneva, Switzerland: The Eurographics Association; 2008. [20] Cignoni P, Rocchini C, Scopigno R. Metro: measuring error on simplified surfaces. Computer Graphics Forum 1998;17:167e 174. [21] Zhang D, He F, Han S, Zou L, Wu Y, Chen Y. An efficient approach to directly compute the exact Hausdorff distance for 3D point sets. Integrated Comput Aid Eng 2017;24:261e277. [22] An Z. Accuracy assessment of 3D point cloud generated by photogrammetry from different distances. Houghton, MI: Michigan Technological University; 2017.