- Email: [email protected]
An algorithm to obtain a theoretical model of the bronchial tree
Available online at www.sciencedirect.com
ScienceDirect Materials Today: Proceedings 4 (2017) 5761–5766
www.materialstoday.com/proceedings
DAS 2016
An algorithm to obtain a theoretical model of the bronchial tree C. Ciobircaa, G. Gruionub, T. Langoc, H. O. Leirad, L. G. Gruionue, T. Amundsend, E. Nutua, S. D. Pastramaa * a
University Politehnica, Department of Strength of Materials, Splaiul Independentei 313, Sector 6, 060042, Bucharest, Romania b Edwin L. Steele Laboratory for Tumor Biology, Harvard University, 55 Fruit Street Boston, MA 02114, USA c SINTEF Technology and Society, Department of Medical Technology, Olav Kyrres gate 9, Trondheim, Norway d Departament of Circulation and Medical Imaging, Faculty of Medicine, Norwegian University of Science and Technology & St. Olavs Hospital, Department of Thoracic Medicine, Trondheim, Norway e University of Craiova, Department of Mechanics, Calea Bucuresti nr. 107, 200512,Craiova, Dolj County, Romania
Abstract The paper presents an algorithm proposed by the authors in order to obtain a theoretical model of the bronchial and used to test the virtual bronchoscopy procedure implemented in an innovative system for bronchoscopy with electromagnetically tracked and steerable biopsy forceps. The procedure is based on the Weibel theoretical model of the human tracheobronchial tree with new features added by the authors. The algorithm can be an important helping tool in the development of sophisticated algorithms for bronchoscopy applications and may be further implemented in clinical studies of navigated bronchoscopy. © 2017 Elsevier Ltd. All rights reserved. Selection and/or Peer-review under responsibility of 33rd Danubia Adria Symposium on Advances in Experimental
Mechanics.
Keywords: Bronchial tree; Virtual Bronchoscopy; Visualization Toolkit; Computed Tomography; Transbronchial Biopsy.
1. Introduction Diagnostic bronchoscopy in thoracic medicine is frequently performed by pulmonologists world-wide, looking for a broad spectrum of conditions. Investigation of pulmonary infiltrates is a time and resource consuming
* Corresponding author. Tel.: +40-21-402-9206; fax: +40-21-402-9477. E-mail address: [email protected] 2214-7853 © 2017 Elsevier Ltd. All rights reserved. Selection and/or Peer-review under responsibility of 33rd Danubia Adria Symposium on Advances in Experimental Mechanics.
5762
Ciobirca et al. / Materials Today: Proceedings 4 (2017) 5761–5766
specialized health care service, where lung cancer is the leading cause. Lung cancer is one of the most frequent cancers and the incidence increases, and it owns the highest mortality rate of all cancers [1]. Computed Tomography (CT), Magnetic Resonance Imaging (MRI) or Positron Emission Tomography (PET) scan help the pulmonologist in performing purposeful bronchoscopy, finding the instrumental track to target via the central airways, from the mouth or nose into the lungs. Transbronchial Lung Biopsy (TLB) is the only endoscopic access to targets laying outside the airways and therefore represents an important sampling procedures performed during flexible bronchoscopy. Here, samplings tools like a biopsy forceps or aspiration needles are introduced through the working channel of the bronchoscope in order to obtain cytology or tissue specimen from peripheral lung masses and focal or diffuse lung infiltrates [2]. The success rate of diagnostic procedure is variable and often poor, depending on the size of the nodule, the proximity of the nodule to the bronchial tree, the skills of the operator and the accessible equipment, and in the end the success in sampling the nodule [3]. If the procedure fails to reach the peripheral targets, the procedure should be repeated or changed, increasing the delay in diagnosis and treatment. That is why there is a huge and widespread research ongoing in order to diminish the diagnostic obstacles of bronchoscopy. Virtual bronchoscopy (VB) represents a non-invasive alternative to the classical procedure, when it comes to pure visual investigation and orientation inside the airways. VB, similar to the visual part of bronchoscopy, depicts the inside of the bronchial tree, but lacks the ability to look into or describe the surrounding structures, i.e. peripheral tumors [4]. In VB, a computer simulation of the video bronchoscope image is created from the 3-D CT volume (as the most common source), with similar view angle and zoom settings [5]. With VB, the study of the bronchial tree becomes possible in a non-invasive manner with no additional radiation exposure relative to standard CT scan of the chest in patients with benign and malignant disease, even in patients where ordinary video bronchoscopy is too invasive or not possible [6]. In comparison with real bronchoscopy, VB has some advantages: it is a non-invasive procedure that can visualize the entire airway lumen that can be segmented from the CT, it is able to detect bronchial stenosis and obstruction caused by both endoluminal pathology (tumor, mucus, foreign bodies) and external compression (anatomical structures, tumor, lymph nodes), and can be helpful in the preoperative planning of stent placement. It can be also used to evaluate surgical sutures after lung transplantations, lobectomy or pneumectomy [7]. In order to perform a reliable VB as a navigation tool or in combination with another navigation system, complex software is needed, involving segmentation of human lungs from the CT scans, virtual navigation and procedure planning, tracking of medical instruments during the real procedure. Models of the bronchial tree are necessary during the implementation and testing phases of new or modified technical applications, concerning programs (software) as well as equipment (hardware). They are worked out usually from CT scans, which involve a complicated procedure and computer resources. Kitaoka et al. [8] proposed a 3-D model of the human airway tree using an algorithm that generates a branching duct system, based on two principles: i. the amount of fluid delivery through a branch is proportional to the volume of the region it supplies; and ii. the terminal branches are arranged homogeneously within the organ. In this algorithm, generation of the dimensions and directionality of two daughter branches is governed by the properties of the parent branch and the region the parent supplies. It should be mentioned that the model assumes each branch to be a circular, rigid tube with a constant diameter, in contrast to decreasing continuously, like in human airways tree from a 2 cm diameter of the trachea to a 2 mm diameter of the bronchioles, right ahead of the alveolus/ sacculus. A systematic geometrical model of the bronchial tree, described as a self-similar assembly of rigid pipes was proposed by Florens et al. [9]. The model includes the specific geometry of the upper bronchial tree and a self-similar intermediary tree with a systematic branching asymmetry. It ends by the terminal bronchioles whose generations range from 8 to 22. A systematic branching asymmetry is assumed for all generations: each parent airway gives rise to a larger daughter airway (major airway), and a smaller daughter airway (minor airway). For the proximal airways, the diameter ratios (the ratio of the diameter of the major / minor airway over the diameter of the parent airway) are specific to the human anatomy while for the intermediate tree, the diameter ratios are considered to be constant. Tawhai et al. [10] proposed a procedure for generation of subject-specific geometric models of the bronchial airway tree. In their study, curvilinear airway centerline and diameter models have been fitted to human and ovine bronchial trees using detailed data segmented from computed tomography scans. The trees have been extended to model the entire conducting airway system by using a volume-filling algorithm to generate airway centerline locations within detailed volume descriptions of the lungs or lobes. The proposed algorithm produced airway trees with branching asymmetry
Ciobirca et al. / Materials Today: Proceedings 4 (2017) 5761–5766
5763
appropriate for the human and ovine lung. A 3-D fully automated reconstruction algorithm of the bronchial tree up to the sixth- to seventh-order subdivisions, from X-ray CT volumetric acquisitions, using a database consisting of 30 clinical acquisitions including both normal and pathological airways is described by Fetita et al. [11]. Their 3-D reconstruction approach consists in combining axial and radial propagation potentials to control the growth of a subset of low-order airways extracted from the CT volume by means of a robust mathematical morphology operator - the selective marking and depth constrained (SMDC) connection cost. Lo et al. [12] presented a framework for evaluating airway extraction algorithms in a standardized manner in order to establish reference segmentation. Their study showed that no algorithm is capable of extracting more than 77% of the reference, in terms of both branch count and tree length, concluding that better results may be achieved by combining results from different algorithms. Starting from a theoretical model of the human tracheobronchial tree, this paper presents a method that generates 3-D data sets on which a Visualization Toolkit (VTK) pipeline based on the Marching Cubes algorithm is applied in order to extract the surface of the airways and to generate a model of the bronchial tree up to level 5. VTK is an open-source, freely available software system for 3-D computer graphics, image processing, and visualization [13]. The above mentioned generator was further used to develop and test a collision and detection algorithm [14], to develop and test a rigid registration method and to test a segmentation algorithm implemented in the CustusX platform [15], based on which a new fusion imaging system NAVICAD (Navigation System For Confocal Laser Endomicroscopy To Improve Optical Biopsy Of Peripheral Lesions In The Lungs) for spatial guidance of a customized bronchoscopic forceps is currently developed by the authors. 2. The NAVICAD system NAVICAD is an innovative system for bronchoscopy with electromagnetically tracked and steerable biopsy forceps and a specific channel connected to a port in the forceps handle, to mount an optical fiber. The novel biopsy forceps is used to track the position of the tip of the bronchoscope in real-time and to enable the creation of corresponding virtual bronchoscopic images from a pre-procedure CT scan. The main features of the system are: preoperative procedure planning using VB, Electromagnetic Navigation Bronchoscopy using electromagnetic sensors attached to medical instruments, lungs segmentation, registration between virtual image and tracking device coordinates and constraints for the virtual tools representations in VB and during real navigation (in the virtual 3-D image the objects should be kept inside the airways tubes) The NAVICAD system includes an electromagnetic tracking system AURORA for spatial positioning of multiple disposable tools or sensors situated inside or outside the body [16], navigated bronchoscopy forceps for biopsy which includes a 6DOF electromagnetic sensor at the tip to determine its spatial position and orientation in the AURORA magnetic field, an active marker with positioning electromagnetic sensor, placed on the patient skin at the xiphoid process and a computer with specific software application developed within the project. The navigation software, for which the algorithm to obtain the theoretical model of the bronchial tree was developed, loads the patient CT data and uses multiple technologies for anatomy 3-D reconstruction, image-topatient registration, manual calibration and navigation with two main screens for user interface. The system automatically develops and displays airways segmentation, semi-automated lung nodule segmentation, multiple targets selection (eight maximum), virtual bronchoscopy visualization and geodesic minimal path extraction. 3. Virtual bronchoscopy implemented in the NAVICAD system In the NAVICAD system (Fig. 1), the virtual bronchoscopy is used to identify the optimum airway track to a peripheral target outside the visual reach of video bronchoscopy, preparing for detection, exact localization in 3-D and precise sampling procedure, starting from the CT scan images of the patient. While navigating through the CT planes, the operator locates the exact point of the active marker on the patient skin to be used for the initial registration. Next step, the user indicates on the VB or CT windows the entry point of the bronchoscopy and the anatomical target. A pathway between the two points is automatically calculated and created through several points along the airways. During the real bronchoscopy, the path obtained through VB is shown as a distinct line on the screen of the application. The software computes also the instantaneous position of the tip of the real forceps relative
Ciobirca et al. / Materials Today: Proceedings 4 (2017) 5761–5766
5764
to the patient and CT space and continuously overlays it on the 3-D model. Following the path to the target lesion in the bronchial tree, the user visually compares the live video and virtual images of the bronchoscopy. When the bronchoscope diameter is too big to advance in the sub-segmental bronchi (wedge position), the user extends only the navigation sample equipment, needles or forceps, through the working channel of the bronchoscope precisely to the peripheral target and accomplishes the sampling procedure. The navigation is performed using the VB and the instantaneous position of the forceps tip overlaid on the 3-D model. The user is also able to investigate the surrounding tissues using the CT section. The biopsy using the forceps can be performed when the target is reached. The VB procedure was first tested on a theoretical model of the bronchial tree, developed using an algorithm proposed by the authors. 4. The algorithm The algorithm presented herein is based on the Weibel theoretical model of the human tracheobronchial tree [17] which states that the adult human airway begins at the trachea with a cross-sectional area of about 5 cm2 and, through a series of bifurcations, ramifies into a tree with nearly 17,000,000 branches. The trachea is the first branch (level zero) and, at each level, a parent branch is split in two children branches. The model prescribes the diameter and the length of the airways for each branch.
A C Fig. 1. The NAVICAD system.
C'
A' B'
B
A
P
P'
R
B
a. b. Fig. 2. a. Shortest distances from various points to a binary tree; b. Geometric placement of a point P around a line segment
Besides the original model, for the purpose of generating a 3-D volume represented by voxels, values for the width of the tubes walls were added. Starting from this model, we generate a scalar field (a function that associates a scalar value to a point from a region of the three dimensional space, f : R3 → R) with the symmetry of the tracheobronchial tree. The scalar value depends on the distance from the point to the nearest branch of the tree. In Fig. 2,a, various points (A, B, C) are represented around a binary tree, each point having a projection (A¢, B¢, C¢) on a different tree branch. By projection, one considers the shortest distance from the point to the tree branches. In Fig. 2,b, the projection P¢ of a point P on the segment AB is shown together with a circle centered in the projected point and the intersection P¢R of this circle with a reference plane. The geometrical placement of point P on the segment AP is given by the parameter t = ôAP¢ô/ôABô< 1 and by the angle RP¢P. The scalar field:
ì -150, r < d min ï r - d min ï f ( r ,...) = í-500 + 200 × , r Î [ d min , d max ] (1) d max - d min ï ï -1000, r > d max î can be constructed in a cubic region where three zones are defined: inside the “airways” tubes with the value -1000, corresponding to the Hounsfield units for air, the tubes “walls” with values in the interval [-500, -300], for Hounsfield units of lung tissue and “outside” the tubes with value -150 for Hounsfield units of fat tissue. In the equation above, r is the distance from the point (x, y, z) to the nearest tree branch and dmin, dmax are distances defined
Ciobirca et al. / Materials Today: Proceedings 4 (2017) 5761–5766
5765
according to the nearest branch level in the tree and the segment parameter t of the point projection on the branch segment. The tubes lengths, diameters and wall widths are listed in Table 1. Table 1. Dimensions of the airways branches following the Weibel model, [17] Branch level k Mean diameter D(k) [mm] Length L(k) [mm] Wall width w(k) [mm]
0 18 70 3
1 14 50 3
2 10 30 3
3 8 20 2
4 5 13 2
5 4 11 1
The diameters from Table 1 represent the maximum value according to the tree level, but the actual value decreases along the tube central axis from one extremity to the other according to the equation: d = D ( k ) + t × éë D ( k + 1) - D ( k ) ùû
(2)
The obtained theoretical bronchial tree has the following features: i. A parent line and its two children are all in the same plane; ii. The children lines are symmetrical with respect with the parent and form a 45° angle (parameter that can be changed) and iii. The plane of a parent and its two children is rotated 90° in respect with its parent and parent of its parent plane. These prescriptions were implemented in two C++ classes, namely VolumeGenerator and VolumeSampleFunction which are based on vtkImplicitFunction. An instance of vtkImageData is generated at the output port of a vtkSampleFunction instance (which uses our class VolumeSampleFunction). On the generated data set, we applied the following VTK pipeline: vtkImageData → vtkImageShrink3D → vtkGaussianSmooth → vtkMarchingCubes, and render the “airways” surface obtained at the output port (Fig. 3). Trees up to level 5 (63 branches) were generated.
a. b. Fig. 3. “Airways” surface obtained at the output port: a. Center lines; b. Final tree
5. Applications The theoretical data sets obtained as described above were further used to implement a collision detection and resolution algorithm used for virtual navigation and tracking corrections in bronchoscopy procedures – see Fig. 4. Also, they were used to test a segmentation method implemented in the CustusX platform [15]. A rigid registration method for electromagnetic tracking based on a fiducial point (a reference sensor placed on the patient) and a constraint surface is currently developed. The data sets from the generator were used during development and initial testing of this method. A screen of the VB procedure tested using the theoretical bronchial tree is shown in Fig. 5. It features visualization in-scene and 2-D sections, selection of the target in 2D scenes and their visualization in the 3-D scene, possibility to choose the start point for an automatically generated path and the target for the bronchoscopy procedure (outside air tubes – colored in yellow; inside air tubes – colored in green), air target search – an area inside air tubes, close to the bronchoscopy target and development of an automatic path between start point and air target – colored in yellow, close to the tubes walls. VB features also manual navigation with the mouse inside the air tubes and path, manual trajectory and recording of the virtual camera trajectory during virtual bronchoscopy (in red).
5766
Ciobirca et al. / Materials Today: Proceedings 4 (2017) 5761–5766
Fig. 4. Constraints applied on virtual tools in order to keep them inside the airways tubes
Fig. 5. Window of the virtual bronchoscopy procedure using the developed theoretical model.
6. Conclusions The paper presents a new algorithm for obtaining a theoretical model of the bronchial tree studied in an airway phantom. It was subsequently used to implement refined applications for virtual bronchoscopy combined with an electromagnetic navigation platform, with the ultimate purpose of improved navigation feasibility and diagnostic yield. The model proved to be an important helping tool in the development of sophisticated algorithms for bronchoscopy applications. The virtual bronchoscopy used for preoperative planning, virtual training, and guidance helps doctors to detect, locate and precisely sample targets, based on the optimum airway paths in the non-visual areas, created by the new model. The algorithm seems promising for future implementation in clinical studies of navigated bronchoscopy. Acknowledgements The research leading to these results has received funding from EEA Financial Mechanism 2009 - 2014 under the project EEA-JRP-RO-NO-2013-1-0123 - Navigation System For Confocal Laser Endomicroscopy To Improve Optical Biopsy Of Peripheral Lesions In The Lungs (NaviCAD), contract no. 3SEE/30.06.2014. References [1] http://gco.iarc.fr/today/home, accessed on 01.12.2015. [2] Mehta A.C., Jain P. (eds.). Respir Med 2013; 10, DOI 10.1007/978-1-62703-395-4_2. [3] Eberhardt R., Ernst A., Herth F.J.F. Eur Respir J 2009; 34: 1284–1287. [4] Ferguson J. S., McLennan G. Proc Am Thorac Soc 2005; 2: 488–491. [5] Ferguson J. S., McLennan G., Proc Am Thorac Soc vol. 2, 2005, pp. 488-491. [6] Smistad E., Falch T.L., Bozorgi M. Elster A.C., Lindseth F. Med Image Anal 2015; 20: 1-18. [7] De Wever W., Bogaert J., Verschakelen J.A. Semin Ultrasound CT MR 2005; 26: 364–373. [8] Kitaoka H., Takaki R., Suki B. J Appl Physiol 1999; 87:2207-17. [9] Florens M., Sapoval B., Filoche M. J Appl Physiol 2011; 110: 756–763. [10] Tawhai M. H., Hunter P., Tschirren J., Reinhardt J., McLennan G., Hoffman E. A. J Appl Physiol 2004; 97: 2310–2321. [11] Fetita C.I., Prêteux F., Beigelman-Aubry C., Grenier P. IEEE Trans Med Imag 2004; 23: 1353-1364. [12] Lo P., van Ginneken B., Reinhardt J. M., de Bruijne M. IEEE Trans Med Imag 2012; 31; 2093-2107. [13] * * * , http://www.vtk.org/, accessed on 01.12.2015. [14] Ciobirca C., Popa T., Gruionu G., Lango T., Leira H.O., Pastrama S.D., Gruionu L.G. Ciência & Tecnologia dos Materiais, 2016, in print (accepted 15.12.2015). [15] Askeland C., Solberg O.V., Bakeng J.B.L., Reinertsen I., Tangen G.A., Hofstad E.F., Iversen D.H., Vapenstad C., Selbekk T., Lango T., Nagelhus T.A.H., Leira H.O., Unsgård G., Lindseth F. Int J Comput Assist Radiol Surg 2016; 11: 505-519. [16] http://www.ndigital.com/medical/products/aurora/, accessed on 01.12.2015. [17] Weibel E. Morphometry of the Human Lung, New York: Academic Press Inc; 1963.
ScienceDirect Materials Today: Proceedings 4 (2017) 5761–5766
www.materialstoday.com/proceedings
DAS 2016
An algorithm to obtain a theoretical model of the bronchial tree C. Ciobircaa, G. Gruionub, T. Langoc, H. O. Leirad, L. G. Gruionue, T. Amundsend, E. Nutua, S. D. Pastramaa * a
University Politehnica, Department of Strength of Materials, Splaiul Independentei 313, Sector 6, 060042, Bucharest, Romania b Edwin L. Steele Laboratory for Tumor Biology, Harvard University, 55 Fruit Street Boston, MA 02114, USA c SINTEF Technology and Society, Department of Medical Technology, Olav Kyrres gate 9, Trondheim, Norway d Departament of Circulation and Medical Imaging, Faculty of Medicine, Norwegian University of Science and Technology & St. Olavs Hospital, Department of Thoracic Medicine, Trondheim, Norway e University of Craiova, Department of Mechanics, Calea Bucuresti nr. 107, 200512,Craiova, Dolj County, Romania
Abstract The paper presents an algorithm proposed by the authors in order to obtain a theoretical model of the bronchial and used to test the virtual bronchoscopy procedure implemented in an innovative system for bronchoscopy with electromagnetically tracked and steerable biopsy forceps. The procedure is based on the Weibel theoretical model of the human tracheobronchial tree with new features added by the authors. The algorithm can be an important helping tool in the development of sophisticated algorithms for bronchoscopy applications and may be further implemented in clinical studies of navigated bronchoscopy. © 2017 Elsevier Ltd. All rights reserved. Selection and/or Peer-review under responsibility of 33rd Danubia Adria Symposium on Advances in Experimental
Mechanics.
Keywords: Bronchial tree; Virtual Bronchoscopy; Visualization Toolkit; Computed Tomography; Transbronchial Biopsy.
1. Introduction Diagnostic bronchoscopy in thoracic medicine is frequently performed by pulmonologists world-wide, looking for a broad spectrum of conditions. Investigation of pulmonary infiltrates is a time and resource consuming
* Corresponding author. Tel.: +40-21-402-9206; fax: +40-21-402-9477. E-mail address: [email protected] 2214-7853 © 2017 Elsevier Ltd. All rights reserved. Selection and/or Peer-review under responsibility of 33rd Danubia Adria Symposium on Advances in Experimental Mechanics.
5762
Ciobirca et al. / Materials Today: Proceedings 4 (2017) 5761–5766
specialized health care service, where lung cancer is the leading cause. Lung cancer is one of the most frequent cancers and the incidence increases, and it owns the highest mortality rate of all cancers [1]. Computed Tomography (CT), Magnetic Resonance Imaging (MRI) or Positron Emission Tomography (PET) scan help the pulmonologist in performing purposeful bronchoscopy, finding the instrumental track to target via the central airways, from the mouth or nose into the lungs. Transbronchial Lung Biopsy (TLB) is the only endoscopic access to targets laying outside the airways and therefore represents an important sampling procedures performed during flexible bronchoscopy. Here, samplings tools like a biopsy forceps or aspiration needles are introduced through the working channel of the bronchoscope in order to obtain cytology or tissue specimen from peripheral lung masses and focal or diffuse lung infiltrates [2]. The success rate of diagnostic procedure is variable and often poor, depending on the size of the nodule, the proximity of the nodule to the bronchial tree, the skills of the operator and the accessible equipment, and in the end the success in sampling the nodule [3]. If the procedure fails to reach the peripheral targets, the procedure should be repeated or changed, increasing the delay in diagnosis and treatment. That is why there is a huge and widespread research ongoing in order to diminish the diagnostic obstacles of bronchoscopy. Virtual bronchoscopy (VB) represents a non-invasive alternative to the classical procedure, when it comes to pure visual investigation and orientation inside the airways. VB, similar to the visual part of bronchoscopy, depicts the inside of the bronchial tree, but lacks the ability to look into or describe the surrounding structures, i.e. peripheral tumors [4]. In VB, a computer simulation of the video bronchoscope image is created from the 3-D CT volume (as the most common source), with similar view angle and zoom settings [5]. With VB, the study of the bronchial tree becomes possible in a non-invasive manner with no additional radiation exposure relative to standard CT scan of the chest in patients with benign and malignant disease, even in patients where ordinary video bronchoscopy is too invasive or not possible [6]. In comparison with real bronchoscopy, VB has some advantages: it is a non-invasive procedure that can visualize the entire airway lumen that can be segmented from the CT, it is able to detect bronchial stenosis and obstruction caused by both endoluminal pathology (tumor, mucus, foreign bodies) and external compression (anatomical structures, tumor, lymph nodes), and can be helpful in the preoperative planning of stent placement. It can be also used to evaluate surgical sutures after lung transplantations, lobectomy or pneumectomy [7]. In order to perform a reliable VB as a navigation tool or in combination with another navigation system, complex software is needed, involving segmentation of human lungs from the CT scans, virtual navigation and procedure planning, tracking of medical instruments during the real procedure. Models of the bronchial tree are necessary during the implementation and testing phases of new or modified technical applications, concerning programs (software) as well as equipment (hardware). They are worked out usually from CT scans, which involve a complicated procedure and computer resources. Kitaoka et al. [8] proposed a 3-D model of the human airway tree using an algorithm that generates a branching duct system, based on two principles: i. the amount of fluid delivery through a branch is proportional to the volume of the region it supplies; and ii. the terminal branches are arranged homogeneously within the organ. In this algorithm, generation of the dimensions and directionality of two daughter branches is governed by the properties of the parent branch and the region the parent supplies. It should be mentioned that the model assumes each branch to be a circular, rigid tube with a constant diameter, in contrast to decreasing continuously, like in human airways tree from a 2 cm diameter of the trachea to a 2 mm diameter of the bronchioles, right ahead of the alveolus/ sacculus. A systematic geometrical model of the bronchial tree, described as a self-similar assembly of rigid pipes was proposed by Florens et al. [9]. The model includes the specific geometry of the upper bronchial tree and a self-similar intermediary tree with a systematic branching asymmetry. It ends by the terminal bronchioles whose generations range from 8 to 22. A systematic branching asymmetry is assumed for all generations: each parent airway gives rise to a larger daughter airway (major airway), and a smaller daughter airway (minor airway). For the proximal airways, the diameter ratios (the ratio of the diameter of the major / minor airway over the diameter of the parent airway) are specific to the human anatomy while for the intermediate tree, the diameter ratios are considered to be constant. Tawhai et al. [10] proposed a procedure for generation of subject-specific geometric models of the bronchial airway tree. In their study, curvilinear airway centerline and diameter models have been fitted to human and ovine bronchial trees using detailed data segmented from computed tomography scans. The trees have been extended to model the entire conducting airway system by using a volume-filling algorithm to generate airway centerline locations within detailed volume descriptions of the lungs or lobes. The proposed algorithm produced airway trees with branching asymmetry
Ciobirca et al. / Materials Today: Proceedings 4 (2017) 5761–5766
5763
appropriate for the human and ovine lung. A 3-D fully automated reconstruction algorithm of the bronchial tree up to the sixth- to seventh-order subdivisions, from X-ray CT volumetric acquisitions, using a database consisting of 30 clinical acquisitions including both normal and pathological airways is described by Fetita et al. [11]. Their 3-D reconstruction approach consists in combining axial and radial propagation potentials to control the growth of a subset of low-order airways extracted from the CT volume by means of a robust mathematical morphology operator - the selective marking and depth constrained (SMDC) connection cost. Lo et al. [12] presented a framework for evaluating airway extraction algorithms in a standardized manner in order to establish reference segmentation. Their study showed that no algorithm is capable of extracting more than 77% of the reference, in terms of both branch count and tree length, concluding that better results may be achieved by combining results from different algorithms. Starting from a theoretical model of the human tracheobronchial tree, this paper presents a method that generates 3-D data sets on which a Visualization Toolkit (VTK) pipeline based on the Marching Cubes algorithm is applied in order to extract the surface of the airways and to generate a model of the bronchial tree up to level 5. VTK is an open-source, freely available software system for 3-D computer graphics, image processing, and visualization [13]. The above mentioned generator was further used to develop and test a collision and detection algorithm [14], to develop and test a rigid registration method and to test a segmentation algorithm implemented in the CustusX platform [15], based on which a new fusion imaging system NAVICAD (Navigation System For Confocal Laser Endomicroscopy To Improve Optical Biopsy Of Peripheral Lesions In The Lungs) for spatial guidance of a customized bronchoscopic forceps is currently developed by the authors. 2. The NAVICAD system NAVICAD is an innovative system for bronchoscopy with electromagnetically tracked and steerable biopsy forceps and a specific channel connected to a port in the forceps handle, to mount an optical fiber. The novel biopsy forceps is used to track the position of the tip of the bronchoscope in real-time and to enable the creation of corresponding virtual bronchoscopic images from a pre-procedure CT scan. The main features of the system are: preoperative procedure planning using VB, Electromagnetic Navigation Bronchoscopy using electromagnetic sensors attached to medical instruments, lungs segmentation, registration between virtual image and tracking device coordinates and constraints for the virtual tools representations in VB and during real navigation (in the virtual 3-D image the objects should be kept inside the airways tubes) The NAVICAD system includes an electromagnetic tracking system AURORA for spatial positioning of multiple disposable tools or sensors situated inside or outside the body [16], navigated bronchoscopy forceps for biopsy which includes a 6DOF electromagnetic sensor at the tip to determine its spatial position and orientation in the AURORA magnetic field, an active marker with positioning electromagnetic sensor, placed on the patient skin at the xiphoid process and a computer with specific software application developed within the project. The navigation software, for which the algorithm to obtain the theoretical model of the bronchial tree was developed, loads the patient CT data and uses multiple technologies for anatomy 3-D reconstruction, image-topatient registration, manual calibration and navigation with two main screens for user interface. The system automatically develops and displays airways segmentation, semi-automated lung nodule segmentation, multiple targets selection (eight maximum), virtual bronchoscopy visualization and geodesic minimal path extraction. 3. Virtual bronchoscopy implemented in the NAVICAD system In the NAVICAD system (Fig. 1), the virtual bronchoscopy is used to identify the optimum airway track to a peripheral target outside the visual reach of video bronchoscopy, preparing for detection, exact localization in 3-D and precise sampling procedure, starting from the CT scan images of the patient. While navigating through the CT planes, the operator locates the exact point of the active marker on the patient skin to be used for the initial registration. Next step, the user indicates on the VB or CT windows the entry point of the bronchoscopy and the anatomical target. A pathway between the two points is automatically calculated and created through several points along the airways. During the real bronchoscopy, the path obtained through VB is shown as a distinct line on the screen of the application. The software computes also the instantaneous position of the tip of the real forceps relative
Ciobirca et al. / Materials Today: Proceedings 4 (2017) 5761–5766
5764
to the patient and CT space and continuously overlays it on the 3-D model. Following the path to the target lesion in the bronchial tree, the user visually compares the live video and virtual images of the bronchoscopy. When the bronchoscope diameter is too big to advance in the sub-segmental bronchi (wedge position), the user extends only the navigation sample equipment, needles or forceps, through the working channel of the bronchoscope precisely to the peripheral target and accomplishes the sampling procedure. The navigation is performed using the VB and the instantaneous position of the forceps tip overlaid on the 3-D model. The user is also able to investigate the surrounding tissues using the CT section. The biopsy using the forceps can be performed when the target is reached. The VB procedure was first tested on a theoretical model of the bronchial tree, developed using an algorithm proposed by the authors. 4. The algorithm The algorithm presented herein is based on the Weibel theoretical model of the human tracheobronchial tree [17] which states that the adult human airway begins at the trachea with a cross-sectional area of about 5 cm2 and, through a series of bifurcations, ramifies into a tree with nearly 17,000,000 branches. The trachea is the first branch (level zero) and, at each level, a parent branch is split in two children branches. The model prescribes the diameter and the length of the airways for each branch.
A C Fig. 1. The NAVICAD system.
C'
A' B'
B
A
P
P'
R
B
a. b. Fig. 2. a. Shortest distances from various points to a binary tree; b. Geometric placement of a point P around a line segment
Besides the original model, for the purpose of generating a 3-D volume represented by voxels, values for the width of the tubes walls were added. Starting from this model, we generate a scalar field (a function that associates a scalar value to a point from a region of the three dimensional space, f : R3 → R) with the symmetry of the tracheobronchial tree. The scalar value depends on the distance from the point to the nearest branch of the tree. In Fig. 2,a, various points (A, B, C) are represented around a binary tree, each point having a projection (A¢, B¢, C¢) on a different tree branch. By projection, one considers the shortest distance from the point to the tree branches. In Fig. 2,b, the projection P¢ of a point P on the segment AB is shown together with a circle centered in the projected point and the intersection P¢R of this circle with a reference plane. The geometrical placement of point P on the segment AP is given by the parameter t = ôAP¢ô/ôABô< 1 and by the angle RP¢P. The scalar field:
ì -150, r < d min ï r - d min ï f ( r ,...) = í-500 + 200 × , r Î [ d min , d max ] (1) d max - d min ï ï -1000, r > d max î can be constructed in a cubic region where three zones are defined: inside the “airways” tubes with the value -1000, corresponding to the Hounsfield units for air, the tubes “walls” with values in the interval [-500, -300], for Hounsfield units of lung tissue and “outside” the tubes with value -150 for Hounsfield units of fat tissue. In the equation above, r is the distance from the point (x, y, z) to the nearest tree branch and dmin, dmax are distances defined
Ciobirca et al. / Materials Today: Proceedings 4 (2017) 5761–5766
5765
according to the nearest branch level in the tree and the segment parameter t of the point projection on the branch segment. The tubes lengths, diameters and wall widths are listed in Table 1. Table 1. Dimensions of the airways branches following the Weibel model, [17] Branch level k Mean diameter D(k) [mm] Length L(k) [mm] Wall width w(k) [mm]
0 18 70 3
1 14 50 3
2 10 30 3
3 8 20 2
4 5 13 2
5 4 11 1
The diameters from Table 1 represent the maximum value according to the tree level, but the actual value decreases along the tube central axis from one extremity to the other according to the equation: d = D ( k ) + t × éë D ( k + 1) - D ( k ) ùû
(2)
The obtained theoretical bronchial tree has the following features: i. A parent line and its two children are all in the same plane; ii. The children lines are symmetrical with respect with the parent and form a 45° angle (parameter that can be changed) and iii. The plane of a parent and its two children is rotated 90° in respect with its parent and parent of its parent plane. These prescriptions were implemented in two C++ classes, namely VolumeGenerator and VolumeSampleFunction which are based on vtkImplicitFunction. An instance of vtkImageData is generated at the output port of a vtkSampleFunction instance (which uses our class VolumeSampleFunction). On the generated data set, we applied the following VTK pipeline: vtkImageData → vtkImageShrink3D → vtkGaussianSmooth → vtkMarchingCubes, and render the “airways” surface obtained at the output port (Fig. 3). Trees up to level 5 (63 branches) were generated.
a. b. Fig. 3. “Airways” surface obtained at the output port: a. Center lines; b. Final tree
5. Applications The theoretical data sets obtained as described above were further used to implement a collision detection and resolution algorithm used for virtual navigation and tracking corrections in bronchoscopy procedures – see Fig. 4. Also, they were used to test a segmentation method implemented in the CustusX platform [15]. A rigid registration method for electromagnetic tracking based on a fiducial point (a reference sensor placed on the patient) and a constraint surface is currently developed. The data sets from the generator were used during development and initial testing of this method. A screen of the VB procedure tested using the theoretical bronchial tree is shown in Fig. 5. It features visualization in-scene and 2-D sections, selection of the target in 2D scenes and their visualization in the 3-D scene, possibility to choose the start point for an automatically generated path and the target for the bronchoscopy procedure (outside air tubes – colored in yellow; inside air tubes – colored in green), air target search – an area inside air tubes, close to the bronchoscopy target and development of an automatic path between start point and air target – colored in yellow, close to the tubes walls. VB features also manual navigation with the mouse inside the air tubes and path, manual trajectory and recording of the virtual camera trajectory during virtual bronchoscopy (in red).
5766
Ciobirca et al. / Materials Today: Proceedings 4 (2017) 5761–5766
Fig. 4. Constraints applied on virtual tools in order to keep them inside the airways tubes
Fig. 5. Window of the virtual bronchoscopy procedure using the developed theoretical model.
6. Conclusions The paper presents a new algorithm for obtaining a theoretical model of the bronchial tree studied in an airway phantom. It was subsequently used to implement refined applications for virtual bronchoscopy combined with an electromagnetic navigation platform, with the ultimate purpose of improved navigation feasibility and diagnostic yield. The model proved to be an important helping tool in the development of sophisticated algorithms for bronchoscopy applications. The virtual bronchoscopy used for preoperative planning, virtual training, and guidance helps doctors to detect, locate and precisely sample targets, based on the optimum airway paths in the non-visual areas, created by the new model. The algorithm seems promising for future implementation in clinical studies of navigated bronchoscopy. Acknowledgements The research leading to these results has received funding from EEA Financial Mechanism 2009 - 2014 under the project EEA-JRP-RO-NO-2013-1-0123 - Navigation System For Confocal Laser Endomicroscopy To Improve Optical Biopsy Of Peripheral Lesions In The Lungs (NaviCAD), contract no. 3SEE/30.06.2014. References [1] http://gco.iarc.fr/today/home, accessed on 01.12.2015. [2] Mehta A.C., Jain P. (eds.). Respir Med 2013; 10, DOI 10.1007/978-1-62703-395-4_2. [3] Eberhardt R., Ernst A., Herth F.J.F. Eur Respir J 2009; 34: 1284–1287. [4] Ferguson J. S., McLennan G. Proc Am Thorac Soc 2005; 2: 488–491. [5] Ferguson J. S., McLennan G., Proc Am Thorac Soc vol. 2, 2005, pp. 488-491. [6] Smistad E., Falch T.L., Bozorgi M. Elster A.C., Lindseth F. Med Image Anal 2015; 20: 1-18. [7] De Wever W., Bogaert J., Verschakelen J.A. Semin Ultrasound CT MR 2005; 26: 364–373. [8] Kitaoka H., Takaki R., Suki B. J Appl Physiol 1999; 87:2207-17. [9] Florens M., Sapoval B., Filoche M. J Appl Physiol 2011; 110: 756–763. [10] Tawhai M. H., Hunter P., Tschirren J., Reinhardt J., McLennan G., Hoffman E. A. J Appl Physiol 2004; 97: 2310–2321. [11] Fetita C.I., Prêteux F., Beigelman-Aubry C., Grenier P. IEEE Trans Med Imag 2004; 23: 1353-1364. [12] Lo P., van Ginneken B., Reinhardt J. M., de Bruijne M. IEEE Trans Med Imag 2012; 31; 2093-2107. [13] * * * , http://www.vtk.org/, accessed on 01.12.2015. [14] Ciobirca C., Popa T., Gruionu G., Lango T., Leira H.O., Pastrama S.D., Gruionu L.G. Ciência & Tecnologia dos Materiais, 2016, in print (accepted 15.12.2015). [15] Askeland C., Solberg O.V., Bakeng J.B.L., Reinertsen I., Tangen G.A., Hofstad E.F., Iversen D.H., Vapenstad C., Selbekk T., Lango T., Nagelhus T.A.H., Leira H.O., Unsgård G., Lindseth F. Int J Comput Assist Radiol Surg 2016; 11: 505-519. [16] http://www.ndigital.com/medical/products/aurora/, accessed on 01.12.2015. [17] Weibel E. Morphometry of the Human Lung, New York: Academic Press Inc; 1963.