- Email: [email protected]
An optically based tactile system for interactive gradual surface scanning
International Congress Series 1268 (2004) 573 – 578
www.ics-elsevier.com
An optically based tactile system for interactive gradual surface scanning Andreas Rose a,*, Andreas Hein b, Tim C. Lueth a a Department for Maxillofacial Surgery-Clinical Navigation and Robotics, BCMM-Berlin Center for Mechatronical Medical Devices, Charite´-Universitary Medicine Berlin, Fraunhofer-Institut fuer Produktionsanlagen und Konstruktionstechnik, Augustenburger Platz 1, 13353 Berlin, Germany b Department for Computer Science, Carl v. Ossietzky University, Oldenburg, Germany
Abstract. This article describes a new interactive surface scanning system for the use in navigated dental implantology. It allows preoperative plannings based on image data that were produced by simple surface scanning. This can be achieved through a tactile interactive gradual scanning method using a navigated spatula-like scanning tool. The system guides the user interactively. A virtual patient surface model is created. This model is adapted, based on scanning information from the spatula tool. Once the virtual surface is modelled, it can be stored as a compatible volume data set for following planning systems. The described method is embedded in the context of the CT-free navigation approach. D 2004 Published by Elsevier B.V. Keywords: Mechanical surface scanning; Maxillofacial surgery; CT-free navigation
1. Introduction In this article, a newly developed tactile system for interactive gradual surface scanning is described. The radiation exposure of X-ray-based medical image devices steps more and more into the awareness of the patients. For that reason, it is important to develop methods that represent alternatives. Particularly, the creation of patient surface models should be done, if possible, completely without X-ray oriented procedures. The spreading of medical assistance systems that support surgical interventions based on optical navigation systems also increases for interventions in maxillofacial surgery. They are mostly based on computer tomography (CT) or digital volume tomography (DVT) imaging. The radiation exposure and high costs motivate our
* Corresponding author. Tel.: +49-30-450555179; fax: +49-30-450555912. E-mail address: [email protected] (A. Rose). URL: http://www.srl-berlin.de. 0531-5131/ D 2004 Published by Elsevier B.V. doi:10.1016/j.ics.2004.03.314
574
A. Rose et al. / International Congress Series 1268 (2004) 573–578
search for alternatives. Small implanting interventions within a clear anatomical environment could be done without volume information, e.g. from CT or DVT. It is not always necessary to use tomographical volume data as a base for generating surface data for planning of prothesis models. That leads to the assumption that in the specified cases the necessary information can be produced also by surface scanning. In order to abandon radiation exposure and to keep costs small, a tactile sensing system was developed to extend a system for navigated dental implantology [1]. With the new functionality, the system is able to create virtual surface models of a patients surface. The system uses an optical navigation system to localize a tactile scanning instrument and a patient. By gradual mechanical scanning of the patient surface, the user interactively creates a virtual patient surface. Following planning systems are able to use the DICOM data generated before as basis. The system therefore is a base for CT-free navigation and the planning approach. Various optical or mechanical surface scanner exist in dental medicine (CEREC 3, Sirona; Everest, KaVo; Procera, Nobel Biocare; etc.). The compact systems are used to detect patients surface in small areas. Scanning data serve as a base for later treatment planning and producing of protheses. But this systems are not able to create image data compatible with planning systems. In the technical environment, scanning systems based on optical or mechanical methods exist. This systems are mostly very cost intensive and not for medical use (Faro Arm Platinum, FARO Technologies; VI-910, Minolta). Medical navigation is based on optical position sensors (Polaris, NDI; Ropal, Rohwedder). These sensors detect position and orientation of reflectors that can be attached to instruments and parts of a patient. There is no system available that mechanically scans a patients surface based on optical sensors and stores the scanning data into a medical image data format (e.g. DICOM). 2. Material and methods The developed system is capable of mechanical surface scanning of patient surface and creation of a three dimensional surface model that can be later stored in different data formats (e.g. medical image data). 2.1. System components The system is based on a scanning instrument and a navigation system (RoboDent) with a optical sensor (Polaris, Northern Digital) (Fig. 1). The scanning instrument is similar to a clay modelling tool. It consists of a spatula like tip with dimension of 1 to 10 to 15 mm and a handhold. The instrument is extended using a localiser to measure its 3D position and orientation. The instruments coordination system is described relative to the patients coordination system. So position changes of the patient have no effect. Through the two computer monitors the user has a visual feedback of working in the optimal camera space and the progress of the modelling process.
A. Rose et al. / International Congress Series 1268 (2004) 573–578
575
Fig. 1. The picture shows the components of the tactile surface scanning system: (a) display screens, (b) patient, (c) navigation camera and (d) scanning instrument (nBCMM 2004).
2.2. System workflow The functionality is different from common surface scanning systems. First, a coarse virtual model is created. Then the instrument tip is moved mechanically by the user along the patient contour. If the instrument tip is within the virtual model, segments are removed. The handling is similar to clay modelling. The systems workflow is divided into the following four steps:
Creating a dental splint, Definition of the coarse prospective modelling area, Modelling of the precise patient surface, Saving the created model in the DICOM or STL data format. At the beginning, an individual dental splint is created. Therefore, the localiser can easily and exact be attached. The user defines the prospective scanning area. This is done by defining the position, orientation and shape of the interesting region by moving the scanning instrument along the necessary region of the patient jaw (Fig. 2a). During this phase, the current orientation, position and shape are visualised on the computer screen. At the end, a coarse model of the patient’s surface will be determined from the previously recorded scatterplot. The virtual patient model from the previous phase is visualised on a computer screen. This will now be modelled towards a detailed real patient surface structure. That is done by moving the instrument along the patient outlines. So the interesting
576
A. Rose et al. / International Congress Series 1268 (2004) 573–578
Fig. 2. The pictures shows the initialisation and modelling procedure of the scanning process displayed on the computer screen. (a) The navigation bow, the procpective modelling area (the line) and the tool are shown during the initialisation process. (b) The virtual model of the patient surface and the tool are shown (nBCMM 2004).
details of the coarse virtual model becomes more and more accurate (Fig. 2b). The user pursues in real time, which modelled details need to be extended in further modelling steps. If necessary, sections of the modelling creation can be repeated to reach a higher accuracy. Thus, a precise interactive modelling of the patient’s surface can be achieved and monitored in real time using a computer screen. The virtual modelling technique is based on a voxel based approach [2,3]. Manipulations of the model by an instrument can be handled very easy by modifying each voxel lying within the instrument region. For visualising the model, an incremental marching cubes algorithm [2] is used to generate and adjust a triangle mesh. This mesh will be rendered on the computer screen.
A. Rose et al. / International Congress Series 1268 (2004) 573–578
577
As soon as the user is satisfied with the details of the virtual patient surface model, it is stored in DICOM or STL format. If the DICOM format is chosen necessary information is included to allow automatic image registration [5]. Thus, the volume data set is compatible to the RoboDent system. 3. Results An approach was presented that surface data could be the base for further plannings in simple implant situations. So it is possible to abandon cost-intensive and dangerous X-ray oriented imaging methods. A further advantage is to have no radiation exposure. The loss of volume data can sometimes lead to suppression of important information. Further experiments must show whether a cooperation with individual navigated X-ray images [4] is necessary. The developed system is an alternative to existing image data sources for applications that are only based on surface information.
Fig. 3. (a) A picture of the original dental model that was used for scanning. (b) The visualised exported STL surface built from (a). (c) The surface model created from the voxel data and a voxel data slice created by the scanning system from the model shown in (a) (a screenshot of the RoboDent system with imported DICOM data). (nBCMM 2004).
578
A. Rose et al. / International Congress Series 1268 (2004) 573–578
Fig. 3b and c shows the results of scanning a dental model (Fig. 3a). Fig. 3b is a screenshot of the RoboDent system with imported DICOM data that was produced by the surface scanner. The visualised exported STL data is shown in Fig. 3c. An initial experiment was performed to determine the accuracy of the developed system. Therefore, a well-defined rectangular block was scanned using the spatula like instrument. A CT scan of the rectangular block was performed. As reference, a surface model was generated from the CT scan. Both surface models were compared. This was done by calculating the shortest distance from each vertex of the first surface to all components of the second surface. An approximate accuracy of 0.5 mm with maximum deviations of 0.8 mm were achieved. Whether this accuracy satisfies the requirements of different applications, must likewise checked in further experiments. 4. Conclusion In this article, a newly developed tactile system was presented for the gradual mechanical scanning of patient surfaces. The goal of this work was to point out an alternative data source to X-ray oriented imaging procedures for planning purposes. The system stores the built patient model as compatible medical image data. Therefore navigation systems (e.g. RoboDent) are able to import the data. The system is based on optical navigation for localizing the position of the scanning instrument and the patient. With real time visualisation feedback of the modelling process, the user can evaluate the quality of the work. Thus, the modelling is focused on relevant patient surface details. Acknowledgements This research work has been performed at the Department for Maxillofacial SurgeryClinical Navigation and Robotics, Prof. Dr. Juergen Bier and Prof. Dr. Tim C. Lueth, Charite´-Universitary Medicine Berlin and the Fraunhofer-Institut fu¨r Produktionsanlagen und Konstruktionstechnik-IPK Berlin, Prof. Dr.-Ing. Eckart Uhlmann. We are also supported by the Alfried Krupp von Bohlen und Halbach-Stiftung. Parts of the research and the equipment have been supported financially by the European Regional Development Fund (ERDF), Deutsche Forschungsgemeinschaft (DFG), Deutsche Krebshilfe (granted to Prof. Dr. J. Bier, PD Dr. P. Wust) and the Berliner Sparkassenstiftung Medizin. Special thanks to the companies RoboDent, Altatec, Ziehm Instrumentarium, Planmeca, Straumann, Medtronic and Philips for their support. References [1] O. Schermeier, Ein Navigationssystem fu¨r die dentale Implantologie. PhD-thesis, Technical University of Berlin, 2001. [2] T.A. Galyean, J.F. Hughes, Sculpting: an interactive volumetric modeling technique, Computer Graphics 25 (4) (1991) 267 – 274. [3] S.W. Wang, A.E. Kaufman, Volume sculpting, Symposium on Interactive 3D Graphics, ACM SIGGRAPH, 1995, pp. 151 – 156. [4] D. Szymanski, A. Hein, T.C. Lueth, Navi-X—a planning and treatment system for dental implantology based on navigated projection images, Proceedings of CARS, 2003, pp. 1243 – 1249. [5] O. Schermeier, et al., Automatic patient registration in computer assisted maxillofacial surgery, Proceedings of MMVR, 2002, pp. 461 – 467.
www.ics-elsevier.com
An optically based tactile system for interactive gradual surface scanning Andreas Rose a,*, Andreas Hein b, Tim C. Lueth a a Department for Maxillofacial Surgery-Clinical Navigation and Robotics, BCMM-Berlin Center for Mechatronical Medical Devices, Charite´-Universitary Medicine Berlin, Fraunhofer-Institut fuer Produktionsanlagen und Konstruktionstechnik, Augustenburger Platz 1, 13353 Berlin, Germany b Department for Computer Science, Carl v. Ossietzky University, Oldenburg, Germany
Abstract. This article describes a new interactive surface scanning system for the use in navigated dental implantology. It allows preoperative plannings based on image data that were produced by simple surface scanning. This can be achieved through a tactile interactive gradual scanning method using a navigated spatula-like scanning tool. The system guides the user interactively. A virtual patient surface model is created. This model is adapted, based on scanning information from the spatula tool. Once the virtual surface is modelled, it can be stored as a compatible volume data set for following planning systems. The described method is embedded in the context of the CT-free navigation approach. D 2004 Published by Elsevier B.V. Keywords: Mechanical surface scanning; Maxillofacial surgery; CT-free navigation
1. Introduction In this article, a newly developed tactile system for interactive gradual surface scanning is described. The radiation exposure of X-ray-based medical image devices steps more and more into the awareness of the patients. For that reason, it is important to develop methods that represent alternatives. Particularly, the creation of patient surface models should be done, if possible, completely without X-ray oriented procedures. The spreading of medical assistance systems that support surgical interventions based on optical navigation systems also increases for interventions in maxillofacial surgery. They are mostly based on computer tomography (CT) or digital volume tomography (DVT) imaging. The radiation exposure and high costs motivate our
* Corresponding author. Tel.: +49-30-450555179; fax: +49-30-450555912. E-mail address: [email protected] (A. Rose). URL: http://www.srl-berlin.de. 0531-5131/ D 2004 Published by Elsevier B.V. doi:10.1016/j.ics.2004.03.314
574
A. Rose et al. / International Congress Series 1268 (2004) 573–578
search for alternatives. Small implanting interventions within a clear anatomical environment could be done without volume information, e.g. from CT or DVT. It is not always necessary to use tomographical volume data as a base for generating surface data for planning of prothesis models. That leads to the assumption that in the specified cases the necessary information can be produced also by surface scanning. In order to abandon radiation exposure and to keep costs small, a tactile sensing system was developed to extend a system for navigated dental implantology [1]. With the new functionality, the system is able to create virtual surface models of a patients surface. The system uses an optical navigation system to localize a tactile scanning instrument and a patient. By gradual mechanical scanning of the patient surface, the user interactively creates a virtual patient surface. Following planning systems are able to use the DICOM data generated before as basis. The system therefore is a base for CT-free navigation and the planning approach. Various optical or mechanical surface scanner exist in dental medicine (CEREC 3, Sirona; Everest, KaVo; Procera, Nobel Biocare; etc.). The compact systems are used to detect patients surface in small areas. Scanning data serve as a base for later treatment planning and producing of protheses. But this systems are not able to create image data compatible with planning systems. In the technical environment, scanning systems based on optical or mechanical methods exist. This systems are mostly very cost intensive and not for medical use (Faro Arm Platinum, FARO Technologies; VI-910, Minolta). Medical navigation is based on optical position sensors (Polaris, NDI; Ropal, Rohwedder). These sensors detect position and orientation of reflectors that can be attached to instruments and parts of a patient. There is no system available that mechanically scans a patients surface based on optical sensors and stores the scanning data into a medical image data format (e.g. DICOM). 2. Material and methods The developed system is capable of mechanical surface scanning of patient surface and creation of a three dimensional surface model that can be later stored in different data formats (e.g. medical image data). 2.1. System components The system is based on a scanning instrument and a navigation system (RoboDent) with a optical sensor (Polaris, Northern Digital) (Fig. 1). The scanning instrument is similar to a clay modelling tool. It consists of a spatula like tip with dimension of 1 to 10 to 15 mm and a handhold. The instrument is extended using a localiser to measure its 3D position and orientation. The instruments coordination system is described relative to the patients coordination system. So position changes of the patient have no effect. Through the two computer monitors the user has a visual feedback of working in the optimal camera space and the progress of the modelling process.
A. Rose et al. / International Congress Series 1268 (2004) 573–578
575
Fig. 1. The picture shows the components of the tactile surface scanning system: (a) display screens, (b) patient, (c) navigation camera and (d) scanning instrument (nBCMM 2004).
2.2. System workflow The functionality is different from common surface scanning systems. First, a coarse virtual model is created. Then the instrument tip is moved mechanically by the user along the patient contour. If the instrument tip is within the virtual model, segments are removed. The handling is similar to clay modelling. The systems workflow is divided into the following four steps:
Creating a dental splint, Definition of the coarse prospective modelling area, Modelling of the precise patient surface, Saving the created model in the DICOM or STL data format. At the beginning, an individual dental splint is created. Therefore, the localiser can easily and exact be attached. The user defines the prospective scanning area. This is done by defining the position, orientation and shape of the interesting region by moving the scanning instrument along the necessary region of the patient jaw (Fig. 2a). During this phase, the current orientation, position and shape are visualised on the computer screen. At the end, a coarse model of the patient’s surface will be determined from the previously recorded scatterplot. The virtual patient model from the previous phase is visualised on a computer screen. This will now be modelled towards a detailed real patient surface structure. That is done by moving the instrument along the patient outlines. So the interesting
576
A. Rose et al. / International Congress Series 1268 (2004) 573–578
Fig. 2. The pictures shows the initialisation and modelling procedure of the scanning process displayed on the computer screen. (a) The navigation bow, the procpective modelling area (the line) and the tool are shown during the initialisation process. (b) The virtual model of the patient surface and the tool are shown (nBCMM 2004).
details of the coarse virtual model becomes more and more accurate (Fig. 2b). The user pursues in real time, which modelled details need to be extended in further modelling steps. If necessary, sections of the modelling creation can be repeated to reach a higher accuracy. Thus, a precise interactive modelling of the patient’s surface can be achieved and monitored in real time using a computer screen. The virtual modelling technique is based on a voxel based approach [2,3]. Manipulations of the model by an instrument can be handled very easy by modifying each voxel lying within the instrument region. For visualising the model, an incremental marching cubes algorithm [2] is used to generate and adjust a triangle mesh. This mesh will be rendered on the computer screen.
A. Rose et al. / International Congress Series 1268 (2004) 573–578
577
As soon as the user is satisfied with the details of the virtual patient surface model, it is stored in DICOM or STL format. If the DICOM format is chosen necessary information is included to allow automatic image registration [5]. Thus, the volume data set is compatible to the RoboDent system. 3. Results An approach was presented that surface data could be the base for further plannings in simple implant situations. So it is possible to abandon cost-intensive and dangerous X-ray oriented imaging methods. A further advantage is to have no radiation exposure. The loss of volume data can sometimes lead to suppression of important information. Further experiments must show whether a cooperation with individual navigated X-ray images [4] is necessary. The developed system is an alternative to existing image data sources for applications that are only based on surface information.
Fig. 3. (a) A picture of the original dental model that was used for scanning. (b) The visualised exported STL surface built from (a). (c) The surface model created from the voxel data and a voxel data slice created by the scanning system from the model shown in (a) (a screenshot of the RoboDent system with imported DICOM data). (nBCMM 2004).
578
A. Rose et al. / International Congress Series 1268 (2004) 573–578
Fig. 3b and c shows the results of scanning a dental model (Fig. 3a). Fig. 3b is a screenshot of the RoboDent system with imported DICOM data that was produced by the surface scanner. The visualised exported STL data is shown in Fig. 3c. An initial experiment was performed to determine the accuracy of the developed system. Therefore, a well-defined rectangular block was scanned using the spatula like instrument. A CT scan of the rectangular block was performed. As reference, a surface model was generated from the CT scan. Both surface models were compared. This was done by calculating the shortest distance from each vertex of the first surface to all components of the second surface. An approximate accuracy of 0.5 mm with maximum deviations of 0.8 mm were achieved. Whether this accuracy satisfies the requirements of different applications, must likewise checked in further experiments. 4. Conclusion In this article, a newly developed tactile system was presented for the gradual mechanical scanning of patient surfaces. The goal of this work was to point out an alternative data source to X-ray oriented imaging procedures for planning purposes. The system stores the built patient model as compatible medical image data. Therefore navigation systems (e.g. RoboDent) are able to import the data. The system is based on optical navigation for localizing the position of the scanning instrument and the patient. With real time visualisation feedback of the modelling process, the user can evaluate the quality of the work. Thus, the modelling is focused on relevant patient surface details. Acknowledgements This research work has been performed at the Department for Maxillofacial SurgeryClinical Navigation and Robotics, Prof. Dr. Juergen Bier and Prof. Dr. Tim C. Lueth, Charite´-Universitary Medicine Berlin and the Fraunhofer-Institut fu¨r Produktionsanlagen und Konstruktionstechnik-IPK Berlin, Prof. Dr.-Ing. Eckart Uhlmann. We are also supported by the Alfried Krupp von Bohlen und Halbach-Stiftung. Parts of the research and the equipment have been supported financially by the European Regional Development Fund (ERDF), Deutsche Forschungsgemeinschaft (DFG), Deutsche Krebshilfe (granted to Prof. Dr. J. Bier, PD Dr. P. Wust) and the Berliner Sparkassenstiftung Medizin. Special thanks to the companies RoboDent, Altatec, Ziehm Instrumentarium, Planmeca, Straumann, Medtronic and Philips for their support. References [1] O. Schermeier, Ein Navigationssystem fu¨r die dentale Implantologie. PhD-thesis, Technical University of Berlin, 2001. [2] T.A. Galyean, J.F. Hughes, Sculpting: an interactive volumetric modeling technique, Computer Graphics 25 (4) (1991) 267 – 274. [3] S.W. Wang, A.E. Kaufman, Volume sculpting, Symposium on Interactive 3D Graphics, ACM SIGGRAPH, 1995, pp. 151 – 156. [4] D. Szymanski, A. Hein, T.C. Lueth, Navi-X—a planning and treatment system for dental implantology based on navigated projection images, Proceedings of CARS, 2003, pp. 1243 – 1249. [5] O. Schermeier, et al., Automatic patient registration in computer assisted maxillofacial surgery, Proceedings of MMVR, 2002, pp. 461 – 467.