- Email: [email protected]
A digital approach to dynamic jaw tracking using a target tracking system and a structured-light three-dimensional scanner
G Model JPOR 476 No. of Pages 6
journal of prosthodontic research xxx (2018) xxx–xxx
Contents lists available at ScienceDirect
Journal of Prosthodontic Research journal homepage: www.elsevier.com/locate/jpor
Technical procedure
A digital approach to dynamic jaw tracking using a target tracking system and a structured-light three-dimensional scanner Joo Hyun Kwona , Sungbin Imb , Minho Changb , Jong-Eun Kimc,* , June-Sung Shimc a
Department of Prosthodontics, Gangnam Severance Dental Hospital, Seoul, Republic of Korea Department of Mechanical Engineering, Korea University, Seoul, Republic of Korea c Department of Prosthodontics, College of Dentistry, Yonsei University, Seoul, Republic of Korea b
A R T I C L E I N F O
A B S T R A C T
Article history: Received 12 July 2017 Received in revised form 22 March 2018 Accepted 7 May 2018 Available online xxx
Purpose: This technical procedure describes a method for tracking mandibular movement using a threedimensional (3D) optical scanner and target tracking system to digitally portray the motion of the mandible and temporomandibular joints by merging cone beam computed tomography (CBCT) data. Methods: Four nonreflective targets were attached to the labial surface of the incisors in a noncolinear arrangement. Mandibular movement was tracked directly using a 3D facial scanner and target tracking software after merging facial scanning data, digital data obtained from a diagnostic cast, and CBCT scan data based on several landmarks of the anterior teeth. The moving path of the subjects’ mandible was converted to CBCT-based data to confirm the actual movement of the mandible and temporomandibular joints. Conclusions: The digital implementation of mandibular movement using a 3D optical scanner and target tracking system is not prone to the same restrictions and limitations inherent in mechanical equipment; therefore, it is possible to reconstruct more realistic movement(s). This technique can be used in a wide variety of dental applications involving movement of the mandibular jaw, such as fabrication of dental prostheses, or for the diagnosis and treatment of temporomandibular joint disease. © 2018 Japan Prosthodontic Society. Published by Elsevier Ltd. All rights reserved.
Keywords: Mandibular movement Target tracking Three-dimensional scanner Digital dentistry CAD/CAM
1. Introduction Fully understanding functional movement of the jaw is very difficult because the mandible consists of one body and two joints that undergo mobile articulation involving complicated muscular actions. In dentistry, this has resulted in the use of mechanical articulators and a facebow device to simulate the border movements and pathways of the mandible because most procedures involved in the fabrication of prostheses for dental restorations, or the diagnosis of temporomandibular joint (TMJ) dysfunction, cannot be performed directly on patients [1,2]. The facebow is used to orient a maxillary working cast on a mechanical articulator, more specifically, to locate and transfer the arbitrary transverse hinge axis to the articulator using the ear-piece component. However, the location of the hinge axis varies among individuals and, therefore, any measurement chosen as an arbitrary hinge axis could lead to large errors and
* Corresponding author. E-mail address: [email protected] (J.-E. Kim).
poor accuracies due to the wide diversity of true hinge axis point locations [3]. These errors tend to accumulate as mandibular positions deviate from maximum intercuspation. In addition, a recently published systematic review stated that there was no evidence that more satisfactory clinical results could be obtained if facebow transfers were performed during prosthetic reconstruction. Thus, they hypothesize that the use of the facebow is not essential in the manufacture of prostheses and analysis of TMJ movement [4,5]. Developments in digital dentistry over the past few decades have resulted in mechanical articulators that simulate mandibular movements being replaced and/or supplemented with virtual articulators in dental computer-aided design/computer-aided manufacturing (CAD/CAM) systems [6]. In contrast to conventional mechanical procedures, the virtual articulator enables the visualization of three-dimensional (3D) calculated jaw movements for specific TMJ parameters or based on patient-specific dynamic motion data obtained using special devices [7,8]. Several CAD/CAM systems that provide various types of virtual articulator simulations are currently available [9]. However, the virtual articulator in CAD software, such as exocad DentalCAD (Exocad GmbH,
https://doi.org/10.1016/j.jpor.2018.05.001 1883-1958/ © 2018 Japan Prosthodontic Society. Published by Elsevier Ltd. All rights reserved.
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001
G Model JPOR 476 No. of Pages 6
2
J.H. Kwon et al. / journal of prosthodontic research xxx (2018) xxx–xxx
Darmstadt, Germany) or the 3shape dental system (3shape A/S, Copenhagen, Denmark), adopts a method of mounting the mechanical articulator using a facebow and then inputting the parameters obtained from the mechanical articulator to the software again based on a mathematical simulation. This system, however, remains bound to the inherent restraints and limitations of the face-bow device. The aim of the present article is to introduce a novel digital approach that portrays actual mandibular movements using a 3D optical scanner and target tracking system, which reproduces the motion of the TMJ region in three dimensions by merging cone beam computed tomography (CBCT) data and overcoming the limitations of the conventional facebow technique and mechanical articulator. 2. Materials and methods 2.1. Dental cast scan data and CBCT data preparation To implement this technique, one participant without tooth loss or TMJ problems participated in the study. Digital 3D images of the maxilla and mandible were acquired using CBCT [PaXZenith3D, Vatech Co., Ltd., Hwaseong, Korea; voxel size = 0.4 mm, field of view (FOV) = 24 19 cm]. According to manufacturer’s instructions, CBCT scanning took 24 s with tube voltage from 100– 120 kVp and tube current of 6 mA. The segmentation and 3D reconstruction of the skull and jaws are performed using CBCT data in stereolithography (STL) format from digital imaging and communications in medicine (DICOM) and image processing software (OnDemand3D, Cybermed Co., Ltd., Seoul, Korea.). Maxillary and mandibular stone casts are fabricated using prefabricated custom tray (Bosworth Fastray acrylic material, Harry J. Bosworth, Skokie, IL, USA), polyvinyl siloxane elastomeric impression material (Aquasil Ultra, Dentsply Caulk, Milford, DE, USA) and improved dental stone (Hi-Koseton, Maruishi gypsum Co., Ltd., Osaka, Japan.). 3D digital casts are obtained using a dental model scanner (Identica hybrid, Medit Co., Ltd., Seoul, Korea). The original 3D surface model data are aligned and transformed from the CBCT scan and the scanned digital cast to reorganize the triangulated mesh points using an image registration tool (Ezscan8, Medit Co., Ltd.). 2.2. Attaching the target sticker and aligning acquired data A lip and cheek retractor (Optiview, Kerr Hawe, Bioggio, Switzerland) are placed in the mouth of the subject, and four sticky nonreflective targets are attached to the labial surface of the incisors in a noncolinear arrangement (Fig. 1). The targets on the maxilla and mandible have inverted colors to distinguish between maxillary and mandibular movements. All reflective surfaces are treated with an antireflective spray (VITA CEREC propellant & CEREC powder; VITA Zahnfabrik, Bad Sackingen, Germany) to minimize scanning errors for teeth with high gloss. The oral cavity is scanned in maximum intercuspation using a structured-light 3D facial scanner (Rexcan CS2, Medit Co., Ltd.) to obtain the 3D spatial relationship of each dental arch (Fig. 2a). The scan data acquired from the facial scanner is loaded into the image registration software and aligned with the dental cast scan data and CBCT STL data using the reference 3D model (Fig. 2b and c). The alignment process is performed to generate the transformation matrix by comparing four reference points between the in vivo situation and those in the digital 3D casts. During this process, a complete real-time movie is constructed using facial scan data, digital data of the maxilla and mandible dental cast, and CBCT data. These data are then used to trace mandibular movements.
Fig. 1. Four nonreflective targets were attached to the incisors in a noncolinear arrangement to prepare for tracking mandibular movement. An antireflective spray was applied to the shiny surfaces.
2.3. Real-time mandibular movement tracking process In vivo tracking of mandibular movement is performed using a facial scanner. The tracking processes are conducted sequentially in the following order: opening and closing movements; protrusion out to the edge-to-edge position; and lateral movements to the left and right sides up to the canine-to-canine position. Because the relationship between the facial scan data and the CBCT data had already been established through landmarks of the anterior segment, the motion of the mandible can be tracked through the movement of the target stickers attached to the lower anterior portion. When the patient moves the mandible, the movement can be confirmed in real time through the movement of the dental cast STL and CBCT data. The merged CBCT data confirm the movement of the condyle in the trajectory of mouth opening and closing (Fig. 3a and b). As reported in previous studies, the authors found that there is more than a single hinge axis in the early mouth-opening period, which is generally known to occur only in rotational movement [10]. It was also confirmed that the actual motion of the subject was reproduced in the digital software, even in the anterior and lateral movement tracking data (Fig. 3c and d). 2.4. Logging the trajectory of movement Tracked mandibular movements can also be recorded by the STEYX function in spreadsheet software (Excel, Microsoft Corp., Redmond, WA, USA). To record movement using CBCT data, a specific point to be measured can be selected, and the path of forward and lateral movement can be recorded and evaluated in various planes (Figs. 4 and 5). 2.5. Tracking stability evaluation of target tracking system To evaluate the tracking stability of the target tracking system using an optical scanner, we recorded changes in the coordinates, measured using real-time target tracking in a stationary state in the absence of mandibular movement. The coordinates of the target attached to the maxilla were fixed at an immobile point and movement was tracked by setting the tracking points on both condyle portions through the hinge axis of the mandibular CBCT data and the mandible target. The optical scanner was recorded at a rate of 50 frames per second. We acquired 450 coordinates for 9 s per tracking point. The distance between the coordinates of the 450 frames (x, y, z) and
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001
G Model JPOR 476 No. of Pages 6
J.H. Kwon et al. / journal of prosthodontic research xxx (2018) xxx–xxx
3
Fig. 2. (a) Three-dimensional (3D) facial scan data. (b) Alignment of the dental cast with the facial scan data using an image registration tool. (c) Alignment of the original 3D model with the cone beam computed tomography scan.
Fig. 3. Real-time scan and digital representation of various mandibular motions of dental cast STL data and cone beam computed tomography data reproduced in digital software. (a) Initial opening motion. (b) Translational movement occurs when the opening is large. (c) Right lateral movement; (d) left lateral movement.
Fig. 4. Reference planes and points for data analysis. (a) Frankfort horizontal plane. (b) Hinge axis points of bilateral condyles.
mean coordinates (xa, ya, za) were calculated using the following formula: qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Distance ¼ ðx xa Þ2 þ ðy ya Þ2 þ ðz za Þ2 3. Difference from conventional methods Despite the efforts of many clinicians and researchers, hinge axis recording via facebow is limited using arbitrary methods.
Currently, traditional mechanical articulators cannot accurately reproduce unique mandibular movements in individual patients [6,7,11]. In recent years, there have also been efforts to digitally track the movements of the mandible by introducing various methods: using an optoelectronic system [12,13]; combining an optical camera with magnetic resonance imaging [14]; electromagnetic tracking devices [15]; fluoroscopy [16]; ultrasonic devices [17]; and combining an optical camera with computed tomography data [18]. Most of these
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001
G Model JPOR 476 No. of Pages 6
4
J.H. Kwon et al. / journal of prosthodontic research xxx (2018) xxx–xxx
Fig. 5. Kinematics of the mandibular condyle. All charts, except d and e, are the horizontal view with the hinge axis as the starting point of motion. The horizontal axis of the charts d and e is the Frankfort horizontal plane. (a) Protrusion. (b) Lateral movement to the left side. (c) Lateral movement to the right side. (d) Protrusion. Right sagittal view. (e) Protrusion. Left sagittal view. (f) Protrusion. Enlarged-scale view of chart a. (g) Lateral movement to the left side rearranged with a fixed rotation axis. (h) Lateral movement to the right side rearranged with a fixed rotation axis.
Fig. 6. The tracking stability of (a) the mandibular target attachment site, (b) left condyle tracking point and (c) right condyle tracking point. Changes in the coordinates during the recording time of 450 s using the optical scanner was implemented as a graph (mm).
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001
G Model JPOR 476 No. of Pages 6
J.H. Kwon et al. / journal of prosthodontic research xxx (2018) xxx–xxx
5
Fig. 7. The mean tracking stability of the target tracking system by tracking site (mm).
methods, however, have required the construction of patient-specific transoral appliances and/or heavy extraoral equipment worn on the patient’s head. In contrast, the current tracking method using a facial scanner requiring only the placement of a simple retractor and some small targets, which also makes it useful for the accurate measurement of less-limited mandibular motions. 4. Effect or performance Such a digital approach would have significant potential in many applications. First, by tracking the condyle movement of the mandible using patient-specific CBCT data and a precision optical scanner, we can provide more accurate values to mechanical articulators. In addition, it can be applied to virtual articulators in CAD software, which can overcome the limitations of the existing face-bow technique. When the patient’s gypsum model is superimposed, the patient’s own mandibular movement can be measured on the occlusal plane. This can overcome the limitations of the mechanical articulator. The stability of tracking in this study was 4.36 mm in the mandible target area, 30.78 mm in the left condyle area, and 37.74 mm in the right condyle area (Figs. 6 and 7). The accuracy of the target tracking system, which evaluated whether the distance between the actual targets could be accurately reproduced during static position, lateral movement, protrusive movement and opening movement, also showed a value of 4.1–6.9 mm (Fig. 8). Through this, it can be expected that the reproduction of the condylar movement through this tracking system is precisely possible. This is a small error compared to the total scan accuracy of laboratory scanners used in the digital prosthesis manufacturing process, which is reported to be about 78 mm [19]. In addition, the reproducibility of the condylar movement through target tracking is much smaller than the error of about 752 mm that occurs in the process of positioning the maxillary arch data on the virtual articulator through face-bow transfer, as reported by Solaberrieta et al. [20]. In addition, the analysis of occlusal patterns is possible using this technique. When the patient performs anterior and lateral movements, it is possible to identify which teeth are guided, and through which occlusal interferences can also be detected. This can contribute to more accurate virtual articulation and to fabricating more accurate prostheses when uploaded to dental CAD software [21]. Because the trajectory of mouth opening and closing can be accurately recorded, it can also be useful in the preparation of prostheses that require
Fig. 8. Reference positions for accuracy evaluation and the mean accuracy of the target tracking system by tracking distance between outermost targets. (a) The distance between the actual outermost targets recorded by the optical scanner is measured. (b) The distance between the targets on the tracking data after the CBCT data were superimposed. (c) The mean accuracy data during static position, lateral movement, protrusive movement and opening movement (mm).
changing the occlusal vertical dimension of the patient. This overcomes the limitations of mechanical articulators. 5. Conclusion This article introduced a novel methodology in which mandibular movements are tracked during functional motion, and the actual movement of the mandibular body could be confirmed digitally through the merging of dental cast scan data and CBCT data. It can be used for the fabrication of prostheses, and the diagnosis and treatment of TMJ disease using recordings of the mandibular movement pathway. In particular, the use of only a target sticker, which is simply attached to the labial surface of the anterior tooth, can be used to track mandibular movements, which is very simple compared with current marker technology, and also minimizes patient inconvenience. Acknowledgment This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2017R1D1A1B03034835). References [1] Gärtner C., Kordass B. The virtual articulator: development and evaluation. Int J Comput Dent 2003;6:11–24. [2] Fang JJ, Kuo TH. Modelling of mandibular movement. Comput Biol Med 2008;38:1152–62. [3] Walker PM. Discrepancies between arbitrary and true hinge axes. J Prosthet Dent 1980;43:279–85. [4] Farias-Neto A, Dias AH, de Miranda BF, de Oliveira AR. Face-bow transfer in prosthodontics: a systematic review of the literature. J Oral Rehabil 2013;40:686–92. [5] Yohn K. The face bow is irrelevant for making prostheses and planning orthognathic surgery. J Am Dent Assoc 2016;147:421–6. [6] Bisler A, Bockholt U, Kordass B, Suchan M, Voss G. The virtual articulator. Int J Comput Dent 2002;5:101–6. [7] Kordass B, Gärtner C, Söhnel A, Bisler A, Voss G, Bockholt U, et al. The virtual articulator in dentistry: concept and development. Dent Clin North Am 2002;46:493–506.
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001
G Model JPOR 476 No. of Pages 6
6
J.H. Kwon et al. / journal of prosthodontic research xxx (2018) xxx–xxx
[8] Koralakunte PR, Aljanakh M. The role of virtual articulator in prosthetic and restorative dentistry. J Clin Diagn Res 2014;8:ZE25–8. [9] Solaberrieta E, Garmendia A, Minguez R, Brizuela A, Pradies G. Virtual facebow technique. J Prosthet Dent 2015;114:751–5. [10] Mapelli A, Galante D, Lovecchio N, Sforza C, Ferrario VF. Translation and rotation movements of the mandible during mouth opening and closing. Clin Anat 2009;22:311–8. [11] Pandita A, Dod A, Bhat R. Virtual articulators: a digital excellence in prosthetic and restorative dentistry. J Appl Dent Med Sci 2016;2:110–7. [12] Peck CC, Murray GM, Johnson CW, Klineberg IJ. The variability of condylar point pathways in open-close jaw movements. J Prosthet Dent 1997;77:394– 404. [13] Röhrle O, Waddell JN, Foster KD, Saini H, Pullan AJ. Using a motion-capture system to record dynamic articulation for application in CAD/CAM software. J Prosthodont 2009;18:703–10. [14] Leader JK, Boston JR, Rudy TE, Greco CM, Zaki HS. Relation of jaw sounds and kinematics visualized and quantified using 3-D computer animation. Med Eng Phys 2003;25:191–200. [15] Baeyens JP, Gilomen H, Erdmann B, Clijsen R, Cabri J, Vissers D. In vivo measurement of the 3D kinematics of the temporomandibular joint using miniaturized electromagnetic trackers: technical report. Med Biol Eng Comput 2013;51:479–84.
[16] Chen CC, Lin CC, Chen YJ, Hong SW, Lu TW. A method for measuring threedimensional mandibular kinematics in vivo using single-plane fluoroscopy. Dentomaxillofac Radiol 2013;42:95958184. [17] Sójka A, Huber J, Kaczmarek E, He˛dzelek W. Evaluation of mandibular movement functions using instrumental ultrasound system. J Prosthodont 2017;26:123–8. [18] Chang AR, Han JJ, Kim DS, Yi WJ, Hwang SJ. Evaluation of intra-articular distance narrowing during temporomandibular joint movement in patients with facial asymmetry using 3-dimensional computed tomography image and tracking camera system. J Craniomaxillofac Surg 2015;43:342–8. [19] Jeong ID, Lee JJ, Jeon JH, Kim JH, Kim HY, Kim WC. Accuracy of complete-arch model using an intraoral video scanner: an in vitro study. J Prosthet Dent 2016;115:755–9. [20] Solaberrieta E, Mínguez R, Barrenetxea L, Otegi JR, Szentpétery A. Comparison of the accuracy of a 3-dimensional virtual method and the conventional method for transferring the maxillary cast to a virtual articulator. J Prosthet Dent 2015;113:191–7. [21] Ruge S, Quooss A, Kordass B. Variability of closing movements, dynamic occlusion, and occlusal contact patterns during mastication. Int J Comput Dent 2011;14:119–27.
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001
journal of prosthodontic research xxx (2018) xxx–xxx
Contents lists available at ScienceDirect
Journal of Prosthodontic Research journal homepage: www.elsevier.com/locate/jpor
Technical procedure
A digital approach to dynamic jaw tracking using a target tracking system and a structured-light three-dimensional scanner Joo Hyun Kwona , Sungbin Imb , Minho Changb , Jong-Eun Kimc,* , June-Sung Shimc a
Department of Prosthodontics, Gangnam Severance Dental Hospital, Seoul, Republic of Korea Department of Mechanical Engineering, Korea University, Seoul, Republic of Korea c Department of Prosthodontics, College of Dentistry, Yonsei University, Seoul, Republic of Korea b
A R T I C L E I N F O
A B S T R A C T
Article history: Received 12 July 2017 Received in revised form 22 March 2018 Accepted 7 May 2018 Available online xxx
Purpose: This technical procedure describes a method for tracking mandibular movement using a threedimensional (3D) optical scanner and target tracking system to digitally portray the motion of the mandible and temporomandibular joints by merging cone beam computed tomography (CBCT) data. Methods: Four nonreflective targets were attached to the labial surface of the incisors in a noncolinear arrangement. Mandibular movement was tracked directly using a 3D facial scanner and target tracking software after merging facial scanning data, digital data obtained from a diagnostic cast, and CBCT scan data based on several landmarks of the anterior teeth. The moving path of the subjects’ mandible was converted to CBCT-based data to confirm the actual movement of the mandible and temporomandibular joints. Conclusions: The digital implementation of mandibular movement using a 3D optical scanner and target tracking system is not prone to the same restrictions and limitations inherent in mechanical equipment; therefore, it is possible to reconstruct more realistic movement(s). This technique can be used in a wide variety of dental applications involving movement of the mandibular jaw, such as fabrication of dental prostheses, or for the diagnosis and treatment of temporomandibular joint disease. © 2018 Japan Prosthodontic Society. Published by Elsevier Ltd. All rights reserved.
Keywords: Mandibular movement Target tracking Three-dimensional scanner Digital dentistry CAD/CAM
1. Introduction Fully understanding functional movement of the jaw is very difficult because the mandible consists of one body and two joints that undergo mobile articulation involving complicated muscular actions. In dentistry, this has resulted in the use of mechanical articulators and a facebow device to simulate the border movements and pathways of the mandible because most procedures involved in the fabrication of prostheses for dental restorations, or the diagnosis of temporomandibular joint (TMJ) dysfunction, cannot be performed directly on patients [1,2]. The facebow is used to orient a maxillary working cast on a mechanical articulator, more specifically, to locate and transfer the arbitrary transverse hinge axis to the articulator using the ear-piece component. However, the location of the hinge axis varies among individuals and, therefore, any measurement chosen as an arbitrary hinge axis could lead to large errors and
* Corresponding author. E-mail address: [email protected] (J.-E. Kim).
poor accuracies due to the wide diversity of true hinge axis point locations [3]. These errors tend to accumulate as mandibular positions deviate from maximum intercuspation. In addition, a recently published systematic review stated that there was no evidence that more satisfactory clinical results could be obtained if facebow transfers were performed during prosthetic reconstruction. Thus, they hypothesize that the use of the facebow is not essential in the manufacture of prostheses and analysis of TMJ movement [4,5]. Developments in digital dentistry over the past few decades have resulted in mechanical articulators that simulate mandibular movements being replaced and/or supplemented with virtual articulators in dental computer-aided design/computer-aided manufacturing (CAD/CAM) systems [6]. In contrast to conventional mechanical procedures, the virtual articulator enables the visualization of three-dimensional (3D) calculated jaw movements for specific TMJ parameters or based on patient-specific dynamic motion data obtained using special devices [7,8]. Several CAD/CAM systems that provide various types of virtual articulator simulations are currently available [9]. However, the virtual articulator in CAD software, such as exocad DentalCAD (Exocad GmbH,
https://doi.org/10.1016/j.jpor.2018.05.001 1883-1958/ © 2018 Japan Prosthodontic Society. Published by Elsevier Ltd. All rights reserved.
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001
G Model JPOR 476 No. of Pages 6
2
J.H. Kwon et al. / journal of prosthodontic research xxx (2018) xxx–xxx
Darmstadt, Germany) or the 3shape dental system (3shape A/S, Copenhagen, Denmark), adopts a method of mounting the mechanical articulator using a facebow and then inputting the parameters obtained from the mechanical articulator to the software again based on a mathematical simulation. This system, however, remains bound to the inherent restraints and limitations of the face-bow device. The aim of the present article is to introduce a novel digital approach that portrays actual mandibular movements using a 3D optical scanner and target tracking system, which reproduces the motion of the TMJ region in three dimensions by merging cone beam computed tomography (CBCT) data and overcoming the limitations of the conventional facebow technique and mechanical articulator. 2. Materials and methods 2.1. Dental cast scan data and CBCT data preparation To implement this technique, one participant without tooth loss or TMJ problems participated in the study. Digital 3D images of the maxilla and mandible were acquired using CBCT [PaXZenith3D, Vatech Co., Ltd., Hwaseong, Korea; voxel size = 0.4 mm, field of view (FOV) = 24 19 cm]. According to manufacturer’s instructions, CBCT scanning took 24 s with tube voltage from 100– 120 kVp and tube current of 6 mA. The segmentation and 3D reconstruction of the skull and jaws are performed using CBCT data in stereolithography (STL) format from digital imaging and communications in medicine (DICOM) and image processing software (OnDemand3D, Cybermed Co., Ltd., Seoul, Korea.). Maxillary and mandibular stone casts are fabricated using prefabricated custom tray (Bosworth Fastray acrylic material, Harry J. Bosworth, Skokie, IL, USA), polyvinyl siloxane elastomeric impression material (Aquasil Ultra, Dentsply Caulk, Milford, DE, USA) and improved dental stone (Hi-Koseton, Maruishi gypsum Co., Ltd., Osaka, Japan.). 3D digital casts are obtained using a dental model scanner (Identica hybrid, Medit Co., Ltd., Seoul, Korea). The original 3D surface model data are aligned and transformed from the CBCT scan and the scanned digital cast to reorganize the triangulated mesh points using an image registration tool (Ezscan8, Medit Co., Ltd.). 2.2. Attaching the target sticker and aligning acquired data A lip and cheek retractor (Optiview, Kerr Hawe, Bioggio, Switzerland) are placed in the mouth of the subject, and four sticky nonreflective targets are attached to the labial surface of the incisors in a noncolinear arrangement (Fig. 1). The targets on the maxilla and mandible have inverted colors to distinguish between maxillary and mandibular movements. All reflective surfaces are treated with an antireflective spray (VITA CEREC propellant & CEREC powder; VITA Zahnfabrik, Bad Sackingen, Germany) to minimize scanning errors for teeth with high gloss. The oral cavity is scanned in maximum intercuspation using a structured-light 3D facial scanner (Rexcan CS2, Medit Co., Ltd.) to obtain the 3D spatial relationship of each dental arch (Fig. 2a). The scan data acquired from the facial scanner is loaded into the image registration software and aligned with the dental cast scan data and CBCT STL data using the reference 3D model (Fig. 2b and c). The alignment process is performed to generate the transformation matrix by comparing four reference points between the in vivo situation and those in the digital 3D casts. During this process, a complete real-time movie is constructed using facial scan data, digital data of the maxilla and mandible dental cast, and CBCT data. These data are then used to trace mandibular movements.
Fig. 1. Four nonreflective targets were attached to the incisors in a noncolinear arrangement to prepare for tracking mandibular movement. An antireflective spray was applied to the shiny surfaces.
2.3. Real-time mandibular movement tracking process In vivo tracking of mandibular movement is performed using a facial scanner. The tracking processes are conducted sequentially in the following order: opening and closing movements; protrusion out to the edge-to-edge position; and lateral movements to the left and right sides up to the canine-to-canine position. Because the relationship between the facial scan data and the CBCT data had already been established through landmarks of the anterior segment, the motion of the mandible can be tracked through the movement of the target stickers attached to the lower anterior portion. When the patient moves the mandible, the movement can be confirmed in real time through the movement of the dental cast STL and CBCT data. The merged CBCT data confirm the movement of the condyle in the trajectory of mouth opening and closing (Fig. 3a and b). As reported in previous studies, the authors found that there is more than a single hinge axis in the early mouth-opening period, which is generally known to occur only in rotational movement [10]. It was also confirmed that the actual motion of the subject was reproduced in the digital software, even in the anterior and lateral movement tracking data (Fig. 3c and d). 2.4. Logging the trajectory of movement Tracked mandibular movements can also be recorded by the STEYX function in spreadsheet software (Excel, Microsoft Corp., Redmond, WA, USA). To record movement using CBCT data, a specific point to be measured can be selected, and the path of forward and lateral movement can be recorded and evaluated in various planes (Figs. 4 and 5). 2.5. Tracking stability evaluation of target tracking system To evaluate the tracking stability of the target tracking system using an optical scanner, we recorded changes in the coordinates, measured using real-time target tracking in a stationary state in the absence of mandibular movement. The coordinates of the target attached to the maxilla were fixed at an immobile point and movement was tracked by setting the tracking points on both condyle portions through the hinge axis of the mandibular CBCT data and the mandible target. The optical scanner was recorded at a rate of 50 frames per second. We acquired 450 coordinates for 9 s per tracking point. The distance between the coordinates of the 450 frames (x, y, z) and
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001
G Model JPOR 476 No. of Pages 6
J.H. Kwon et al. / journal of prosthodontic research xxx (2018) xxx–xxx
3
Fig. 2. (a) Three-dimensional (3D) facial scan data. (b) Alignment of the dental cast with the facial scan data using an image registration tool. (c) Alignment of the original 3D model with the cone beam computed tomography scan.
Fig. 3. Real-time scan and digital representation of various mandibular motions of dental cast STL data and cone beam computed tomography data reproduced in digital software. (a) Initial opening motion. (b) Translational movement occurs when the opening is large. (c) Right lateral movement; (d) left lateral movement.
Fig. 4. Reference planes and points for data analysis. (a) Frankfort horizontal plane. (b) Hinge axis points of bilateral condyles.
mean coordinates (xa, ya, za) were calculated using the following formula: qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Distance ¼ ðx xa Þ2 þ ðy ya Þ2 þ ðz za Þ2 3. Difference from conventional methods Despite the efforts of many clinicians and researchers, hinge axis recording via facebow is limited using arbitrary methods.
Currently, traditional mechanical articulators cannot accurately reproduce unique mandibular movements in individual patients [6,7,11]. In recent years, there have also been efforts to digitally track the movements of the mandible by introducing various methods: using an optoelectronic system [12,13]; combining an optical camera with magnetic resonance imaging [14]; electromagnetic tracking devices [15]; fluoroscopy [16]; ultrasonic devices [17]; and combining an optical camera with computed tomography data [18]. Most of these
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001
G Model JPOR 476 No. of Pages 6
4
J.H. Kwon et al. / journal of prosthodontic research xxx (2018) xxx–xxx
Fig. 5. Kinematics of the mandibular condyle. All charts, except d and e, are the horizontal view with the hinge axis as the starting point of motion. The horizontal axis of the charts d and e is the Frankfort horizontal plane. (a) Protrusion. (b) Lateral movement to the left side. (c) Lateral movement to the right side. (d) Protrusion. Right sagittal view. (e) Protrusion. Left sagittal view. (f) Protrusion. Enlarged-scale view of chart a. (g) Lateral movement to the left side rearranged with a fixed rotation axis. (h) Lateral movement to the right side rearranged with a fixed rotation axis.
Fig. 6. The tracking stability of (a) the mandibular target attachment site, (b) left condyle tracking point and (c) right condyle tracking point. Changes in the coordinates during the recording time of 450 s using the optical scanner was implemented as a graph (mm).
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001
G Model JPOR 476 No. of Pages 6
J.H. Kwon et al. / journal of prosthodontic research xxx (2018) xxx–xxx
5
Fig. 7. The mean tracking stability of the target tracking system by tracking site (mm).
methods, however, have required the construction of patient-specific transoral appliances and/or heavy extraoral equipment worn on the patient’s head. In contrast, the current tracking method using a facial scanner requiring only the placement of a simple retractor and some small targets, which also makes it useful for the accurate measurement of less-limited mandibular motions. 4. Effect or performance Such a digital approach would have significant potential in many applications. First, by tracking the condyle movement of the mandible using patient-specific CBCT data and a precision optical scanner, we can provide more accurate values to mechanical articulators. In addition, it can be applied to virtual articulators in CAD software, which can overcome the limitations of the existing face-bow technique. When the patient’s gypsum model is superimposed, the patient’s own mandibular movement can be measured on the occlusal plane. This can overcome the limitations of the mechanical articulator. The stability of tracking in this study was 4.36 mm in the mandible target area, 30.78 mm in the left condyle area, and 37.74 mm in the right condyle area (Figs. 6 and 7). The accuracy of the target tracking system, which evaluated whether the distance between the actual targets could be accurately reproduced during static position, lateral movement, protrusive movement and opening movement, also showed a value of 4.1–6.9 mm (Fig. 8). Through this, it can be expected that the reproduction of the condylar movement through this tracking system is precisely possible. This is a small error compared to the total scan accuracy of laboratory scanners used in the digital prosthesis manufacturing process, which is reported to be about 78 mm [19]. In addition, the reproducibility of the condylar movement through target tracking is much smaller than the error of about 752 mm that occurs in the process of positioning the maxillary arch data on the virtual articulator through face-bow transfer, as reported by Solaberrieta et al. [20]. In addition, the analysis of occlusal patterns is possible using this technique. When the patient performs anterior and lateral movements, it is possible to identify which teeth are guided, and through which occlusal interferences can also be detected. This can contribute to more accurate virtual articulation and to fabricating more accurate prostheses when uploaded to dental CAD software [21]. Because the trajectory of mouth opening and closing can be accurately recorded, it can also be useful in the preparation of prostheses that require
Fig. 8. Reference positions for accuracy evaluation and the mean accuracy of the target tracking system by tracking distance between outermost targets. (a) The distance between the actual outermost targets recorded by the optical scanner is measured. (b) The distance between the targets on the tracking data after the CBCT data were superimposed. (c) The mean accuracy data during static position, lateral movement, protrusive movement and opening movement (mm).
changing the occlusal vertical dimension of the patient. This overcomes the limitations of mechanical articulators. 5. Conclusion This article introduced a novel methodology in which mandibular movements are tracked during functional motion, and the actual movement of the mandibular body could be confirmed digitally through the merging of dental cast scan data and CBCT data. It can be used for the fabrication of prostheses, and the diagnosis and treatment of TMJ disease using recordings of the mandibular movement pathway. In particular, the use of only a target sticker, which is simply attached to the labial surface of the anterior tooth, can be used to track mandibular movements, which is very simple compared with current marker technology, and also minimizes patient inconvenience. Acknowledgment This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2017R1D1A1B03034835). References [1] Gärtner C., Kordass B. The virtual articulator: development and evaluation. Int J Comput Dent 2003;6:11–24. [2] Fang JJ, Kuo TH. Modelling of mandibular movement. Comput Biol Med 2008;38:1152–62. [3] Walker PM. Discrepancies between arbitrary and true hinge axes. J Prosthet Dent 1980;43:279–85. [4] Farias-Neto A, Dias AH, de Miranda BF, de Oliveira AR. Face-bow transfer in prosthodontics: a systematic review of the literature. J Oral Rehabil 2013;40:686–92. [5] Yohn K. The face bow is irrelevant for making prostheses and planning orthognathic surgery. J Am Dent Assoc 2016;147:421–6. [6] Bisler A, Bockholt U, Kordass B, Suchan M, Voss G. The virtual articulator. Int J Comput Dent 2002;5:101–6. [7] Kordass B, Gärtner C, Söhnel A, Bisler A, Voss G, Bockholt U, et al. The virtual articulator in dentistry: concept and development. Dent Clin North Am 2002;46:493–506.
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001
G Model JPOR 476 No. of Pages 6
6
J.H. Kwon et al. / journal of prosthodontic research xxx (2018) xxx–xxx
[8] Koralakunte PR, Aljanakh M. The role of virtual articulator in prosthetic and restorative dentistry. J Clin Diagn Res 2014;8:ZE25–8. [9] Solaberrieta E, Garmendia A, Minguez R, Brizuela A, Pradies G. Virtual facebow technique. J Prosthet Dent 2015;114:751–5. [10] Mapelli A, Galante D, Lovecchio N, Sforza C, Ferrario VF. Translation and rotation movements of the mandible during mouth opening and closing. Clin Anat 2009;22:311–8. [11] Pandita A, Dod A, Bhat R. Virtual articulators: a digital excellence in prosthetic and restorative dentistry. J Appl Dent Med Sci 2016;2:110–7. [12] Peck CC, Murray GM, Johnson CW, Klineberg IJ. The variability of condylar point pathways in open-close jaw movements. J Prosthet Dent 1997;77:394– 404. [13] Röhrle O, Waddell JN, Foster KD, Saini H, Pullan AJ. Using a motion-capture system to record dynamic articulation for application in CAD/CAM software. J Prosthodont 2009;18:703–10. [14] Leader JK, Boston JR, Rudy TE, Greco CM, Zaki HS. Relation of jaw sounds and kinematics visualized and quantified using 3-D computer animation. Med Eng Phys 2003;25:191–200. [15] Baeyens JP, Gilomen H, Erdmann B, Clijsen R, Cabri J, Vissers D. In vivo measurement of the 3D kinematics of the temporomandibular joint using miniaturized electromagnetic trackers: technical report. Med Biol Eng Comput 2013;51:479–84.
[16] Chen CC, Lin CC, Chen YJ, Hong SW, Lu TW. A method for measuring threedimensional mandibular kinematics in vivo using single-plane fluoroscopy. Dentomaxillofac Radiol 2013;42:95958184. [17] Sójka A, Huber J, Kaczmarek E, He˛dzelek W. Evaluation of mandibular movement functions using instrumental ultrasound system. J Prosthodont 2017;26:123–8. [18] Chang AR, Han JJ, Kim DS, Yi WJ, Hwang SJ. Evaluation of intra-articular distance narrowing during temporomandibular joint movement in patients with facial asymmetry using 3-dimensional computed tomography image and tracking camera system. J Craniomaxillofac Surg 2015;43:342–8. [19] Jeong ID, Lee JJ, Jeon JH, Kim JH, Kim HY, Kim WC. Accuracy of complete-arch model using an intraoral video scanner: an in vitro study. J Prosthet Dent 2016;115:755–9. [20] Solaberrieta E, Mínguez R, Barrenetxea L, Otegi JR, Szentpétery A. Comparison of the accuracy of a 3-dimensional virtual method and the conventional method for transferring the maxillary cast to a virtual articulator. J Prosthet Dent 2015;113:191–7. [21] Ruge S, Quooss A, Kordass B. Variability of closing movements, dynamic occlusion, and occlusal contact patterns during mastication. Int J Comput Dent 2011;14:119–27.
Please cite this article in press as: J.H. Kwon, et al., A digital approach to dynamic jaw tracking using a target tracking system and a structuredlight three-dimensional scanner, J Prosthodont Res (2018), https://doi.org/10.1016/j.jpor.2018.05.001