3D digital surface imaging for quantification of facial development and asymmetry in juvenile idiopathic arthritis Tron A. Darvann, Per Larsen, Nuno V. Hermann, and Sven Kreiborg 3D digital surface imaging (digital stereophotogrammetry or “3D photography”) is becoming an increasingly popular tool for quantification of the face, in clinical contexts as well as for research. The modality is easy to apply and is free from motion artifacts and harmful radiation. It is an obvious choice for comprehensive documentation and analysis of facial development and treatment progression and outcome in individuals with juvenile idiopathic arthritis (JIA). An acquisition protocol for 3D digital surface imaging of children and adolescents with JIA using a 3dMDtrio stereophotogrammetric system is presented. Methodology for processing and analysis of the acquired surfaces is presented and applied to two patient cases in order to illustrate quantification of facial development and asymmetry progression. It is concluded that surface imaging is a powerful technique for monitoring of facial development and treatment outcome, and it is proposed that the method would be suitable for multi-center comparisons of treatment outcomes. (Semin Orthod 2015; ]:]]]–]]].) & 2015 Elsevier Inc. All rights reserved.
3D
digital surface acquisition systems have become reliable and accurate, and provide detailed shape and color representation of objects in many contexts, including dentistry and medicine.1 Systems based on stereophotogrammetry are very fast (less than 2 ms for an acquisition) thus avoiding motion artifacts. The result is a spatially detailed polygonal mesh with color texture (a color photo “pasted” onto the surface) that provides a sub-millimeter accurate representation of the subject's face. The surface 3D Craniofacial Image Research Laboratory (School of Dentistry, University of Copenhagen; Centre of Head and Orthopaedics, Copenhagen University Hospital Rigshospitalet; and DTU Compute, Technical University of Denmark), Copenhagen, Denmark; Department of Oral and Maxillofacial Surgery, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark; Pediatric Dentistry and Clinical Genetics, School of Dentistry, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark; Department of Clinical Genetics, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark. Correspondence to: Tron A. Darvann, MSc, PhD, 3D Craniofacial Image Research Laboratory, School of Dentistry, University of Copenhagen, Nørre Alle 20, DK 2200, Copenhagen N, Denmark. Email:
[email protected] & 2015 Elsevier Inc. All rights reserved. 1073-8746/15/1801-$30.00/0 http://dx.doi.org/10.1053/j.sodo.2015.02.008
may be stored as documentation or conveniently allow measurement of the face after the patient has left the clinic, for use in e.g., diagnostics, treatment planning, or treatment outcome evaluation. A fairly large body of literature published on measurement of the face using surface imaging deals with population studies,2–5 demonstrating beyond doubt the usability of 3D surface imaging for these purposes. Of these, one study carried out by our research group investigates the facial asymmetry in a population of children with JIA in the temporomandibular joint (TMJ) and compares them to a control population that has no TMJ involvement.4 The study also proposes a method of comparison of the asymmetry in a single individual with JIA to the mean asymmetry in the control population, thus providing a clinical tool for measurement in the individual. Several studies show the use of surface imaging in a clinical context with convincing results.6–8 Main sources of error that typically influence the measurement are (1) the imaging device itself (system accuracy and reproducibility),9 (2) the process of landmarking (the capability of recognizing and marking particular anatomical landmarks on the surface, either directly (manually) by a human operator or by some automatic algorithm),10,11 (3)
Seminars in Orthodontics, Vol ], No ] (), 2015: pp ]]]–]]]
1
2
Darvann et al
deformation of the face due to facial expression,12,13 (4) spatial orientation of the face, and (5) determination of detailed point correspondence (interpolation between landmark locations; necessary in order to obtain values of e.g., development and asymmetry at every spatial location across the face). Error sources (4) and (5) have been less studied but are very dependent on the application. Reproducibility may be assessed by repeated measurement and accuracy by comparison with visual assessment.14,15 Methodology of particular relevance for assessment of soft tissue development may be found in references.16–18
Surface acquisition protocol Our experience since 2007 with surface imaging of children and adolescents with JIA has led to an acquisition protocol as presented in the Table. The protocol is for a 3dMDtrio system (3dMD, Atlanta, GA, USA) which has three camera pairs: one located in front of the patient and two on either side (Fig. 1). The front camera is mounted Table. Protocol for 3D Surface Imaging at School of Dentistry, Faculty of Health and Medical Sciences, University of Copenhagen, Denmark, March 2015 System Space requirement Room lighting Hardware setup Screen windows Calibration Patient chair Patient information Patient preparation Patient positioning
Facial expression Exposures
3dMDtrio stereophotogrammetric system with projection of random light pattern 3.0 3.0 m Ordinary roof lighting 3dMD factory setup The central camera window should be positioned at the central bottom of the screen Every day Chair with electric up/down function; 3601 rotation capability; arm and head rest Date of birth, date of acquisition, body height, and weight should be recorded at each time of image acquisition No clothing around neck; no glasses or jewelry; head band to control hair Natural upright sitting position with the chair in a low position. Natural head to neck posture. The chair is raised to the level where the patient's eyes coincides with the horizontal central grid line in the central camera window on the screen, and the midline of the face should be adjusted according to the vertical grid line in the central camera window The patient is instructed to maintain a relaxed facial expression, light contact on the posterior teeth, and lips in light contact It is recommended that several exposures are made (and reviewed) in the same position, especially when imaging young children
Figure 1. 3D digital surface imaging using the 3dMDtrio stereophotogrammetric system. Compare Table. Letters C indicate cameras (two cameras for geometry and one for color texture in each of the three camera housings), F indicate flashes, M is the monitor showing live images, and S is the system computer.
high, while the side cameras are mounted lower in order to avoid occlusion in the chin area and in order to get good ear coverage. The computer screen showing live images from the cameras is mounted in front of the patient such that he/she can assist in adjustment of head orientation according to vertical/horizontal grid lines superimposed on the screen.
Surface processing and analysis In broad terms, the pipeline of the data handling could be described as consisting of (1) orientation, (2) segmentation, and (3) analysis. The main purpose of orientation is to assure that two (or more) surfaces (e.g., at different time points) have a similar and valid overall relative orientation and is typically carried out by a rigid registration based on landmarks or surface patches. The validity of the relative orientation is achieved through selection of landmarks or surface patches that are “stable” (i.e., fairly unaffected by the temporal progression one wishes to uncover). The main purpose of segmentation is to assign anatomical knowledge to particular points or regions on the surface. One particular type of segmentation is the process of achieving detailed point correspondence19 between the involved surfaces, resulting in all points in the face surfaces having a known correspondence in all other face surfaces. In particular, if detailed point correspondence is also known between the surfaces and an already
3D digital surface imaging
3
segmented reference surface (called an atlas), the atlas segmentation may be automatically transferred to the individual surface. In the analysis step, measurements and analysis may now be carried out meaningfully since, due to the detailed point correspondence, arithmetic (e.g., for surface comparisons) may be carried out point-wise across all involved surfaces.
Examples of quantification of development and asymmetry Case 1 is a girl with JIA and a radiographic diagnosis of unilateral involvement of the TMJ, where surface imaging had taken place at four examinations approximately 1 year apart (age at first surface acquisition 11 years 7 months). A modification of the above pipeline (4,14,15,20–22) was created that included a relative alignment23 of the four surfaces using a “mask” (Fig. 2) representing a noiseless, temporally stable region. Deformation vectors between temporally corresponding points were computed and visualized.24 Fig. 2 shows deformation vectors between first and last examination in Case 1. In addition, Fig. 2 shows a result of the same method applied to a control subject (a girl with JIA without radiographic signs of TMJ affection). Fig. 3 shows the progression of asymmetry in Case 1, with a plot of the mean and standard
Figure 3. Progression of facial asymmetry of Case 1. Top images: 3D facial surfaces in a frontal view, color coded according to magnitude of asymmetry in mm, are shown for four different time points in the same individual with unilateral involvement of the TMJ. Red and blue parts of the color table are being used on opposite sides of the face. Plot: Star symbols indicate the mean value of asymmetry across the entire face region, shown with vertical bars corresponding to 1 SD. Filled circles indicate chin asymmetry. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
deviation of asymmetry as a function of time. Clear differences in the pattern of development may be seen between Case 1 and the control, in particular in the direction of deformation vectors in the chin region that has a large transverse component in Case 1, while it is very vertical in the control. The pattern of asymmetry in Case 1 is similar at all the examinations, but with a magnitude increasing with time.
Conclusion
Figure 2. Left: The “mask” defining the region used for rigid alignment of surfaces of the same individual over time. Middle: Mean facial surface of Case 1 with deformation vectors superimposed demonstrating the development between the first and last examination (age at first examination: 11 years 7 months; time span 4 years 1 month). Color and direction of vectors indicate amount and direction of development, respectively. Blue to red range of the color bar corresponds to ranging from no change to development in excess of 2.5 mm/year. Right: Mean facial surface of the control subject with deformation vectors (age at first examination: 11 years 8 months; time span 3 years 8 months).
A methodology, including an acquisition protocol, suitable for quantification of development and asymmetry in the face of individuals with TMJ disorders due to JIA was presented. 3D surface imaging seems like a tool that is useful for longitudinal monitoring of facial development in individuals with JIA, and could also be expected to be an effective tool for comparison of treatment outcomes between treatment centers.
References 1. Kau CH, Richmond S, Incrapera A, et al. Threedimensional surface acquisition systems for the study of facial morphology and their application to maxillofacial surgery. Int J Med Robot. 2007;3:97–110.
4
Darvann et al
2. Hammond P. The use of 3D face shape modelling in dysmorphology. Arch Dis Child. 2007;92:1120–1126. 3. Douglas TS, Mutsvangwa TE. A review of facial image analysis for delineation of the facial phenotype associated with fetal alcohol syndrome. Am J Med Genet A. 2010;152A:528–536. 4. Demant S, Hermann NV, Darvann TA, et al. 3D analysis of facial asymmetry in subjects with juvenile idiopathic arthritis. Rheumatology (Oxford). 2011;50:586–592. 5. Talbert L, Kau CH, Christou T, et al. A 3D analysis of Caucasian and African American facial morphologies in a US population. J Orthod. 2014;41:19–29. 6. Hajeer MY, Millett DT, Ayoub AF, et al. Applications of 3D imaging in orthodontics: part I. J Orthod. 2004;31:62–70. 7. Moss JP. The use of three-dimensional imaging in orthodontics. Eur J Orthod. 2006;28:416–425. 8. Kau CH, Richmond S. Three-dimensional analysis of facial morphology surface changes in untreated children from 12 to 14 years of age. Am J Orthod Dentofacial Orthop. 2008;134:751–760. 9. Lübbers HT, Medinger L, Kruse A, et al. Precision and accuracy of the 3dMD photogrammetric system in craniomaxillofacial application. J Craniofac Surg. 2010;21: 763–767. 10. Aldridge K, Boyadjiev SA, Capone GT, et al. Precision and error of three-dimensional phenotypic measures acquired from 3dMD photogrammetric images. Am J Med Genet A. 2005;138A:247–253. 11. Gwilliam JR, Cunningham SJ, Hutton T. Reproducibility of soft tissue landmarks on three-dimensional facial scans. Eur J Orthod. 2006;28:408–415. 12. Larsen P, Darvann TA, Hermann NV, et al. Reproducibility of normal facial expression in 3dMD surface scans. In: Takada K, Kreiborg S, eds. In Silico Dentistry—The Evolution of Computational Oral Health Science. Osaka, Japan: Medigit; 2008:71–73. 13. Lübbers HT, Medinger L, Kruse AL, et al. The influence of involuntary facial movements on craniofacial anthropometry: a survey using a three-dimensional photographic system. Br J Oral Maxillofac Surg. 2012;50:171–175. 14. Lanche S, Darvann TA, Ólafsdóttir H, et al. A statistical model of head asymmetry in infants with deformational plagiocephaly. In: Ersbøll BK, Pedersen KS, eds. Image Analysis. Lecture Notes in Computer Science. Berlin, Germany: Springer-Verlag; 2007;4522:898-907.
15. Lanche S, Darvann TA, Ólafsdottir H, et al. Validation of a statistical model of head asymmetry in infants with deformational plagiocephaly. In: Takada K, Kreiborg S, eds. In Silico Dentistry—The Evolution of Computational Oral Health Science. Osaka, Japan: Medigit, 2008:42–46. 16. Hoefert CS, Bacher M, Herberts T, et al. Implementing a superimposition and measurement model for 3D sagittal analysis of therapy-induced changes in facial soft tissue: a pilot study. J Orofac Orthop. 2010;71:221–234. 17. Heike CL, Upson K, Stuhaug E, et al. 3D digital stereophotogrammetry: a practical guide to facial image acquisition. Head Face Med. 2010;6:18. http://dx.doi.org/ 10.1186/1746-160X-6-18. 18. Brons S, van Beusichem ME, Maal TJ, et al. Development and reproducibility of a 3D stereophotogrammetric reference frame for facial soft tissue growth of babies and young children with and without orofacial clefts. Int J Oral Maxillofac Surg. 2013;42:2–8. 19. Hutton TJ, Buxton BF, Hammond P, et al. Estimating average growth trajectories in shape-space using kernel smoothing. IEEE Trans Med Imaging. 2003;22:747–753. 20. Darvann TA, Hermann NV, Demant S, et al. Automated quantification and analysis of facial asymmetry in children with arthritis in the temporomandibular joint. In: Pan X, Liebling M, eds. Proceedings of ISBI 2011: IEEE Computer Society International Symposium on Biomedical Imaging: From Nano to Macro. Chicago; March 30–April 2 2011: 1193–1196. 21. Darvann TA, Ólafsdóttir H, Hermann NV, et al. On the measurement of craniofacial asymmetry. In: Takada K, Kreiborg S, eds. In Silico Dentistry—The Evolution of Computational Oral Health Science. Osaka, Japan: Medigit, 2008:37–41. 22. Darvann TA, Einarsdóttir EB, Larsen P, et al. Assessment of facial asymmetry in a normal population. In: Long R, ed. Proceedings of the 12th International Congress on Cleft Lip/ Palate and Related Craniofacial Anomalies, May 5–10, Orlando, FL, USA, Chapel Hill, NC: American Cleft Palate Craniofacial Association; 2013[17 pages]. 23. Zhang Z. Iterative point matching for registration of freeform curves and surfaces. Int J Comput Vision. 1994; 13:119–152. 24. Darvann TA. Landmarker: a VTK-based tool for landmarking of polygonal surfaces. In: Takada K, Kreiborg S, eds. In Silico Dentistry—The Evolution of Computational Oral Health Science. Osaka, Japan: Medigit; 2008:160–162.