- Email: [email protected]
Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1
G Model
JORMAS-679; No. of Pages 8 J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
Available online at
ScienceDirect www.sciencedirect.com
Original Article
Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1 C. Savoldelli a,*, G. Benat a, L. Castillo a, E. Chamorey b, J.-C. Lutz c a
Department of oral and maxillofacial surgery, head and neck institute, university hospital of Nice, 06000 Nice, France Biostatistic department, centre Antoine-Lacassagne, 06000 Nice, France c Maxillo-facial and plastic surgery department, Strasbourg university hospital, 67000 Strasbourg, France b
A R T I C L E I N F O
A B S T R A C T
Article history: Received 6 March 2019 Accepted 18 March 2019
Introduction: Three-dimensional imaging of facial surfaces is a useful tool in different fields of craniofacial, maxillo-facial and facial aesthetic surgery. Many devices that use several stereoscopic cameras are available but require a dedicated room. In contrast, the Vectra H-1 is a handheld device that can be used regardless of location but requires three consecutive acquisition and might therefore provide less accuracy. The aim of this study was to assess the accuracy, repeatability and reproducibility of the Vectra H1 device to validate its use in daily medical practice. Material and methods: A comparative analysis of the Vectra H1 device and a digital calliper was performed based on 23 distances measured among 11 facial landmarks. One operator repeated the procedure six times on a 24-year-old male volunteer to assess repeatability, and six operators performed the measurement procedure on a 22-year-old female volunteer to assess reproducibility. Repeatability, reproducibility and accuracy versus the distances measured were successively assessed by testing the correlations between the distances measured versus the coefficient of variation (CV) calculated for repeatability, reproducibility and accuracy. Results: The CVs for all distances ranged from 0.34% to 1.53% and decreased linearly when distances measured increased, and this correlation was significant (P = 0.0026) for repeatability. The CVs for all distances ranged from 0.23% to 2.90% and decreased linearly as distances measured increased; there was a significant correlation (P = 0.00045) for reproducibility. Conclusions: This study shows that the Vectra H1 provides an accurate linear assessment of clinical parameters and allows the accurate analysis of craniofacial morphology. Furthermore, this device costs less and requires less space than other multi-pod devices. C 2019 Elsevier Masson SAS. All rights reserved.
Keywords: Stereophotogrammetry 3D facial imaging Measurement error
1. Introduction Facial structure assessment before treatment and during the monitoring stage can be performed by two-dimensional (2D) imaging, but this process has been progressively replaced by threedimensional (3D) imaging. Three-dimensional (3D) facial imaging Abbreviations: 2D, two-dimensional; 3D, three dimensional; CT, computed tomography; VAM, vectra analysis module; SD, standard deviation; M, mean; CV, coefficient of variation; SYST_E, systematic error; JUST, justness (accuracy). * Corresponding author at: University Hospital of Nice, head and neck institute, department of oral and maxillofacial surgery, 31, avenue de Valombrose, 06000 Nice, France. E-mail addresses: [email protected] (C. Savoldelli), [email protected] (G. Benat), [email protected] (L. Castillo), [email protected] (E. Chamorey), [email protected] (J.-C. Lutz).
systems are useful tools that have been developed to objectively evaluate treatment outcomes in different fields of head and face medicine in which soft tissues are affected, such as orthodontics [1,2], dental rehabilitation [3] or cranio-facial [4], maxillo-facial [5–7], facial aesthetic surgery [8] and facial dermatology [9]. The objective assessment of facial shapes has also become increasingly important for research on dysmorphology and genetics in younger patients [10]. The principle underlying these technologies is the comparison of landmark positions, distances (along a straight line or across surfaces between landmarks), surfaces or volumes on a 3D numerical patient model before and after a surgical or medical treatment is performed. Most photogrammetric devices directly generate a 3D model through a unique acquisition thanks to the use of a multi-angle camera system. This technique has several advantages over 3D computed tomography (CT) [11] or laser
https://doi.org/10.1016/j.jormas.2019.03.012 C 2019 Elsevier Masson SAS. All rights reserved. 2468-7855/
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
2
scanning, including the fact that repetitive photographs can be obtained with the eyes open. Furthermore, photogrammetric devices allow the excellent representation of skin texture and colour and require only one image capture for numerical acquisition. These systems have been shown to yield reliable results with good reproducibility and reliability but are bulky and require to be set-up in a dedicated room. The Vectra H11 system is a non-ionizing and handheld device that does not require any specific environment and is more affordable than the previously described systems. However, this system requires 3 consecutive acquisitions to be performed in order to generate a 3D model. Therefore, the process could result in less accuracy and less reproducibility. The aim of this study was to validate the measurements accuracy, repeatability and reproducibility of the Vectra H1 system in clinical conditions. 2. Material and methods Eleven facial landmarks were chosen from the classical cutaneous points defined in cephalometry and twenty-three distances among these facial landmarks were defined (Fig. 1). These distances were measured consecutively with a digital slide calliper (to the nearest one-hundredth of a millimetre) and a Vectra1 H1 handheld scanner (Canfield Scientific, Parsippany, NJ, USA). The repeatability (measured as intra-operator variation), reproducibility (measured as inter-operator variation) and accuracy of the device were assessed for the validation procedure. The series of 11 facial landmarks (Table 1) was marked on the subject’s face using a dermographic pencil. First, the 23 distances were measured in millimetres (mm) with a digital slide calliper. Second, 3D images of facial surfaces were captured with the Vectra H1 in each subject. Three consecutive facial captures were automatically stitched into one 3D image with the Vectra1 software. The
Fig. 1. Eleven facial landmarks were placed, and 23 distances were measured.
Table 1 Description of the 11 facial landmarks. Coding
Points 1 2 3 4 5 6 7 8 9 10 11
Trichion Nasion Right lateral canthus Right medial canthus Left medial canthus Left lateral canthus Subnasale Top upper lip Right cheilion Left cheilion Pogonion
TRI NAS RLC RMC LMC LLC SN TUL RCHa LCH POG
captures were taken after a brief interval of time in motionless subjects with the face in neutral position and the lips closed without tightening (video file). The patients’ eyes were open, with the straight forward gaze, not looking either up or down, the mouth closed, and the face in a relaxed facial expression. The head was in neutral position. The Vectra1 H1 handheld scanner features a targeting system consisting of two converging green lights projected onto the subject’s face. Overlap of both lights indicated the correct shooting distance was obtained. Thanks to this, the position of the 3 pictures could be standardized. Indeed, the first and third pictures were acquired at a 458 angle from the frontal plane towards the left and right sides of the face. From a low angle, the camera aimed at a point centred on the cheekbone. The green dots were aimed at the middle of the patient’s cheek (at the intersection of imaginary lines running from the lateral canthus and subnasale point of the upper lip). The camera was positioned at a 458 angle from the front towards the left and right sides of the face. The second picture was captured in front of the subject, with the lights converging on the philtrum. The camera was positioned directly in front of the face and held level with the patient’s nose. The green dots were aimed between the upper lip and nose (sub-nasal point) at the midline of the patient’s face (video file). The procedure provides the three acquisitions which were stitched into one 3D image with Vectra1 software. Numerical facial landmarks were implemented manually by the operator to overlap previously plotted points in Vectra1 Analysis Module (VAM1) elaboration software (Canfield Scientific, Inc., Fairfield, NJ, USA). The distances were calculated by measuring the straight-line distance between two landmarks regardless of surface topography. Distances in mm were calculated according to p the following formula: d ¼ Dx2 þ Dy2 þ Dz2 . The variation threshold for clinical use was defined as a variation of less than 5% or a difference of less than 1 mm. First, repeatability (intra-operator variability) was defined as follows: one operator repeated the measurement six times on a 24year-old Caucasian male subject (Fig. 2), without changing the measurement conditions over a short period of time. The subsequent findings represented the variability induced only by the Vectra H1. The assumption was that the measured values of the quantitative variable were as close as possible between the 6 measures. Second, reproducibility (inter-operator variability) was defined as follows: six operators (physicians in the department) performed the measurement procedure in random order on a 22-year-old Caucasian female (Fig. 3) without varying the measurement conditions. These results represented the sum of the variability induced by both, the Vectra H1 and the operators. The assumption was that the distances measured by the different operators did not differ. Six operators (physicians in the department) performed the measurement procedure in random order on a 22-year-old
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
3
Fig. 2. Dermographic and numerical landmarks in the male volunteer.
Fig. 3. Dermographic and numerical landmarks in the female volunteers.
Caucasian female (Fig. 3). The straight-line distances measured between two landmarks were analysed as quantitative variables and are summarized in Table 2. Quantitative variables are expressed as the mean (M), standard deviation (SD), range, coefficient of variation CV % ¼ SD M 100, systematic error (SYST_E = mean of Vectra H1’s measurement - calliper’s measurement) E and accuracy ðJUSTE% ¼ cali per0 SYST 100 Differences bes measurement tween the true values measured using a slide callipers and the numerical values measured with the Vectra H1 were analysed using Student’s t-test. The evolution of repeatability, reproducibility and accuracy versus distances measured was successively
assessed by testing the correlations among measured distances versus CV calculated for repeatability, CV calculated for reproducibility and accuracy. The significance of these correlations was evaluated using linear regression. The variability in the differences between the numerical values measured with Vectra H1 and those obtained with the calliper was plotted according to the measurement number. The evolution of this variability over the measurement number was evaluated using a linear mixed model. Differences were statistically significant when P < 0.05. All statistical tests were two-sided. Statistical analyses were performed using R.3.2.3 software.
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
4 Table 2 Description of the 23 distances measured. n
Distances measured
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Trichion Trichion Trichion Trichion Trichion Right lateral canthus Right medial canthus Nasion Left medial canthus Right lateral canthus Right medial canthus Left lateral canthus Left medial canthus Nasion Subnasale Top upper lip Right cheilion Right cheilion Right cheilion Right cheilion Left cheilion Left cheilion Left cheilion
Coding Right lateral canthus Right medial canthus Nasion Left medial canthus Left lateral canthus Right medial canthus Nasion Left medial canthus Left lateral canthus Right cheilion Right cheilion Left cheilion Left cheilion Subnasale Top upper lip Pogonion Subnasale Top upper lip Left cheilion Pogonion Subnasale Top upper lip Pogonion
TRI_RLC TRI_RMC TRI_NAS TRI_LMC TRI_LLC RLC-RMC RMC_NAS NAS_LMC LMC_LLC RLC_RCH RMC_RCH LLC_LCH LMC_LCH NAS_SN SN_TUL TUL_POG RCH_SN RCH_TUL RCH_LCH RCH_POG LCH_SN LCH_TUL LCH_POG
2.1. Ethics Written informed consent was obtained from all subjects prior to their participation in research activities and released image rights and represented an agreement to the use and publication of data. 3. Results 3.1. Repeatability Table 3 shows the results of the different variables studied in the repeatability assessment. The CVs for all distances ranged from
0.34% to 1.53%, with six measurements being greater than 1%. Fig. 4a shows that the CVs decreased linearly when distances measured increased and that this correlation was significant (P = 0.0026). The systematic error (SYST_E) ranged from 1.6 mm to +2.74 mm with a mean (SD) of 0.96 mm (0.69), and the calculated accuracy (JUSTE) ranged from 3.72% to +3.47% for twenty distances. The accuracies for LMC_LLC, RLC_RMC, and RMC_NAS were 6.11, 7.47 and 8%, respectively. The mean (SD) accuracy was 2.28% (2.20). Fig. 4b shows that accuracy improved linearly (P = 0.047) as distances increased. A histogram of the frequencies of differences between the Vectra’s measurements and the true values for each distance (Fig. 4c) shows that the 138 measured distances exhibited a normal distribution bellcurve with a mean of 0.14 mm 1.25 (SD). Fig. 4d shows that there was no significant difference (P = 0.38) in the mean difference between Vectra-obtained measurements and true values during consecutive procedures. 3.2. Reproducibility Table 4 shows the results obtained for different variables studied to assess reproducibility. The CVs for all distances ranged from 0.23% to 2.90%, with nine measurements greater than 1%. Fig. 5a shows that the CVs decreased linearly as distances measured increased and that this correlation was significant (P = 0.00045). The systematic error (SYST_E) ranged from 0.52 mm to +1.22 mm with a mean (SD) of 0.53 mm (0.43) and a calculated accuracy (JUSTE) ranging from -1.62% to +5.93% for all distances. Fig. 5b shows that accuracy significantly and linearly improved (P = 0.0025) as distances increased. A histogram of the frequencies of the differences between Vectra-obtained measurements and the true values for each distance (Fig. 5c) shows that the 138 distances measured exhibited a normal distribution bell-curve with a mean (SD) of 0.39 1 mm. Fig. 5d shows that there was no significant difference (P = 0.30) in the mean difference between Vectra-obtained measurements and true values obtained during consecutive procedures.
Table 3 Repeatability (True value: measurements with the slide caliper, mean: mean of 6 times measurement by one operator; Min: minimum value measured; Max: maximum value measured; SD: standard deviation; CV: coefficient of variation, SYST_E: systematic error or absolute error in millimetres; JUSTE: accuracy measurement in percent; t-test (P)). n
Measure
True_Value
Mean
Min
Max
SD
CV
SYST_E (mm)
JUSTE (%)
P t-test
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Mean of absolute values (SD)
TRI_RLC TRI_RMC TRI_NAS TRI_LMC TRI_LLC RLC_RMC RMC_NAS NAS_LMC LMC_LLC RLC_RCH RMC_RCH LLC_LCH LMC_LCH NAS_SN SN_TUL TUL_POG RCH_SN RCH_TUL RCH_LCH RCH_POG LCH_SN LCH_TUL LCH_POG
86.75 66.80 56.31 67.78 84.80 36.68 21.49 24.22 36.50 70.09 73.69 75.76 74.71 57.45 16.42 46.73 42.52 36.42 61.37 50.80 43.94 37.87 52.57
87.30 65.47 55.24 67.36 86.05 39.42 23.21 25.06 38.73 71.53 73.83 75.60 74.23 57.30 16.77 47.84 42.93 36.35 59.77 49.88 42.95 36.46 51.78
86.57 64.87 54.36 66.77 85.14 38.98 22.75 24.78 38.40 70.78 73.41 75.23 73.45 56.60 16.47 46.93 42.15 35.70 58.99 49.42 42.38 35.95 51.38
88.37 66.53 56.71 68.35 86.82 39.83 23.48 25.40 39.37 72.47 74.27 75.96 74.89 57.74 17.18 48.74 43.44 37.00 60.30 50.66 43.58 36.88 52.36
0.66 0.58 0.84 0.56 0.74 0.35 0.27 0.24 0.33 0.59 0.35 0.26 0.50 0.39 0.24 0.66 0.51 0.47 0.46 0.46 0.40 0.35 0.35
0.75 0.88 1.53 0.83 0.86 0.89 1.16 0.95 0.85 0.83 0.48 0.34 0.68 0.68 1.46 1.39 1.18 1.29 0.77 0.91 0.93 0.95 0.69 0.92 (0.29)
0.55 1.33 1.07 0.42 1.25 2.74 1.72 0.84 2.23 1.44 0.14 0.16 0.48 0.15 0.35 1.11 0.41 0.07 1.60 0.92 0.99 1.41 0.79 0.96 (0.69)
0.63 1.99 1.90 0.62 1.47 7.47 8.00 3.47 6.11 2.05 0.19 0.21 0.64 0.26 2.13 2.38 0.96 0.19 2.61 1.81 2.25 3.72 1.50 2.28 (2.20)
0.09672 0.00248 0.02625 0.12561 0.00902 0.00001 0.00002 0.00036 0.00001 0.00188 0.37218 0.19207 0.06544 0.38939 0.01601 0.00918 0.10603 0.73017 0.00037 0.00448 0.00176 0.00018 0.00265
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
5
Fig. 4. a: Repeatability: coefficients of variation (CV) decreased linearly as distances measured increased, and this correlation was significant (P = 0.0026); b: Repeatability: accuracy improved linearly (P = 0.047) as distances increased; c: Repeatability: histogram of frequencies of differences between Vectra’s measurements and true values for each distance; d: Repeatability: mean differences between Vectra’s measurements and true values obtained during consecutive procedures.
4. Discussion The evaluation and analysis of facial soft tissues is an essential part of orthodontic and maxillofacial diagnosis and treatment planning. Among the various techniques used for 3D surface imaging and patient data acquisition, 3D photogrammetry is attractive as it rapidly provides 3D models from 2D pictures and does not generate any radiation. Three-dimensional (3D) stereophotogrammetry systems based on multiple pod camera setups have attracted increasing interest. Indeed, they are quite convenient for the routine analysis of facial soft tissue outcomes in different fields of medicine and surgery. Tripod camera devices are stationary and allow capture of the model through a single acquisition. However, their drawback is the requirement of a dedicated room. Moreover, pod devices are expensive and require frequent calibration [12]. The handheld Vectra-H1 3D facial imaging system is a specific double-lens reflex camera device that brings a solution to space limitations but requires a three-shot procedure. The procedure may represent a source of errors caused by patients who do not remain strictly motionless or a badly angled
camera during consecutive captures. Hence, our aim was to ensure that this medical device meets the user’s needs and that it consistently provides the intended medical benefit in clinical conditions. This process is usually achieved by tests, inspections, and, in some cases, analyses. The goal of validation is to ensure that the user’s needs are met when using a medical device and that the device consistently provides the intended medical benefit under actual-use conditions. Camison et al. [13] have proposed a validation for the Vectra-H1 but compared the digital measurements obtained by Vectra H1 to those obtained using another numerical system, the 3dMDface1. The authors found that these devices were highly comparable. Repeatability was assessed on living subjects, while reproducibility was tested on a mannequin. The authors chose the 3dMDface numerical system as a reference because of its accuracy and precision has been validated by a number of investigators, the studies uniformly showing sub-millimeter global error margins. Gibelli et al. [14] and Kim et al. [15] validated the portable Vectra H1 device in a comparison with a static Vectra M3 device in which they analysed linear, angular, surface area and volume measure-
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
6
Table 4 Reproducibility (True value: measurements with the slide caliper, mean of distances measured by six operators; Min: minimum value measured; Max: maximum value measured; SD: standard deviation; CV: coefficient of variation; SYST_E: systematic error or absolute error in millimetres; JUSTE: accuracy measurement in percent; t-test (P)). N
Measure
True_Value
Mean
Min
Max
SD
CV
SYST_E
JUSTE
p (t_test)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Mean (SD)
TRI_RLC TRI_RMC TRI_NAS TRI_LMC TRI_LLC RLC_RMC RMC_NAS NAS_LMC LMC_LLC RLC_RCH RMC_RCH LLC_LCH LMC_LCH NAS_SN SN_TUL TUL_POG RCH_SN RCH_TUL RCH_LCH RCH_POG LCH_SN LCH_TUL LCH_POG
81.00 61.00 47.80 62.60 82.25 39.67 20.59 21.98 39.41 68.74 61.54 68.00 61.87 51.67 11.66 33.03 35.61 32.06 53.04 36.90 35.85 32.67 38.07
80.64 61.24 47.75 62.77 82.36 39.92 21.81 22.23 39.81 69.15 62.34 68.11 62.52 53.09 12.29 34.63 35.24 31.54 53.69 37.88 35.90 32.47 38.81
80.29 60.89 47.42 62.50 81.40 39.73 21.46 21.79 39.59 68.16 61.30 67.29 62.05 52.39 11.78 34.09 34.69 30.99 52.85 37.25 35.42 31.85 38.22
80.81 61.89 48.21 63.09 83.03 40.10 22.47 22.55 39.95 69.82 63.29 69.04 62.80 53.65 12.74 35.56 35.59 31.95 54.47 38.44 36.44 32.80 39.27
0.19 0.37 0.30 0.28 0.61 0.16 0.35 0.26 0.13 0.56 0.70 0.57 0.29 0.49 0.36 0.55 0.34 0.40 0.53 0.41 0.47 0.37 0.35
0.23 0.60 0.63 0.44 0.74 0.39 1.60 1.16 0.32 0.81 1.12 0.83 0.46 0.93 2.90 1.60 0.96 1.25 0.99 1.08 1.30 1.14 0.91 0.97 (0.56)
-0.36 0.24 -0.05 0.17 0.11 0.25 1.22 0.25 0.40 0.41 0.80 0.11 0.65 1.42 0.63 1.60 -0.37 -0.52 0.65 0.98 0.05 -0.20 0.74 0.53 (0.43)
-0.44 0.39 -0.10 0.27 0.13 0.63 5.93 1.14 1.01 0.60 1.30 0.16 1.05 2.75 5.40 4.84 -1.04 -1.62 1.23 2.66 0.14 -0.61 1.94 1.54 (1.7)
0.00563 0.17296 0.69999 0.19712 0.67715 0.01228 0.00036 0.06513 0.00065 0.13289 0.03802 0.65634 0.00274 0.00086 0.00781 0.00084 0.04458 0.02442 0.02996 0.00206 0.80480 0.24278 0.00353
ments. That study assessed only the repeatability of the Vectra H1 device and not its reproducibility. Although both previous studies provided important contributions regarding the reliability of the Vectra H1, some aspects related to its repeatability and reproducibility under clinical-use conditions were not fully addressed. Our methodology, in which we compared numerical measurements obtained by a Vectra H1 and direct values (distances measured with a digital slide calliper) only on living subjects only, appear more suitable for the refine an assessment of this specific device. Previous studies [16–21] were performed to investigate the accuracy of other stereophotogrammetry commercial systems such as the 3dMDface System [3dMD LLC, Atlanta, USA], 3DSensoren FaceSCAN [3D Shape GmbH, Erlangen, Germany], and Di3D [Dimensional Imaging, Glasgow, UK]). These sutdies focused on the reliability of these tools for distance measurements. To the best of our knowledge, no study has investigated the repeatability, reproducibility and accuracy of the Vectra Analysis Module1 (VAM) software by comparing data collected by traditional methods (e.g., direct anthropometry) to data obtained by the Vectra H1. Our study showed that the Vectra H1 was highly accurate as it showed negligible systematic error. Regarding repeatability and reproducibility, the systematic error (respectively standard deviation) was 0.96 mm (0.89) and 0.53 mm (0.43), respectively, which are fully acceptable for clinical practice. This study describes the results of a previous study of a tripod Vectra device [22]. In clinical use, it is very important that the registration of the 3D facial photographs provides a precise accuracy within 1 mm. Our study found the same accuracy, while repeatability and reproducibility drastically improved as distances increased. The mean error can be affected by several factors that can influence the registration accuracy obtained by the acquisition system. One of these factors was the ability to capture the same facial expression of the subject throughout the whole assessment procedure. Interestingly, our study found no significant difference in the mean differences between Vectra measurements and the true values during consecutive procedures (Fig. 5d). The hypothesis was that the
dermographic points were imprecise and have generated errors when measuring short distances with a caliper. Indeed, the caliper was not precise enough to measure the distances considering the center of every point, whereas the magnification tool included in the measurement software ensured that the middle of the mark was easily identified with the Vector H1 (Fig. 6). Moreover, in this study, we compared straight distances between two points regardless of the surface area. A study based on surface-weighted distances [11] should support this validation by comparing tape measurements to numerical surface measurements obtained with a Vectra H1. Fig. 4d shows that over time and after consecutive repeated procedures, there was no improvement in accuracy. This indicates that there was no learning curve for using the Vectra H1, suggesting that it could be made available to non-initiated practitioners without the requirement for any complex training. Although our study found that there was no learning curve and that the three consecutive captures did not affect the accuracy, this device might be of uneasy use in toddlers and infants. Indeed, this system requires a motionless subject who can maintain the face in a neutral position with lips closed (without tightening) to avoid artefacts during the image stitching procedure. Hence, it cannot be used for surgical planning or syndrome assessment in young patients who cannot be fully compliant. Otherwise, the use of 3D digital photogrammetry may substantially reduce the clinical chair time and provide flexibility to the user for post-chair analyses using the software. Indeed, the user would be able to digitally recreate, store, manipulate and reanalyse the 3D facial profile of the patient. Kim et al. [15] showed that 8 minutes were needed per subject to perform a direct anthropometry. The time to perform the measurements using the handheld camera (including image capture, render, landmark placement and distance calculation) was 3 minutes per subject and the time to perform the conventional camera measurement (including image capture, render, landmark placement and distance calculation) was also approximately 4 minutes per subject.
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
7
Fig. 5. a: Reproducibility: coefficients of variation (CV) decreased linearly as distances measured increased, and this correlation was significant (P = 0.00045); b: Reproducibility: accuracy improved linearly (P = 0.0025) as distances increased; c: Reproducibility: histogram of frequencies of differences between Vectra’s measurements and true values for each distance; d: Reproducibility: there was no significant difference (P = 0.30) in the mean differences between Vectra’s measurements and true values obtained during consecutive procedures.
In a future perspective with technological evolution, Vectra H1 should be replaced by smartphones but actual applications are still lower than that of dedicated medical devices. Within a short period of time, industries and manufacturers are likely to develop mobile [23] technologies that can achieve high-definition 3D photography.
5. Conclusions Virtual 3D models generated through the Vectra H1 provide a high level of technical precision regarding landmark identification. Test and re-test reliability data demonstrated that the findings obtained using Vectra H1 did not change over time. The results of this study show that Vectra H1 provides an accurate linear assessment of clinical parameters by analysing facial morphology with good accuracy, at a lower cost and while requiring less space than multi-pod devices. Disclosure of interest The authors declare that they have no competing interest.
Acknowledgment
Fig. 6. Non-strict superimposition of the centres of dermographic (circled in red) and numerical facial landmarks (Nasion).
This study was declared to the Commission Nationale Informatique et Liberte´ (CNIL No. 2209316) and both volunteers agreed that their data could be used for biomedical study.
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
8
Appendix A. Supplementary data Supplementary material related to this article can be found, in the online version, at https://doi.org/10.1016/j.jormas.2019.03. 012. References [1] Masoud MI, Bansal N, Castillo J , Manosudprasit A, Allareddy V, Haghi A, et al. 3D dentofacial photogrammetry reference values: a novel approach to orthodontic diagnosis. Eur J Orthod 2017. http://dx.doi.org/10.1093/ejo/cjw055. [2] Manosudprasit A, Haghi A, Allareddy V, Masoud MI. Diagnosis and treatment planning of orthodontic patients with 3-dimensional dentofacial records. Am J Orthod Dentofac Orthop 2017. http://dx.doi.org/10.1016/j.ajodo.2016.10.037. [3] Hanawa S, Kitaoka A, Koyama S, Sasaki K. Influence of maxillary obturator prostheses on facial morphology in patients with unilateral maxillary defects. J Prosthet Dent 2015. http://dx.doi.org/10.1016/j.prosdent.2014.06.016. [4] Mertens C, Wessel E, Berger M, Ristow O, Hoffmann J, Kansy K, et al. The value of three-dimensional photogrammetry in isolated sagittal synostosis: impact of age and surgical technique on intracranial volume and cephalic index – a retrospective cohort study. J Cranio-Maxillofacial Surg 2017. http://dx.doi.org/ 10.1016/j.jcms.2017.09.019. [5] Choi JW, Lee JY, Oh TS, Kwon SM, Yang SJ, Koh KS. Frontal soft tissue analysis using a 3 dimensional camera following two-jaw rotational orthognathic surgery in skeletal class III patients. J Cranio-Maxillofacial Surg 2014. http://dx.doi.org/10.1016/j.jcms.2013.05.004. [6] Jung J, Lee C-H, Lee J-W, Choi B-J. Three dimensional evaluation of soft tissue after orthognathic surgery. Head Face Med 2018;14:21. [7] Terzic A, Schouman T, Scolozzi P. Accuracy of morphological simulation for orthognatic surgery. Assessment of a 3D image fusion software. Rev Stomatol Chir Maxillofac Chir Orale 2013. [8] Verhulst A, Hol M, Vreeken R, Becking A, Ulrich D, Maal T. Three-dimensional imaging of the face: a comparison between three different imaging modalities. Aesthetic Surg J 2018. http://dx.doi.org/10.1093/asj/sjx227. [9] Hermans DJJ, Maal TJJ, Berge´ SJ, Van Der Vleuten CJM. Three-dimensional stereophotogrammetry: A novel method in volumetric measurement of infantile hemangioma. Pediatr Dermatol 2014. http://dx.doi.org/10.1111/ pde.12224. [10] Heike CL, Upson K, Stuhaug E, Weinberg SM. 3D digital stereophotogrammetry: a practical guide to facial image acquisition. Head Face Med 2010. http:// dx.doi.org/10.1186/1746-160X-6-18. [11] Lutz JC, Hostettler A, Agnus V, Nicolau S, George D, Soler LRY. A new software suite in orthognathic surgery: patient specific modeling, simulation and navigation. Surg Innov 2018;1:1553.
[12] Tzou CH, Frey M. Evolution of 3D surface imaging systems in facial plastic surgery. Facial Plast Surg Clin North Am 2011. http://dx.doi.org/10.1016/ j.fsc.2011.07.003. [13] Camison L, Bykowski M, Lee WW, Carlson JC, Roosenboom J, Goldstein JA, et al. Validation of the Vectra H1 portable three-dimensional photogrammetry system for facial imaging. Int J Oral Maxillofac Surg 2017. http://dx.doi.org/ 10.1016/j.ijom.2017.08.008. [14] Gibelli D, Pucciarelli V, Cappella A, Dolci C, Sforza C. Are portable stereophotogrammetric devices reliable in facial imaging? a validation study of Vectra H1 Device. J Oral Maxillofac Surg 2018. http://dx.doi.org/10.1016/ j.joms.2018.01.021. [15] Kim AJ, Gu D, Chandiramani R, Linjawi I, Deutsch ICK, Allareddy V, et al. Accuracy and reliability of digital craniofacial measurements using a smallformat, handheld 3D camera. Orthod Craniofacial Res 2018. http://dx.doi.org/ 10.1111/ocr.12228. [16] Naini FB, Akram S, Kepinska J, Garagiola U, McDonald F, Wertheim D. Validation of a new three-dimensional imaging system using comparative craniofacial anthropometry. Maxillofac Plast Reconstr Surg 2017. http://dx.doi.org/ 10.1186/s40902-017-0123-3. [17] Knoops PGM, Beaumont CAA, Borghi A, Rodriguez-Florez N, Breakey RWF, Rodgers W, et al. Comparison of three-dimensional scanner systems for craniomaxillofacial imaging. J Plast Reconstr Aesthetic Surg 2017. http:// dx.doi.org/10.1016/j.bjps.2016.12.015. [18] Nord F, Ferjencik R, Seifert B, Lanzer M, Gander T, Matthews F, et al. The 3dMD photogrammetric photo system in cranio-maxillofacial surgery: Validation of interexaminer variations and perceptions. J Cranio-Maxillofacial Surg 2015. http://dx.doi.org/10.1016/j.jcms.2015.08.017. [19] Ort R, Metzler P, Kruse AL, Matthews F, Zemann W, Gra¨tz KW, et al. The Reliability of a three-dimensional photo system- (3dMDface-) based evaluation of the face in cleft lip infants. Plast Surg Int 2012. http://dx.doi.org/ 10.1155/2012/138090. [20] Secher JJ, Darvann TA, Pinholt EM. Accuracy and reproducibility of the DAVID SLS-2 scanner in three-dimensional facial imaging. J Cranio-Maxillofacial Surg 2017. http://dx.doi.org/10.1016/j.jcms.2017.07.006. [21] Khambay B, Nairn N, Bell A, Miller J, Bowman A, Ayoub AF. Validation and reproducibility of a high-resolution three-dimensional facial imaging system. Br J Oral Maxillofac Surg 2008;46:27–32. http://dx.doi.org/10.1016/ j.bjoms.2007.04.017. [22] Metzler P, Sun Y, Zemann W, Bartella A, Lehner M, Obwegeser JA, et al. Validity of the 3D VECTRA photogrammetric surface imaging system for cranio-maxillofacial anthropometric measurements. Oral Maxillofac Surg 2014;18:297– 304. http://dx.doi.org/10.1007/s10006-013-0404-7 [nto.]. [23] Salazar-Gamarra R, Seelaus R, Da Silva JVL, Da Silva AM, Dib LL. Monoscopic photogrammetry to obtain 3D models by a mobile device: A method for making facial prostheses. J Otolaryngol - Head Neck Surg 2016. http:// dx.doi.org/10.1186/s40463-016-0145-3.
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
JORMAS-679; No. of Pages 8 J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
Available online at
ScienceDirect www.sciencedirect.com
Original Article
Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1 C. Savoldelli a,*, G. Benat a, L. Castillo a, E. Chamorey b, J.-C. Lutz c a
Department of oral and maxillofacial surgery, head and neck institute, university hospital of Nice, 06000 Nice, France Biostatistic department, centre Antoine-Lacassagne, 06000 Nice, France c Maxillo-facial and plastic surgery department, Strasbourg university hospital, 67000 Strasbourg, France b
A R T I C L E I N F O
A B S T R A C T
Article history: Received 6 March 2019 Accepted 18 March 2019
Introduction: Three-dimensional imaging of facial surfaces is a useful tool in different fields of craniofacial, maxillo-facial and facial aesthetic surgery. Many devices that use several stereoscopic cameras are available but require a dedicated room. In contrast, the Vectra H-1 is a handheld device that can be used regardless of location but requires three consecutive acquisition and might therefore provide less accuracy. The aim of this study was to assess the accuracy, repeatability and reproducibility of the Vectra H1 device to validate its use in daily medical practice. Material and methods: A comparative analysis of the Vectra H1 device and a digital calliper was performed based on 23 distances measured among 11 facial landmarks. One operator repeated the procedure six times on a 24-year-old male volunteer to assess repeatability, and six operators performed the measurement procedure on a 22-year-old female volunteer to assess reproducibility. Repeatability, reproducibility and accuracy versus the distances measured were successively assessed by testing the correlations between the distances measured versus the coefficient of variation (CV) calculated for repeatability, reproducibility and accuracy. Results: The CVs for all distances ranged from 0.34% to 1.53% and decreased linearly when distances measured increased, and this correlation was significant (P = 0.0026) for repeatability. The CVs for all distances ranged from 0.23% to 2.90% and decreased linearly as distances measured increased; there was a significant correlation (P = 0.00045) for reproducibility. Conclusions: This study shows that the Vectra H1 provides an accurate linear assessment of clinical parameters and allows the accurate analysis of craniofacial morphology. Furthermore, this device costs less and requires less space than other multi-pod devices. C 2019 Elsevier Masson SAS. All rights reserved.
Keywords: Stereophotogrammetry 3D facial imaging Measurement error
1. Introduction Facial structure assessment before treatment and during the monitoring stage can be performed by two-dimensional (2D) imaging, but this process has been progressively replaced by threedimensional (3D) imaging. Three-dimensional (3D) facial imaging Abbreviations: 2D, two-dimensional; 3D, three dimensional; CT, computed tomography; VAM, vectra analysis module; SD, standard deviation; M, mean; CV, coefficient of variation; SYST_E, systematic error; JUST, justness (accuracy). * Corresponding author at: University Hospital of Nice, head and neck institute, department of oral and maxillofacial surgery, 31, avenue de Valombrose, 06000 Nice, France. E-mail addresses: [email protected] (C. Savoldelli), [email protected] (G. Benat), [email protected] (L. Castillo), [email protected] (E. Chamorey), [email protected] (J.-C. Lutz).
systems are useful tools that have been developed to objectively evaluate treatment outcomes in different fields of head and face medicine in which soft tissues are affected, such as orthodontics [1,2], dental rehabilitation [3] or cranio-facial [4], maxillo-facial [5–7], facial aesthetic surgery [8] and facial dermatology [9]. The objective assessment of facial shapes has also become increasingly important for research on dysmorphology and genetics in younger patients [10]. The principle underlying these technologies is the comparison of landmark positions, distances (along a straight line or across surfaces between landmarks), surfaces or volumes on a 3D numerical patient model before and after a surgical or medical treatment is performed. Most photogrammetric devices directly generate a 3D model through a unique acquisition thanks to the use of a multi-angle camera system. This technique has several advantages over 3D computed tomography (CT) [11] or laser
https://doi.org/10.1016/j.jormas.2019.03.012 C 2019 Elsevier Masson SAS. All rights reserved. 2468-7855/
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
2
scanning, including the fact that repetitive photographs can be obtained with the eyes open. Furthermore, photogrammetric devices allow the excellent representation of skin texture and colour and require only one image capture for numerical acquisition. These systems have been shown to yield reliable results with good reproducibility and reliability but are bulky and require to be set-up in a dedicated room. The Vectra H11 system is a non-ionizing and handheld device that does not require any specific environment and is more affordable than the previously described systems. However, this system requires 3 consecutive acquisitions to be performed in order to generate a 3D model. Therefore, the process could result in less accuracy and less reproducibility. The aim of this study was to validate the measurements accuracy, repeatability and reproducibility of the Vectra H1 system in clinical conditions. 2. Material and methods Eleven facial landmarks were chosen from the classical cutaneous points defined in cephalometry and twenty-three distances among these facial landmarks were defined (Fig. 1). These distances were measured consecutively with a digital slide calliper (to the nearest one-hundredth of a millimetre) and a Vectra1 H1 handheld scanner (Canfield Scientific, Parsippany, NJ, USA). The repeatability (measured as intra-operator variation), reproducibility (measured as inter-operator variation) and accuracy of the device were assessed for the validation procedure. The series of 11 facial landmarks (Table 1) was marked on the subject’s face using a dermographic pencil. First, the 23 distances were measured in millimetres (mm) with a digital slide calliper. Second, 3D images of facial surfaces were captured with the Vectra H1 in each subject. Three consecutive facial captures were automatically stitched into one 3D image with the Vectra1 software. The
Fig. 1. Eleven facial landmarks were placed, and 23 distances were measured.
Table 1 Description of the 11 facial landmarks. Coding
Points 1 2 3 4 5 6 7 8 9 10 11
Trichion Nasion Right lateral canthus Right medial canthus Left medial canthus Left lateral canthus Subnasale Top upper lip Right cheilion Left cheilion Pogonion
TRI NAS RLC RMC LMC LLC SN TUL RCHa LCH POG
captures were taken after a brief interval of time in motionless subjects with the face in neutral position and the lips closed without tightening (video file). The patients’ eyes were open, with the straight forward gaze, not looking either up or down, the mouth closed, and the face in a relaxed facial expression. The head was in neutral position. The Vectra1 H1 handheld scanner features a targeting system consisting of two converging green lights projected onto the subject’s face. Overlap of both lights indicated the correct shooting distance was obtained. Thanks to this, the position of the 3 pictures could be standardized. Indeed, the first and third pictures were acquired at a 458 angle from the frontal plane towards the left and right sides of the face. From a low angle, the camera aimed at a point centred on the cheekbone. The green dots were aimed at the middle of the patient’s cheek (at the intersection of imaginary lines running from the lateral canthus and subnasale point of the upper lip). The camera was positioned at a 458 angle from the front towards the left and right sides of the face. The second picture was captured in front of the subject, with the lights converging on the philtrum. The camera was positioned directly in front of the face and held level with the patient’s nose. The green dots were aimed between the upper lip and nose (sub-nasal point) at the midline of the patient’s face (video file). The procedure provides the three acquisitions which were stitched into one 3D image with Vectra1 software. Numerical facial landmarks were implemented manually by the operator to overlap previously plotted points in Vectra1 Analysis Module (VAM1) elaboration software (Canfield Scientific, Inc., Fairfield, NJ, USA). The distances were calculated by measuring the straight-line distance between two landmarks regardless of surface topography. Distances in mm were calculated according to p the following formula: d ¼ Dx2 þ Dy2 þ Dz2 . The variation threshold for clinical use was defined as a variation of less than 5% or a difference of less than 1 mm. First, repeatability (intra-operator variability) was defined as follows: one operator repeated the measurement six times on a 24year-old Caucasian male subject (Fig. 2), without changing the measurement conditions over a short period of time. The subsequent findings represented the variability induced only by the Vectra H1. The assumption was that the measured values of the quantitative variable were as close as possible between the 6 measures. Second, reproducibility (inter-operator variability) was defined as follows: six operators (physicians in the department) performed the measurement procedure in random order on a 22-year-old Caucasian female (Fig. 3) without varying the measurement conditions. These results represented the sum of the variability induced by both, the Vectra H1 and the operators. The assumption was that the distances measured by the different operators did not differ. Six operators (physicians in the department) performed the measurement procedure in random order on a 22-year-old
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
3
Fig. 2. Dermographic and numerical landmarks in the male volunteer.
Fig. 3. Dermographic and numerical landmarks in the female volunteers.
Caucasian female (Fig. 3). The straight-line distances measured between two landmarks were analysed as quantitative variables and are summarized in Table 2. Quantitative variables are expressed as the mean (M), standard deviation (SD), range, coefficient of variation CV % ¼ SD M 100, systematic error (SYST_E = mean of Vectra H1’s measurement - calliper’s measurement) E and accuracy ðJUSTE% ¼ cali per0 SYST 100 Differences bes measurement tween the true values measured using a slide callipers and the numerical values measured with the Vectra H1 were analysed using Student’s t-test. The evolution of repeatability, reproducibility and accuracy versus distances measured was successively
assessed by testing the correlations among measured distances versus CV calculated for repeatability, CV calculated for reproducibility and accuracy. The significance of these correlations was evaluated using linear regression. The variability in the differences between the numerical values measured with Vectra H1 and those obtained with the calliper was plotted according to the measurement number. The evolution of this variability over the measurement number was evaluated using a linear mixed model. Differences were statistically significant when P < 0.05. All statistical tests were two-sided. Statistical analyses were performed using R.3.2.3 software.
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
4 Table 2 Description of the 23 distances measured. n
Distances measured
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Trichion Trichion Trichion Trichion Trichion Right lateral canthus Right medial canthus Nasion Left medial canthus Right lateral canthus Right medial canthus Left lateral canthus Left medial canthus Nasion Subnasale Top upper lip Right cheilion Right cheilion Right cheilion Right cheilion Left cheilion Left cheilion Left cheilion
Coding Right lateral canthus Right medial canthus Nasion Left medial canthus Left lateral canthus Right medial canthus Nasion Left medial canthus Left lateral canthus Right cheilion Right cheilion Left cheilion Left cheilion Subnasale Top upper lip Pogonion Subnasale Top upper lip Left cheilion Pogonion Subnasale Top upper lip Pogonion
TRI_RLC TRI_RMC TRI_NAS TRI_LMC TRI_LLC RLC-RMC RMC_NAS NAS_LMC LMC_LLC RLC_RCH RMC_RCH LLC_LCH LMC_LCH NAS_SN SN_TUL TUL_POG RCH_SN RCH_TUL RCH_LCH RCH_POG LCH_SN LCH_TUL LCH_POG
2.1. Ethics Written informed consent was obtained from all subjects prior to their participation in research activities and released image rights and represented an agreement to the use and publication of data. 3. Results 3.1. Repeatability Table 3 shows the results of the different variables studied in the repeatability assessment. The CVs for all distances ranged from
0.34% to 1.53%, with six measurements being greater than 1%. Fig. 4a shows that the CVs decreased linearly when distances measured increased and that this correlation was significant (P = 0.0026). The systematic error (SYST_E) ranged from 1.6 mm to +2.74 mm with a mean (SD) of 0.96 mm (0.69), and the calculated accuracy (JUSTE) ranged from 3.72% to +3.47% for twenty distances. The accuracies for LMC_LLC, RLC_RMC, and RMC_NAS were 6.11, 7.47 and 8%, respectively. The mean (SD) accuracy was 2.28% (2.20). Fig. 4b shows that accuracy improved linearly (P = 0.047) as distances increased. A histogram of the frequencies of differences between the Vectra’s measurements and the true values for each distance (Fig. 4c) shows that the 138 measured distances exhibited a normal distribution bellcurve with a mean of 0.14 mm 1.25 (SD). Fig. 4d shows that there was no significant difference (P = 0.38) in the mean difference between Vectra-obtained measurements and true values during consecutive procedures. 3.2. Reproducibility Table 4 shows the results obtained for different variables studied to assess reproducibility. The CVs for all distances ranged from 0.23% to 2.90%, with nine measurements greater than 1%. Fig. 5a shows that the CVs decreased linearly as distances measured increased and that this correlation was significant (P = 0.00045). The systematic error (SYST_E) ranged from 0.52 mm to +1.22 mm with a mean (SD) of 0.53 mm (0.43) and a calculated accuracy (JUSTE) ranging from -1.62% to +5.93% for all distances. Fig. 5b shows that accuracy significantly and linearly improved (P = 0.0025) as distances increased. A histogram of the frequencies of the differences between Vectra-obtained measurements and the true values for each distance (Fig. 5c) shows that the 138 distances measured exhibited a normal distribution bell-curve with a mean (SD) of 0.39 1 mm. Fig. 5d shows that there was no significant difference (P = 0.30) in the mean difference between Vectra-obtained measurements and true values obtained during consecutive procedures.
Table 3 Repeatability (True value: measurements with the slide caliper, mean: mean of 6 times measurement by one operator; Min: minimum value measured; Max: maximum value measured; SD: standard deviation; CV: coefficient of variation, SYST_E: systematic error or absolute error in millimetres; JUSTE: accuracy measurement in percent; t-test (P)). n
Measure
True_Value
Mean
Min
Max
SD
CV
SYST_E (mm)
JUSTE (%)
P t-test
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Mean of absolute values (SD)
TRI_RLC TRI_RMC TRI_NAS TRI_LMC TRI_LLC RLC_RMC RMC_NAS NAS_LMC LMC_LLC RLC_RCH RMC_RCH LLC_LCH LMC_LCH NAS_SN SN_TUL TUL_POG RCH_SN RCH_TUL RCH_LCH RCH_POG LCH_SN LCH_TUL LCH_POG
86.75 66.80 56.31 67.78 84.80 36.68 21.49 24.22 36.50 70.09 73.69 75.76 74.71 57.45 16.42 46.73 42.52 36.42 61.37 50.80 43.94 37.87 52.57
87.30 65.47 55.24 67.36 86.05 39.42 23.21 25.06 38.73 71.53 73.83 75.60 74.23 57.30 16.77 47.84 42.93 36.35 59.77 49.88 42.95 36.46 51.78
86.57 64.87 54.36 66.77 85.14 38.98 22.75 24.78 38.40 70.78 73.41 75.23 73.45 56.60 16.47 46.93 42.15 35.70 58.99 49.42 42.38 35.95 51.38
88.37 66.53 56.71 68.35 86.82 39.83 23.48 25.40 39.37 72.47 74.27 75.96 74.89 57.74 17.18 48.74 43.44 37.00 60.30 50.66 43.58 36.88 52.36
0.66 0.58 0.84 0.56 0.74 0.35 0.27 0.24 0.33 0.59 0.35 0.26 0.50 0.39 0.24 0.66 0.51 0.47 0.46 0.46 0.40 0.35 0.35
0.75 0.88 1.53 0.83 0.86 0.89 1.16 0.95 0.85 0.83 0.48 0.34 0.68 0.68 1.46 1.39 1.18 1.29 0.77 0.91 0.93 0.95 0.69 0.92 (0.29)
0.55 1.33 1.07 0.42 1.25 2.74 1.72 0.84 2.23 1.44 0.14 0.16 0.48 0.15 0.35 1.11 0.41 0.07 1.60 0.92 0.99 1.41 0.79 0.96 (0.69)
0.63 1.99 1.90 0.62 1.47 7.47 8.00 3.47 6.11 2.05 0.19 0.21 0.64 0.26 2.13 2.38 0.96 0.19 2.61 1.81 2.25 3.72 1.50 2.28 (2.20)
0.09672 0.00248 0.02625 0.12561 0.00902 0.00001 0.00002 0.00036 0.00001 0.00188 0.37218 0.19207 0.06544 0.38939 0.01601 0.00918 0.10603 0.73017 0.00037 0.00448 0.00176 0.00018 0.00265
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
5
Fig. 4. a: Repeatability: coefficients of variation (CV) decreased linearly as distances measured increased, and this correlation was significant (P = 0.0026); b: Repeatability: accuracy improved linearly (P = 0.047) as distances increased; c: Repeatability: histogram of frequencies of differences between Vectra’s measurements and true values for each distance; d: Repeatability: mean differences between Vectra’s measurements and true values obtained during consecutive procedures.
4. Discussion The evaluation and analysis of facial soft tissues is an essential part of orthodontic and maxillofacial diagnosis and treatment planning. Among the various techniques used for 3D surface imaging and patient data acquisition, 3D photogrammetry is attractive as it rapidly provides 3D models from 2D pictures and does not generate any radiation. Three-dimensional (3D) stereophotogrammetry systems based on multiple pod camera setups have attracted increasing interest. Indeed, they are quite convenient for the routine analysis of facial soft tissue outcomes in different fields of medicine and surgery. Tripod camera devices are stationary and allow capture of the model through a single acquisition. However, their drawback is the requirement of a dedicated room. Moreover, pod devices are expensive and require frequent calibration [12]. The handheld Vectra-H1 3D facial imaging system is a specific double-lens reflex camera device that brings a solution to space limitations but requires a three-shot procedure. The procedure may represent a source of errors caused by patients who do not remain strictly motionless or a badly angled
camera during consecutive captures. Hence, our aim was to ensure that this medical device meets the user’s needs and that it consistently provides the intended medical benefit in clinical conditions. This process is usually achieved by tests, inspections, and, in some cases, analyses. The goal of validation is to ensure that the user’s needs are met when using a medical device and that the device consistently provides the intended medical benefit under actual-use conditions. Camison et al. [13] have proposed a validation for the Vectra-H1 but compared the digital measurements obtained by Vectra H1 to those obtained using another numerical system, the 3dMDface1. The authors found that these devices were highly comparable. Repeatability was assessed on living subjects, while reproducibility was tested on a mannequin. The authors chose the 3dMDface numerical system as a reference because of its accuracy and precision has been validated by a number of investigators, the studies uniformly showing sub-millimeter global error margins. Gibelli et al. [14] and Kim et al. [15] validated the portable Vectra H1 device in a comparison with a static Vectra M3 device in which they analysed linear, angular, surface area and volume measure-
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
6
Table 4 Reproducibility (True value: measurements with the slide caliper, mean of distances measured by six operators; Min: minimum value measured; Max: maximum value measured; SD: standard deviation; CV: coefficient of variation; SYST_E: systematic error or absolute error in millimetres; JUSTE: accuracy measurement in percent; t-test (P)). N
Measure
True_Value
Mean
Min
Max
SD
CV
SYST_E
JUSTE
p (t_test)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Mean (SD)
TRI_RLC TRI_RMC TRI_NAS TRI_LMC TRI_LLC RLC_RMC RMC_NAS NAS_LMC LMC_LLC RLC_RCH RMC_RCH LLC_LCH LMC_LCH NAS_SN SN_TUL TUL_POG RCH_SN RCH_TUL RCH_LCH RCH_POG LCH_SN LCH_TUL LCH_POG
81.00 61.00 47.80 62.60 82.25 39.67 20.59 21.98 39.41 68.74 61.54 68.00 61.87 51.67 11.66 33.03 35.61 32.06 53.04 36.90 35.85 32.67 38.07
80.64 61.24 47.75 62.77 82.36 39.92 21.81 22.23 39.81 69.15 62.34 68.11 62.52 53.09 12.29 34.63 35.24 31.54 53.69 37.88 35.90 32.47 38.81
80.29 60.89 47.42 62.50 81.40 39.73 21.46 21.79 39.59 68.16 61.30 67.29 62.05 52.39 11.78 34.09 34.69 30.99 52.85 37.25 35.42 31.85 38.22
80.81 61.89 48.21 63.09 83.03 40.10 22.47 22.55 39.95 69.82 63.29 69.04 62.80 53.65 12.74 35.56 35.59 31.95 54.47 38.44 36.44 32.80 39.27
0.19 0.37 0.30 0.28 0.61 0.16 0.35 0.26 0.13 0.56 0.70 0.57 0.29 0.49 0.36 0.55 0.34 0.40 0.53 0.41 0.47 0.37 0.35
0.23 0.60 0.63 0.44 0.74 0.39 1.60 1.16 0.32 0.81 1.12 0.83 0.46 0.93 2.90 1.60 0.96 1.25 0.99 1.08 1.30 1.14 0.91 0.97 (0.56)
-0.36 0.24 -0.05 0.17 0.11 0.25 1.22 0.25 0.40 0.41 0.80 0.11 0.65 1.42 0.63 1.60 -0.37 -0.52 0.65 0.98 0.05 -0.20 0.74 0.53 (0.43)
-0.44 0.39 -0.10 0.27 0.13 0.63 5.93 1.14 1.01 0.60 1.30 0.16 1.05 2.75 5.40 4.84 -1.04 -1.62 1.23 2.66 0.14 -0.61 1.94 1.54 (1.7)
0.00563 0.17296 0.69999 0.19712 0.67715 0.01228 0.00036 0.06513 0.00065 0.13289 0.03802 0.65634 0.00274 0.00086 0.00781 0.00084 0.04458 0.02442 0.02996 0.00206 0.80480 0.24278 0.00353
ments. That study assessed only the repeatability of the Vectra H1 device and not its reproducibility. Although both previous studies provided important contributions regarding the reliability of the Vectra H1, some aspects related to its repeatability and reproducibility under clinical-use conditions were not fully addressed. Our methodology, in which we compared numerical measurements obtained by a Vectra H1 and direct values (distances measured with a digital slide calliper) only on living subjects only, appear more suitable for the refine an assessment of this specific device. Previous studies [16–21] were performed to investigate the accuracy of other stereophotogrammetry commercial systems such as the 3dMDface System [3dMD LLC, Atlanta, USA], 3DSensoren FaceSCAN [3D Shape GmbH, Erlangen, Germany], and Di3D [Dimensional Imaging, Glasgow, UK]). These sutdies focused on the reliability of these tools for distance measurements. To the best of our knowledge, no study has investigated the repeatability, reproducibility and accuracy of the Vectra Analysis Module1 (VAM) software by comparing data collected by traditional methods (e.g., direct anthropometry) to data obtained by the Vectra H1. Our study showed that the Vectra H1 was highly accurate as it showed negligible systematic error. Regarding repeatability and reproducibility, the systematic error (respectively standard deviation) was 0.96 mm (0.89) and 0.53 mm (0.43), respectively, which are fully acceptable for clinical practice. This study describes the results of a previous study of a tripod Vectra device [22]. In clinical use, it is very important that the registration of the 3D facial photographs provides a precise accuracy within 1 mm. Our study found the same accuracy, while repeatability and reproducibility drastically improved as distances increased. The mean error can be affected by several factors that can influence the registration accuracy obtained by the acquisition system. One of these factors was the ability to capture the same facial expression of the subject throughout the whole assessment procedure. Interestingly, our study found no significant difference in the mean differences between Vectra measurements and the true values during consecutive procedures (Fig. 5d). The hypothesis was that the
dermographic points were imprecise and have generated errors when measuring short distances with a caliper. Indeed, the caliper was not precise enough to measure the distances considering the center of every point, whereas the magnification tool included in the measurement software ensured that the middle of the mark was easily identified with the Vector H1 (Fig. 6). Moreover, in this study, we compared straight distances between two points regardless of the surface area. A study based on surface-weighted distances [11] should support this validation by comparing tape measurements to numerical surface measurements obtained with a Vectra H1. Fig. 4d shows that over time and after consecutive repeated procedures, there was no improvement in accuracy. This indicates that there was no learning curve for using the Vectra H1, suggesting that it could be made available to non-initiated practitioners without the requirement for any complex training. Although our study found that there was no learning curve and that the three consecutive captures did not affect the accuracy, this device might be of uneasy use in toddlers and infants. Indeed, this system requires a motionless subject who can maintain the face in a neutral position with lips closed (without tightening) to avoid artefacts during the image stitching procedure. Hence, it cannot be used for surgical planning or syndrome assessment in young patients who cannot be fully compliant. Otherwise, the use of 3D digital photogrammetry may substantially reduce the clinical chair time and provide flexibility to the user for post-chair analyses using the software. Indeed, the user would be able to digitally recreate, store, manipulate and reanalyse the 3D facial profile of the patient. Kim et al. [15] showed that 8 minutes were needed per subject to perform a direct anthropometry. The time to perform the measurements using the handheld camera (including image capture, render, landmark placement and distance calculation) was 3 minutes per subject and the time to perform the conventional camera measurement (including image capture, render, landmark placement and distance calculation) was also approximately 4 minutes per subject.
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
7
Fig. 5. a: Reproducibility: coefficients of variation (CV) decreased linearly as distances measured increased, and this correlation was significant (P = 0.00045); b: Reproducibility: accuracy improved linearly (P = 0.0025) as distances increased; c: Reproducibility: histogram of frequencies of differences between Vectra’s measurements and true values for each distance; d: Reproducibility: there was no significant difference (P = 0.30) in the mean differences between Vectra’s measurements and true values obtained during consecutive procedures.
In a future perspective with technological evolution, Vectra H1 should be replaced by smartphones but actual applications are still lower than that of dedicated medical devices. Within a short period of time, industries and manufacturers are likely to develop mobile [23] technologies that can achieve high-definition 3D photography.
5. Conclusions Virtual 3D models generated through the Vectra H1 provide a high level of technical precision regarding landmark identification. Test and re-test reliability data demonstrated that the findings obtained using Vectra H1 did not change over time. The results of this study show that Vectra H1 provides an accurate linear assessment of clinical parameters by analysing facial morphology with good accuracy, at a lower cost and while requiring less space than multi-pod devices. Disclosure of interest The authors declare that they have no competing interest.
Acknowledgment
Fig. 6. Non-strict superimposition of the centres of dermographic (circled in red) and numerical facial landmarks (Nasion).
This study was declared to the Commission Nationale Informatique et Liberte´ (CNIL No. 2209316) and both volunteers agreed that their data could be used for biomedical study.
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012
G Model
JORMAS-679; No. of Pages 8 C. Savoldelli et al. / J Stomatol Oral Maxillofac Surg xxx (2018) xxx–xxx
8
Appendix A. Supplementary data Supplementary material related to this article can be found, in the online version, at https://doi.org/10.1016/j.jormas.2019.03. 012. References [1] Masoud MI, Bansal N, Castillo J , Manosudprasit A, Allareddy V, Haghi A, et al. 3D dentofacial photogrammetry reference values: a novel approach to orthodontic diagnosis. Eur J Orthod 2017. http://dx.doi.org/10.1093/ejo/cjw055. [2] Manosudprasit A, Haghi A, Allareddy V, Masoud MI. Diagnosis and treatment planning of orthodontic patients with 3-dimensional dentofacial records. Am J Orthod Dentofac Orthop 2017. http://dx.doi.org/10.1016/j.ajodo.2016.10.037. [3] Hanawa S, Kitaoka A, Koyama S, Sasaki K. Influence of maxillary obturator prostheses on facial morphology in patients with unilateral maxillary defects. J Prosthet Dent 2015. http://dx.doi.org/10.1016/j.prosdent.2014.06.016. [4] Mertens C, Wessel E, Berger M, Ristow O, Hoffmann J, Kansy K, et al. The value of three-dimensional photogrammetry in isolated sagittal synostosis: impact of age and surgical technique on intracranial volume and cephalic index – a retrospective cohort study. J Cranio-Maxillofacial Surg 2017. http://dx.doi.org/ 10.1016/j.jcms.2017.09.019. [5] Choi JW, Lee JY, Oh TS, Kwon SM, Yang SJ, Koh KS. Frontal soft tissue analysis using a 3 dimensional camera following two-jaw rotational orthognathic surgery in skeletal class III patients. J Cranio-Maxillofacial Surg 2014. http://dx.doi.org/10.1016/j.jcms.2013.05.004. [6] Jung J, Lee C-H, Lee J-W, Choi B-J. Three dimensional evaluation of soft tissue after orthognathic surgery. Head Face Med 2018;14:21. [7] Terzic A, Schouman T, Scolozzi P. Accuracy of morphological simulation for orthognatic surgery. Assessment of a 3D image fusion software. Rev Stomatol Chir Maxillofac Chir Orale 2013. [8] Verhulst A, Hol M, Vreeken R, Becking A, Ulrich D, Maal T. Three-dimensional imaging of the face: a comparison between three different imaging modalities. Aesthetic Surg J 2018. http://dx.doi.org/10.1093/asj/sjx227. [9] Hermans DJJ, Maal TJJ, Berge´ SJ, Van Der Vleuten CJM. Three-dimensional stereophotogrammetry: A novel method in volumetric measurement of infantile hemangioma. Pediatr Dermatol 2014. http://dx.doi.org/10.1111/ pde.12224. [10] Heike CL, Upson K, Stuhaug E, Weinberg SM. 3D digital stereophotogrammetry: a practical guide to facial image acquisition. Head Face Med 2010. http:// dx.doi.org/10.1186/1746-160X-6-18. [11] Lutz JC, Hostettler A, Agnus V, Nicolau S, George D, Soler LRY. A new software suite in orthognathic surgery: patient specific modeling, simulation and navigation. Surg Innov 2018;1:1553.
[12] Tzou CH, Frey M. Evolution of 3D surface imaging systems in facial plastic surgery. Facial Plast Surg Clin North Am 2011. http://dx.doi.org/10.1016/ j.fsc.2011.07.003. [13] Camison L, Bykowski M, Lee WW, Carlson JC, Roosenboom J, Goldstein JA, et al. Validation of the Vectra H1 portable three-dimensional photogrammetry system for facial imaging. Int J Oral Maxillofac Surg 2017. http://dx.doi.org/ 10.1016/j.ijom.2017.08.008. [14] Gibelli D, Pucciarelli V, Cappella A, Dolci C, Sforza C. Are portable stereophotogrammetric devices reliable in facial imaging? a validation study of Vectra H1 Device. J Oral Maxillofac Surg 2018. http://dx.doi.org/10.1016/ j.joms.2018.01.021. [15] Kim AJ, Gu D, Chandiramani R, Linjawi I, Deutsch ICK, Allareddy V, et al. Accuracy and reliability of digital craniofacial measurements using a smallformat, handheld 3D camera. Orthod Craniofacial Res 2018. http://dx.doi.org/ 10.1111/ocr.12228. [16] Naini FB, Akram S, Kepinska J, Garagiola U, McDonald F, Wertheim D. Validation of a new three-dimensional imaging system using comparative craniofacial anthropometry. Maxillofac Plast Reconstr Surg 2017. http://dx.doi.org/ 10.1186/s40902-017-0123-3. [17] Knoops PGM, Beaumont CAA, Borghi A, Rodriguez-Florez N, Breakey RWF, Rodgers W, et al. Comparison of three-dimensional scanner systems for craniomaxillofacial imaging. J Plast Reconstr Aesthetic Surg 2017. http:// dx.doi.org/10.1016/j.bjps.2016.12.015. [18] Nord F, Ferjencik R, Seifert B, Lanzer M, Gander T, Matthews F, et al. The 3dMD photogrammetric photo system in cranio-maxillofacial surgery: Validation of interexaminer variations and perceptions. J Cranio-Maxillofacial Surg 2015. http://dx.doi.org/10.1016/j.jcms.2015.08.017. [19] Ort R, Metzler P, Kruse AL, Matthews F, Zemann W, Gra¨tz KW, et al. The Reliability of a three-dimensional photo system- (3dMDface-) based evaluation of the face in cleft lip infants. Plast Surg Int 2012. http://dx.doi.org/ 10.1155/2012/138090. [20] Secher JJ, Darvann TA, Pinholt EM. Accuracy and reproducibility of the DAVID SLS-2 scanner in three-dimensional facial imaging. J Cranio-Maxillofacial Surg 2017. http://dx.doi.org/10.1016/j.jcms.2017.07.006. [21] Khambay B, Nairn N, Bell A, Miller J, Bowman A, Ayoub AF. Validation and reproducibility of a high-resolution three-dimensional facial imaging system. Br J Oral Maxillofac Surg 2008;46:27–32. http://dx.doi.org/10.1016/ j.bjoms.2007.04.017. [22] Metzler P, Sun Y, Zemann W, Bartella A, Lehner M, Obwegeser JA, et al. Validity of the 3D VECTRA photogrammetric surface imaging system for cranio-maxillofacial anthropometric measurements. Oral Maxillofac Surg 2014;18:297– 304. http://dx.doi.org/10.1007/s10006-013-0404-7 [nto.]. [23] Salazar-Gamarra R, Seelaus R, Da Silva JVL, Da Silva AM, Dib LL. Monoscopic photogrammetry to obtain 3D models by a mobile device: A method for making facial prostheses. J Otolaryngol - Head Neck Surg 2016. http:// dx.doi.org/10.1186/s40463-016-0145-3.
Please cite this article in press as: Savoldelli C, et al. Accuracy, repeatability and reproducibility of a handheld three-dimensional facial imaging device: The Vectra H1. J Stomatol Oral Maxillofac Surg (2019), https://doi.org/10.1016/j.jormas.2019.03.012