- Email: [email protected]
Landmark errors on posteroanterior cephalograms
ORIGINAL ARTICLE
Landmark errors on posteroanterior cephalograms Feyza Ulkur,a Fulya Ozdemir,b Derya Germec-Cakan,b and E. Cigdem Kasparc Istanbul, Turkey
Introduction: It is important to reduce the method errors when evaluating posteroanterior cephalograms to see either small deviations from normal or transverse changes caused by orthodontic treatment. The aim of this study was to determine horizontal and vertical intraexaminer and interexaminer agreement in localization of landmarks in posteroanterior cephalograms of adult patients. Methods: The sample was gathered retrospectively from the archives of the Department of Orthodontics of Yeditepe University in Istanbul, Turkey. Radiographs of 39 patients diagnosed with skeletal asymmetries (20 women, 19 men) were drawn manually, and a coordinate system was established with software. The tracings were made by 2 operators, after a calibration session on 29 landmarks (22 bilateral, 7 midline). Intraclass correlation coefficients and the Bland-Altman test were used for detecting interexaminer and intraexaminer agreement for each cephalometric variable. Results: The interexaminer agreement test showed that the most problematic landmark was crista galli, which showed moderate consistency between 2 examiners in the y coordinates at 2 time points. Condylar and zygomatic landmarks showed good agreement. The greater wing inferior and superior orbit, maxillary point, menton, anterior nasal spine, antegonial notch, mandibular and maxillary molar point, maxillary and mandibular incisor point, and maxillary and mandibular incisor edge landmarks had excellent agreement between the 2 examiners at the 2 time points in both the x and y coordinates. Conclusions: There are fewer errors in intraexaminer than in interexaminer correlations in landmark identifications on posteroanterior radiographs. All landmarks investigated except crista galli showed good agreement between measurements. (Am J Orthod Dentofacial Orthop 2016;150:324-31)
P
osteroanterior (PA) cephalometric radiographs help to evaluate maxillofacial structures in the transverse plane and are requested by the clinician if there is doubt about any problems in the transverse plane. Even though 3-dimensional (3D) cone-beam computed tomography technology is now available at most clinics, and it seems to give more reliable and extensive information on the asymmetry of the face,1 where this technology is not yet available, or when most of the available archive for comparison and longitudinal research is still only in 2-dimensional (2D) records, PA radiographs are still used.2 For the imaging to be useful, there should be reliably identified landmarks From Yeditepe University, Istanbul, Turkey. a Lecturer, Department of Orthodontics, Faculty of Dentistry. b Associate professor, Department of Orthodontics, Faculty of Dentistry. c Assistant professor, Department of Biostatistics and Medical Informatics, Department of Medical Education. All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest, and none were reported. Address correspondence to: Feyza Ulkur, Department of Orthodontics, Faculty of Dentistry, Yeditepe University, Bagdat Cad. No: 238, Goztepe, Istanbul 34728, Turkey; e-mail, [email protected]. Submitted, February 2015; revised and accepted, January 2016. 0889-5406/$36.00 Ó 2016 by the American Association of Orthodontists. All rights reserved. http://dx.doi.org/10.1016/j.ajodo.2016.01.016
324
for accurate measurements.3 A recently published systematic review4 concluded that there is only 1 article on landmark errors that fulfilled the selection criteria, and that study was performed on dry skulls.5 This review article cautioned clinicians to take care in interpreting studies that used dry skulls because another article by H€agg et al5 stated that skeletal and dental angular errors are greater, up to 4 times greater for some measurements, in the presence of soft tissues. There are limitations in using these radiographs: eg, difficulty in positioning the head and reproducing head posture, radiographic errors related to projection, errors in identifying landmarks, and finding a straight reference line. Regarding errors in identifying landmarks, the most problematic points lie on the curved lines, in areas of low contrast, and with other structures superimposed.6 Errors of interpretation are reduced by clear definitions and training.7-9 Therefore, the aim of this study was to determine the horizontal and vertical intraexaminer and interexaminer identification errors in the localization of landmarks in the PA radiographs of adult patients. The null hypothesis was that there are no differences in the accuracy and reproducibility of landmark identifications between 2 orthodontists at 2 time points, 1 month apart.
Ulkur et al
325
Table I. Definitions for the cephalometric landmarks used in the study
Fig 1. PA cephalometric landmarks and coordinate system used in the study. MATERIAL AND METHODS
The sample PA radiographs of this study, which had the same resolution and quality, were gathered randomly from the 10-year archive at the Department of Orthodontics, Faculty of Dentistry, of Yeditepe University, Istanbul, Turkey. The radiographs belonged to 39 asymmetric patients (20 women, 19 men) ranging in age from 18 to 45 years (mean 6 SD, 31.3 6 12.2 years). The following were the inclusion criteria: no cleft and lip, and no diagnosed systemic diseases or craniofacial syndromes. Patient data were handled according to the requirements and recommendations of the Declaration of Helsinki. Ethical approval was obtained from the institutional review board of Yeditepe University. The cephalometric radiographs used in this study were created using a Promax x-ray device (Planmeca, Helsinki, Finland) with the patients were in natural head position. Source to ear rod distance was 160 cm, and ear rod to film distance was 17.5 mm. All radiographs were covered with tracing paper. For each radiograph, an individual coordinate system was formed. On each corner of the paper, a large plus sign was drawn to trace the papers identically, and the sign on the upper right corner was used used as the x and y coordinates (Fig 1). The tracings were made by 2 operators (F.U. and D.G.C.), who are experienced orthodontists, after a calibration session with the written definitions of the points and practical trials were carried out together. A
Landmark Definition Bilateral skeletal landmarks (18 landmarks) Greater wing superior Intersection of the superior border of orbit (GWSO) the greater wing of the sphenoid bone and the lateral orbital margin Greater wing inferior Intersection of the inferior border of orbit (GWIO) the greater wing of the sphenoid bone and the lateral orbital margin Zygomatic (Z) Most lateral aspect of the zygomatic arch Condyle superior (CS) Most superior aspect of the condyle Mastoid process (MP) Most inferior point on the mastoid process Nasal cavity (NC) Most lateral point on the nasal cavity Maxillary point (MP) Center of the concavity of the zygomatic process of the maxilla Gonion (G) Midpoint on the curvature at the angle of the mandible (gonion) Antegonial (AG) Deepest point on the curvature of the antegonial notch Midline skeletal landmarks (3 landmarks) Crista galli (CG) Geometric center of crista galli Anterior nasal spine Center of the intersection of the nasal (ANS) septum and the palate Menton (ME) Midpoint on the inferior border of the mental protuberance Bilateral dental landmarks (4 landmarks) Maxillary molar (MX6) Midpoint on the buccal surface of the maxillary first molar Mandibular molar Midpoint on the buccal surface of the (ML6) mandibular first molar Midline dentoalveolar landmarks (4 landmarks) Upper incisor point Crest of the alveolus between the (U1P) maxillary central incisors Upper incisor edge Midpoint on the incisal edges of the (U1E) maxillary incisors Lower incısor edge Midpoint on the incisal edges of the (L1E) mandibular incisors Lower incisor point Crest of the alveolus between the (L1E) mandibular central incisors
total of 29 landmarks were marked on tracing paper manually using a 0.1-mm pen (Ecco pigment; FaberCastell, Stein, Germany): 22 landmarks were bilateral, and 7 were on the midline. The list and the definitions of landmarks marked on each PA cephalometric radiograph are given in Table I. The operators repeated the tracings 1 month later. The tracing papers were scanned into digital format at 300 dpi using a scanner (1680 Pro; Epson, Nagano, Japan) with 1600 dpi imaging, 40 3 800 pixels per line, and 48bit color depth for both film and reflective scanning, and displayed on a 15-in, 1024 3 768 high-pixel resolution monitor (model FP581, Benq, Taipei, Taiwan) with a pixel pitch of 0.297 mm, a contrast ratio of 450:1, and brightness of 250 candela per square meter. Each scanned paper
American Journal of Orthodontics and Dentofacial Orthopedics
August 2016 Vol 150 Issue 2
Ulkur et al
326
Table II. Intraclass correlation coefficients and 95% confidence interval for intra- and interexaminer agreement Intraexaminer (F.U.) Landmark Greater wing superior orbit, right Greater wing superior orbit, left Greater wing inferior orbit, right Greater wing inferior orbit, left Zygomatic, right Zygomatic, left Condyle superior, right Condyle superior, left Mastoid process, right Mastoid process, left Nasal cavity, right Nasal cavity, left Maxillary point, right Maxillary point, left Gonion, right Gonion, left Antegonial, right Antegonial, left Crista galli Anterior nasal spine Menton Maxillary first molar, right Maxillary first molar, left Mandibular first molar, right Mandibular first molar, left Upper incisor point Upper incisor edge Lower incısor edge Lower incisor point
Intraexaminer (D.G.C.)
Interexaminer marking 1
Interexaminer marking 2
x-axis 0.9965
y-axis 0.9873
x-axis 0.9924
y-axis 0.9846
x-axis 0.9867
y-axis 0.9715
x-axis 0.9905
y-axis 0.9683
0.9959
0.9643
0.9958
0.9557
0.9944
0.9815
0.9915
0.9681
0.9953
0.9912
0.9919
0.9844
0.9927
0.9632
0.9948
0.9690
0.9971
0.9666
0.9932
0.9558
0.9972
0.9811
0.9949
0.9785
0.9809 0.9960 0.9892 0.9959 0.9892 0.9931 0.9938 0.9964 0.9911 0.9929 0.9861 0.8770 0.9854 0.9919 0.9966 0.9946 0.9901 0.9931
0.9878 0.9600 0.9612 0.9424 0.9984 0.9788 0.9569 0.9708 0.9821 0.9729 0.9972 0.9918 0.9972 0.9911 0.9740 0.9782 0.9977 0.9936
0.9915 0.9962 0.9129 0.9935 0.8834 0.9917 0.9935 0.9954 0.9913 0.9928 0.9854 0.9325 0.9880 0.9931 0.9938 0.9690 0.9874 0.9910
0.9873 0.9525 0.9743 0.9722 0.9863 0.9829 0.9752 0.9654 0.9587 0.9585 0.9410 0.9876 0.9985 0.9878 0.9158 0.9711 0.9975 0.9950
0.8603 0.9988 0.8968 0.9799 0.9847 0.9954 0.9970 0.9972 0.9912 0.9982 0.9882 0.8757 0.9937 0.9979 0.9914 0.9974 0.9774 0.9908
0.9671 0.9777 0.8556 0.8878 0.9978 0.9987 0.9064 0.8975 0.9275 0.9745 0.9855 0.9934 0.9968 0.9967 0.7961 0.9790 0.9994 0.9661
0.8704 0.9960 0.8691 0.9801 0.9178 0.9937 0.9929 0.9954 0.9933 0.9951 0.9900 0.9369 0.9903 0.9935 0.9939 0.9632 0.9850 0.9890
0.9766 0.9666 0.8815 0.9149 0.9899 0.9839 0.9665 0.9493 0.9688 0.9739 0.9338 0.9940 0.9972 0.9941 0.7878 0.9818 0.9986 0.9693
0.9933
0.9856
0.9934
0.9774
0.9977
0.9552
0.9960
0.9551
0.9937
0.9950
0.9887
0.9897
0.9957
0.9784
0.9938
0.9759
0.9929
0.9859
0.9934
0.9825
0.9976
0.9845
0.9961
0.9812
0.9937 0.9876 0.9851 0.9926
0.9929 0.9941 0.9889 0.9950
0.9911 0.9880 0.9866 0.9906
0.9839 0.9897 0.9897 0.9912
0.9992 0.9973 0.9977 0.9993
0.9696 0.9937 0.9857 0.9814
0.9965 0.9971 0.9960 0.9964
0.9797 0.9929 0.9854 0.9793
was saved as a portable document file (PDF). The coordinate system was established with software (AutoCAD, 2004 version; Autodesk, San Rafael, Calif). Statistical calculations were carried out with software (NCSS, Kaysville, Utah) for Windows. In addition to descriptive statistics (means, standard deviations), intraclass correlation coefficients (ICCs) were used for determining interexaminer and intraexaminer agreement for each cephalometric variable. ICC, derived from analysis of variance, assesses rating reliability by comparing the variability of the different ratings of the same subject to the total variation across all ratings and all subjects. It is a measure of the homogeneity of elements within clusters and has a maximum value of 1 when there is complete homogeneity within clusters.10 For the detection of interexaminer error, the mean of the 2
August 2016 Vol 150 Issue 2
measurements done by each examiner was used. To better understand the disagreement seen for controversial points, Bland-Altman plots were used to explore the results in a graphic approach.11 RESULTS
The results showed that each operator was consistent in the repeated measurements (intraexaminer agreement) in most landmarks marked (Table II). ICC values of 0.75 or above are considered good, and those above 0.90 are excellent.12 In this section, landmarks with an ICC less than 0.90 are reported along with the millimeter difference between measurements for ease of comparison with previous studies. One examiner (F.U.) had ICC values of 0.88 and 0.99 for the x and y coordinates of the left gonion. The mean
American Journal of Orthodontics and Dentofacial Orthopedics
Ulkur et al
0.52 6 0.60 0.17 6 0.86 0.34 6 0.73 0.31 6 1.23 1.04 6 0.68
0.76 6 0.85
0.24 6 1.26
0.60 6 0.54
0.72 6 0.32 0.40 6 1.04 0.34 6 0.92 0.55 6 0.97 0.41 6 0.93 0.75 6 0.97 0.20 6 0.58 0.29 6 1.21 0.20 6 0.79 0.03 6 0.60
0.27 6 0.54 0.16 6 1.35
0.70 6 0.61 0.02 6 1.08
0.50 6 0.58 0.71 6 1.04
y-axis, mean difference 6 SD 0.33 6 1.02 0.90 6 0.16 x-axis, mean difference 6 SD 0.84 6 0.40 0.74 6 036 y-axis, mean difference 6 SD 0.84 6 0.31 0.93 6 0.12 x-axis, mean difference 6 SD 0.73 6 1.94 0.90 6 0.12 y-axis, mean difference 6 SD 0.40 6 1.11 0.35 6 1.03 x-axis, mean difference 6 SD 0.29 6 0.52 0.47 6 0.35 y-axis, mean difference 6 SD 0.84 6 0.09 0.65 6 0.36 x-axis, mean difference 6 SD 0.37 6 1.28 0.00 6 0.69
Landmark Gonion, left Mastoid process, right Zygomatic, right Condyle superior, right Crista galli
Interexaminer marking 2 Interexaminer marking 1 Intraexaminer (D.G.C.) Intraexaminer (F.U.)
and 90%
Table III. Mean difference and standard deviations of the landmarks, which have a confidence interval for intra- and interexaminer agreement between 75%
327
Fig 2. Bland-Altman plot of the landmark crista galli. The interexaminer comparison of markings: A, on the x coordinate; B, on the y coordinate.
difference of the 2 markings for this landmark was 0.37 6 1.28 mm for the x coordinate (Table III). The x coordinate of the right mastoid process showed an ICC of 0.88 by 1 examiner (F.U.). The mean difference of the 2 markings for this landmark was 0.47 6 0.35 mm for the x coordinate. The rest of the intraexaminer ICC values were mostly higher than 0.90 or 0.95, showing excellent agreement (Table II). In the interexaminer evaluation, the right zygomatic landmark had ICC values of 0.86 and 0.87 for the x and y coordinates, respectively; these were greater than 0.75 (Table II). The mean difference of the zygomatic landmark for the x coordinate was 0.37 6 0.93 mm (Table III). In our study, the landmarks with low ICC values also exceeded the confidence interval (Figs 2-4) for the Bland-Altman test.13 Bland-Altman plots show scattergrams of the relationship between the means and the differences
American Journal of Orthodontics and Dentofacial Orthopedics
August 2016 Vol 150 Issue 2
Ulkur et al
328
Fig 3. Bland-Altman plot of the landmark gonion. The interexaminer comparison of markings: A, on the x coordinate; B, on the y coordinate.
Fig 4. Bland-Altman plot of the landmark zygomatic. The interexaminer comparison of markings: A, on the x coordinate; B, on the y coordinate.
of the 2 measurements. If the differences have a normal distribution, the scattering of the differences should be marked around zero, and 95% of the marks should be between 1.96 and 11.96. Then we concluded that there is no relationship between the means and the differences; 61.96 is considered the confidence interval, and points marked between these lines are considered within the limits of agreement. Each Bland-Altman diagram describes either the horizontal (Figs 2, A; 3, A; and 4, A) or vertical (Figs 2, B; 3, B; and 4, B) limits of agreement of the points marked. Bland-Altman plots can be described as follows: x-axis defines the average values of measurements obtained from the first and second examiners, (examiner 1 1 examiner 2)/2, across a range of values. The y-axis defines the difference between the values of the 2 measurements of the first and second examiners, (examiner 2 examiner 1)/2. Each dot represents an individual difference. The horizontal line between the dotted lines represents the
bias: ie, the average of the difference values. The dotted lines represent precision: ie, the limits of agreement. The dots outside the limits of agreement (dotted lines) are called outliers. When all differences are not positive, it means that there is no systematic bias. The dotted horizontal lines represent the 95% confidence limits (limits of agreement). For the horizontal change in the position of the zygomatic point, the agreement level is shown in Figure 4 as a Bland-Altman plot. The range of values at the x-axis is between 20 and 55. The differences vary from 0.4 to 3.6; not all differences are positive, meaning that there is no systematic bias. We noted that 2 difference scores exceeded the limits of agreement; one exceeds the lower limit, and the other exceeds the upper limit. When the Bland-Altman plot was examined to determine whether the differences in values depended on the measurement value (the average of the 2 measurements on the x-axis), we found that the differences were scattered around the
August 2016 Vol 150 Issue 2
American Journal of Orthodontics and Dentofacial Orthopedics
Ulkur et al
bias, with no pattern. For the vertical change in the position of the zygomatic point, the x-axis lies across the range of values between 6 and 22. The y-axis represents the difference in values obtained from the 2 examiners. In this case, the differences varied from 1.62 to 1.31; not all differences are positive, which means that there is no systematic bias. We noted that 3 difference scores exceeded the limits of agreement; 1 exceeded the lower limit, and 2 exceeded the upper limit. When the BlandAltman plot was examined to determine whether the differences in values depended on the measurement value (the average of the 2 measurements on the x-axis), we saw that the differences were scattered around the bias, with no pattern. The right condyle superior landmark showed an interexaminer ICC between 0.85 and 0.90 on both the x and y coordinates. The mean difference for interexaminer tests of the left condyle superior landmark on the x coordinate was 0.65 6 0.97 mm, and 0.65 6 1.0 mm was the mean differences on the y coordinate for both time points (Table III). The left gonion landmark had an interexaminer ICC of 0.87 on the x coordinate, and the mean difference of the 2 markings was 0.73 6 1.94 mm. The limits of agreement for gonion are shown in Figure 3 in the Bland-Altman plot. In Figure 3, A, the horizontal deviation in the position of gonion is shown, where the x-axis represents the average values between 20 and 55. The yaxis represents the difference in values obtained from the 2 examiners. The differences vary from 1.61 to 1.10, and not all differences are positive; this means that there was no systematic bias. We noted that 3 of the difference scores exceeded the limits of agreement; 2 exceeded the lower limit, and 1 the upper limit. When the Bland-Altman plot was examined to determine whether the differences in values depended on the measurement value (the average of the 2 measurements on the x-axis), it appeared to have no pattern, with the differences scattered around the bias. In Figure 3, B, when we examined the vertical deviation in the position of gonion, the x-axis represented the average values between 15 and 40. The y-axis represented the differences obtained from the 2 examiners. The differences varied from 4.3 to 2.8; not all differences were positive; this means that there was no systematic bias. We noted that 2 difference scores exceeded the limits of agreement; one exceeded the lower limit, and the other exceeded the upper limit. When the Bland-Altman plot was examined to determine whether the differences in values depended on the measurement value (the average of the 2 measurements on the x-axis), it appeared to have no pattern, with the differences scattered around the bias.
329
The crista galli landmark had an interexaminer ICC of 0.80 on the y coordinate (Table II). The mean difference was 0.25 6 0.84 mm. The limits of agreement for crista galli are shown in the Bland-Altman plot (Fig 2). In Figure 2, A, the horizontal deviation in its position is shown; the standard deviation of the different scores for the x-axis data was approximately 0.77. We noted that 3 difference scores exceeded the lower limit of agreement. When the Bland-Altman plot was examined to determine whether the differences depended on the measurement value (the average of the 2 measurements on the x-axis), we saw that the differences were scattered around the bias, with no pattern (Fig 2, A). In the examination of the vertical deviation in the position of crista galli, the x-axis represented the average values obtained from the 2 examiners across the range of values between 60 and 40. The y-axis represented the difference in values obtained from the 2 examiners. The differences varied from 1.6 to 2.0; not all differences were positive, meaning that there was no systematic bias. We noted that 3 difference scores exceeded the limits of agreement; 1 exceeded the lower limit, and 2 exceeded the upper limit. When the Bland-Altman plot was examined to determine whether the differences depended on the measurement value (the average of the 2 measurements on the x-axis), it appeared that the differences were scattered around the bias, with no pattern. Other than these, the investigated landmarks showed ICC values greater than 0.90, and a few were greater than or equal to 0.95. The greater wing inferior and superior orbit, maxillary point, menton, anterior nasal spine, antegonial notch, mandibular and maxillary molar point, upper and lower incisor point, and upper and lower incisor edge landmarks showed excellent agreement between the 2 examiners in both the x and y coordinates (Table II). DISCUSSION
Cephalometric evaluations of patients with orthodontic malocclusions have been traditionally carried out on lateral and frontal PA cephalographs since the early 20th century when the orthodontist needed to diagnose facial skeletal imbalances to assess growth or response to treatment. Recently, with the popularity of cone-beam computed tomography, orthodontists gather 3D data mostly to evaluate the airway, impacted teeth, and the temporomandibular joint.14,15 When the patient's dose of radiation is reduced substantively, cone-beam computed tomography can be used instead of panoramic, PA, or lateral cephalometric radiographs.16 However, even if there are 3D data, orthodontists usually generate and measure 2D cephalometric radiographs, with which they are more familiar. Until the
American Journal of Orthodontics and Dentofacial Orthopedics
August 2016 Vol 150 Issue 2
Ulkur et al
330
problems with 3D evaluations are solved and the technology is widely used by orthodontists, the problem of landmark identification still exists for 2D evaluations. Precision and reproducibility are important when evaluating a 2D cephalometric radiograph because quantitative, systematic, and objective measurements are needed for diagnosis of a patient's malocclusion so that a satisfactory treatment plan can be formed. The reason for using ICC calculations in this study was to evaluate whether the results for 2 intraexaminer and interexaminer measurements showed agreement. The ICC assesses rating reliability by comparing the variability of the different ratings of the same subject to the total variations across all ratings and all subjects for continuous data. A graphic method is often used as an alternative to the ICC for analyzing interrater reliability data, as proposed by Bland and Altman.13 It combines a graphic approach and a quantitative analysis of the magnitude of the rating differences. Part of the popularity of the Bland-Altman method stems from its graphic nature; t tests that would compare the means of 2 groups were not used because deviations in a few values could affect group means and might cause mathematical errors regarding the results of this study. Correlation was also not used because it mainly shows a relationship, not agreement, between data sets. Therefore, ICC and agreement were used to assess the data. It is recommended to use Bland-Altman plots mainly as an exploratory technique. They allow researchers to have a first glimpse into the interrater reliability results. Ultimately, an ICC based on the appropriate statistical model should be calculated. The mean differences and deviations of the landmarks with ICC values less than 0.90 were tested with Bland-Altman plots in this study; however, evaluation and comparison of data with previous similar studies that mostly used t tests should be valued accordingly and cautiously. Errors in evaluating PA radiographs include positioning the head, reproducing head posture, radiographic errors related to projection, and errors in identifying landmarks.3,17,18 In this study, only identification errors were evaluated. In order to understand the relative horizontal or vertical divergence between landmarks, all landmarks were measured on both the x-axis and the y-axis using a coordinate plane. The results will help orthodontists to be cautious in using a landmark with a large horizontal divergence for a transverse measurement, or a landmark with a large vertical divergence for detection of a cant of the occlusal or maxillary plane. The most challenging landmarks lie on curved lines, in areas of low contrast, and with other structures superimposed.4 In this context, some midline landmarks have low contrast, and most of the time they are
August 2016 Vol 150 Issue 2
superimposed with other structures. In this study, 6 of 7 midline landmarks showed high ICC values; only crista galli, which is the center of a constructed asymmetric quadrilateral, was found to be problematic in the interexaminer agreement test. Crista galli showed only moderate consistency between the 2 examiners in the y coordinates at both time points. The extent of disagreement is shown in the plots (Figs 2-4). The Bland-Altman test showed that the points were scattered, and some points did not lie within the limits of agreement. Our results are consistent with previous reports, which stated that constructed lines might not help to increase the reliability of landmark identification.3,19 The greater wing inferior and superior orbit, maxillary point, menton, antegonial notch, mandibular and maxillary molar point, upper and lower incisor point, and upper and lower incisor edge landmarks had excellent agreement between the 2 examiners at the 2 time points in both the x and y coordinates; this shows that they are reliable landmarks for research and clinical diagnosis. Dental landmark interexaminer agreement was higher than reported in previous research. Leonardi et al4 reported that no information was given about subjects' age in any study that they reviewed, but age is important because superimposition of the dentition may present challenges in locating dental landmarks. The fact that the ages of the patients in this study were between 18 and 45 years might have enabled us to locate dental landmarks more easily. Similarly, Sicurezza et al20 concluded that the superimposition of the third molars could change the ease of identification of dental landmarks in patients between the ages of 11 and 15 years. Most other studies were carried out on skulls, so that locating landmarks would be easier and more consistent.5 On the other hand, points placed on curves with large diameters or superimposed areas (condyle superior, gonion, mastoid process, nasal cavity, zygomatic process) had poorer ICC values with more scattered points in the Bland-Altman graphs and were hard to locate consistently in this study also, similar to other relevant cephalometric studies.17-20 Some landmarks on curves but not superimposed on other structures had higher ICC values: eg, maxillary point and antegonial notch. On the other hand, most of the ICC values for condylar landmarks, which are usually superimposed with the cranial base or the zygomatic arch, and therefore harder to identify, were not excellent but were acceptable (ICC above 0.85) in this study.3,19 In the context of the greater deviation values of the y coordinate than of the x and higher ICC values for interexaminer than for intraexaminer correlations, our
American Journal of Orthodontics and Dentofacial Orthopedics
Ulkur et al
findings were similar to those in the 2 studies of Major et al.7,8 Parallel with our results, Pirttiniemi et al9 tested the interexaminer correlation; the condyles superior right and left were deviated on the x and y coordinates, and the deviations were lower for measurements made on skulls than on patients. The reason for the low deviations of the landmarks, which are marked on the skull cephalograms, was the absence of soft tissue superposition. Landmarks that showed high interexaminer identification errors should be used as approximate measurements.8 As long as the standard deviation on the x-axis of a landmark is low, this landmark can be used safely in transverse measurements. Major et al8 concluded that landmark identification errors greater than 1.5 mm should probably be avoided, and those greater than 2.5 mm should definitely be avoided. The standard deviations of the landmarks evaluated in our study were in the range between these values that were stated, except for the gonion landmark, for which the standard deviation was higher than 1.5 mm on the x coordinate for the intraexaminer tests.7 Although agreement within the examiners was achieved by clearly defining the landmarks before the study, the results showed slightly more interexaminer variability on both the x-axis and y-axis than did intraexaminer landmark identification. The implication of this result may be that in orthodontic and maxillofacial centers, landmark identifications should be carried out using a guide including precise descriptions of the landmarks or having a calibration session at every identification session to obtain more accurate results. Cephalometric landmark identification errors are based not only on difficulty of identification because of localization on broad and flat curves, but also on inaccurate anatomic descriptions of the landmarks. The source of error can be reduced when the definition of a landmark is precisely written.8 Therefore, special attention was given to clearly explain the anatomic locations of the landmarks in written form to the examiners. We believe that the overall high ICC values may be partly explained by the expertise of the examiners, who are experienced orthodontists. Major et al8 mentioned operator experience as an important factor because of the increased familiarity with the anatomic structures and their radiologic appearance. CONCLUSIONS
There are fewer errors in intraexaminer than in interexaminer correlations in landmark identifications on PA radiographs. All landmarks investigated except crista galli showed good agreement between measurements.
331
REFERENCES 1. de Moraes ME, Hollender LG, Chen CS, Moraes LC, Balducci I. Evaluating craniofacial asymmetry with digital cephalometric images and cone-beam computed tomography. Am J Orthod Dentofacial Orthop 2011;139:523-31. 2. van Vlijmen OJ, Kuijpers MA, Berge SJ, Schols JG, Maal TJ, Breuning H, et al. Evidence supporting the use of cone-beam computed tomography in orthodontics. J Am Dent Assoc 2012; 143:241-52. 3. Baumrind S, Frantz RC. The reliability of head film measurements 1. Landmark identification. Am J Orthod 1971;60:111-27. 4. Leonardi R, Annunziata A, Caltabiano M. Landmark identification error in posteroanterior cephalometric radiography. A systematic review. Angle Orthod 2008;78:761-5. 5. H€agg U, Cooke MS, Chan TC, Tng TT, Lau PY. The reproducibility of cephalometric landmarks: an experimental study on skulls. Aust Orthod J 1998;15:177-85. 6. Adenwalla ST, Kronman JH, Attarzadeh F. Porion and condyle as cephalometric landmarks—an error study. Am J Orthod Dentofacial Orthop 1988;94:411-5. 7. Major PW, Johnson DE, Hesse KL, Glover KE. Effect of head orientation on posterior anterior cephalometric landmark identification. Angle Orthod 1996;66:51-60. 8. Major PW, Johnson DE, Hesse KL, Glover KE. Landmark identification error in posterior anterior cephalometrics. Angle Orthod 1994; 64:447-54. 9. Pirttiniemi P, Miettinen J, Kantomaa T. Combined effects of errors in frontal-view asymmetry diagnosis. Eur J Orthod 1996;18: 629-36. 10. Kish L. Sampling organizations and groups of unequal sizes. Am Sociol Rev 1965;30:564-72. 11. Donatelli RE, Lee SJ. How to report reliability in orthodontic research: part 1. Am J Orthod Dentofacial Orthop 2013;144:156-61. 12. Fleiss JL. Reliability of measurement. In: Fleiss JL, editor. The design and analysis of clinical experiments. New York: John Wiley and Sons; 1986. p. 1-32. 13. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;327: 307-10. 14. Lou L, Lagravere MO, Compton S, Major PW, Flores-Mir C. Accuracy of measurements and reliability of landmark identification with computed tomography (CT) techniques in the maxillofacial area: a systematic review. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2007;104:402-11. 15. Danforth RA, Dus I, Mah J. 3-D volume imaging for dentistry: a new dimension. J Calif Dent Assoc 2003;31:817-23. 16. Ludlow JB, Ludlow-Davies LE, White SC. Patient risk related to common dental radiographic examinations: the impact of 2007 International Commission on Radiological Protection recommendations regarding dose calculation. J Am Dent Assoc 2008;139: 1237-43. 17. Vincent A, West VC. Cephalometric landmark identification error. Aust Orthod J 1987;10:98-104. 18. El-Mangory EH, Shaheen SI, Mostafa YA. Landmark identification in computerized posterior-anterior cephalometrics. Am J Orthod Dentofacial Orthop 1987;91:57-61. 19. Oshagh M, Shahidi SH, Danaei SH. Effects of image enhancement on reliability of landmark identification in digital cephalometry. Indian J Dent Res 2013;24:98-103. 20. Sicurezza E, Greco M, Giordano D, Maiorana F, Leonardi R. Accuracy of landmark identification on postero-anterior cephalograms. Prog Orthod 2012;13:132-40.
American Journal of Orthodontics and Dentofacial Orthopedics
August 2016 Vol 150 Issue 2
Landmark errors on posteroanterior cephalograms Feyza Ulkur,a Fulya Ozdemir,b Derya Germec-Cakan,b and E. Cigdem Kasparc Istanbul, Turkey
Introduction: It is important to reduce the method errors when evaluating posteroanterior cephalograms to see either small deviations from normal or transverse changes caused by orthodontic treatment. The aim of this study was to determine horizontal and vertical intraexaminer and interexaminer agreement in localization of landmarks in posteroanterior cephalograms of adult patients. Methods: The sample was gathered retrospectively from the archives of the Department of Orthodontics of Yeditepe University in Istanbul, Turkey. Radiographs of 39 patients diagnosed with skeletal asymmetries (20 women, 19 men) were drawn manually, and a coordinate system was established with software. The tracings were made by 2 operators, after a calibration session on 29 landmarks (22 bilateral, 7 midline). Intraclass correlation coefficients and the Bland-Altman test were used for detecting interexaminer and intraexaminer agreement for each cephalometric variable. Results: The interexaminer agreement test showed that the most problematic landmark was crista galli, which showed moderate consistency between 2 examiners in the y coordinates at 2 time points. Condylar and zygomatic landmarks showed good agreement. The greater wing inferior and superior orbit, maxillary point, menton, anterior nasal spine, antegonial notch, mandibular and maxillary molar point, maxillary and mandibular incisor point, and maxillary and mandibular incisor edge landmarks had excellent agreement between the 2 examiners at the 2 time points in both the x and y coordinates. Conclusions: There are fewer errors in intraexaminer than in interexaminer correlations in landmark identifications on posteroanterior radiographs. All landmarks investigated except crista galli showed good agreement between measurements. (Am J Orthod Dentofacial Orthop 2016;150:324-31)
P
osteroanterior (PA) cephalometric radiographs help to evaluate maxillofacial structures in the transverse plane and are requested by the clinician if there is doubt about any problems in the transverse plane. Even though 3-dimensional (3D) cone-beam computed tomography technology is now available at most clinics, and it seems to give more reliable and extensive information on the asymmetry of the face,1 where this technology is not yet available, or when most of the available archive for comparison and longitudinal research is still only in 2-dimensional (2D) records, PA radiographs are still used.2 For the imaging to be useful, there should be reliably identified landmarks From Yeditepe University, Istanbul, Turkey. a Lecturer, Department of Orthodontics, Faculty of Dentistry. b Associate professor, Department of Orthodontics, Faculty of Dentistry. c Assistant professor, Department of Biostatistics and Medical Informatics, Department of Medical Education. All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest, and none were reported. Address correspondence to: Feyza Ulkur, Department of Orthodontics, Faculty of Dentistry, Yeditepe University, Bagdat Cad. No: 238, Goztepe, Istanbul 34728, Turkey; e-mail, [email protected]. Submitted, February 2015; revised and accepted, January 2016. 0889-5406/$36.00 Ó 2016 by the American Association of Orthodontists. All rights reserved. http://dx.doi.org/10.1016/j.ajodo.2016.01.016
324
for accurate measurements.3 A recently published systematic review4 concluded that there is only 1 article on landmark errors that fulfilled the selection criteria, and that study was performed on dry skulls.5 This review article cautioned clinicians to take care in interpreting studies that used dry skulls because another article by H€agg et al5 stated that skeletal and dental angular errors are greater, up to 4 times greater for some measurements, in the presence of soft tissues. There are limitations in using these radiographs: eg, difficulty in positioning the head and reproducing head posture, radiographic errors related to projection, errors in identifying landmarks, and finding a straight reference line. Regarding errors in identifying landmarks, the most problematic points lie on the curved lines, in areas of low contrast, and with other structures superimposed.6 Errors of interpretation are reduced by clear definitions and training.7-9 Therefore, the aim of this study was to determine the horizontal and vertical intraexaminer and interexaminer identification errors in the localization of landmarks in the PA radiographs of adult patients. The null hypothesis was that there are no differences in the accuracy and reproducibility of landmark identifications between 2 orthodontists at 2 time points, 1 month apart.
Ulkur et al
325
Table I. Definitions for the cephalometric landmarks used in the study
Fig 1. PA cephalometric landmarks and coordinate system used in the study. MATERIAL AND METHODS
The sample PA radiographs of this study, which had the same resolution and quality, were gathered randomly from the 10-year archive at the Department of Orthodontics, Faculty of Dentistry, of Yeditepe University, Istanbul, Turkey. The radiographs belonged to 39 asymmetric patients (20 women, 19 men) ranging in age from 18 to 45 years (mean 6 SD, 31.3 6 12.2 years). The following were the inclusion criteria: no cleft and lip, and no diagnosed systemic diseases or craniofacial syndromes. Patient data were handled according to the requirements and recommendations of the Declaration of Helsinki. Ethical approval was obtained from the institutional review board of Yeditepe University. The cephalometric radiographs used in this study were created using a Promax x-ray device (Planmeca, Helsinki, Finland) with the patients were in natural head position. Source to ear rod distance was 160 cm, and ear rod to film distance was 17.5 mm. All radiographs were covered with tracing paper. For each radiograph, an individual coordinate system was formed. On each corner of the paper, a large plus sign was drawn to trace the papers identically, and the sign on the upper right corner was used used as the x and y coordinates (Fig 1). The tracings were made by 2 operators (F.U. and D.G.C.), who are experienced orthodontists, after a calibration session with the written definitions of the points and practical trials were carried out together. A
Landmark Definition Bilateral skeletal landmarks (18 landmarks) Greater wing superior Intersection of the superior border of orbit (GWSO) the greater wing of the sphenoid bone and the lateral orbital margin Greater wing inferior Intersection of the inferior border of orbit (GWIO) the greater wing of the sphenoid bone and the lateral orbital margin Zygomatic (Z) Most lateral aspect of the zygomatic arch Condyle superior (CS) Most superior aspect of the condyle Mastoid process (MP) Most inferior point on the mastoid process Nasal cavity (NC) Most lateral point on the nasal cavity Maxillary point (MP) Center of the concavity of the zygomatic process of the maxilla Gonion (G) Midpoint on the curvature at the angle of the mandible (gonion) Antegonial (AG) Deepest point on the curvature of the antegonial notch Midline skeletal landmarks (3 landmarks) Crista galli (CG) Geometric center of crista galli Anterior nasal spine Center of the intersection of the nasal (ANS) septum and the palate Menton (ME) Midpoint on the inferior border of the mental protuberance Bilateral dental landmarks (4 landmarks) Maxillary molar (MX6) Midpoint on the buccal surface of the maxillary first molar Mandibular molar Midpoint on the buccal surface of the (ML6) mandibular first molar Midline dentoalveolar landmarks (4 landmarks) Upper incisor point Crest of the alveolus between the (U1P) maxillary central incisors Upper incisor edge Midpoint on the incisal edges of the (U1E) maxillary incisors Lower incısor edge Midpoint on the incisal edges of the (L1E) mandibular incisors Lower incisor point Crest of the alveolus between the (L1E) mandibular central incisors
total of 29 landmarks were marked on tracing paper manually using a 0.1-mm pen (Ecco pigment; FaberCastell, Stein, Germany): 22 landmarks were bilateral, and 7 were on the midline. The list and the definitions of landmarks marked on each PA cephalometric radiograph are given in Table I. The operators repeated the tracings 1 month later. The tracing papers were scanned into digital format at 300 dpi using a scanner (1680 Pro; Epson, Nagano, Japan) with 1600 dpi imaging, 40 3 800 pixels per line, and 48bit color depth for both film and reflective scanning, and displayed on a 15-in, 1024 3 768 high-pixel resolution monitor (model FP581, Benq, Taipei, Taiwan) with a pixel pitch of 0.297 mm, a contrast ratio of 450:1, and brightness of 250 candela per square meter. Each scanned paper
American Journal of Orthodontics and Dentofacial Orthopedics
August 2016 Vol 150 Issue 2
Ulkur et al
326
Table II. Intraclass correlation coefficients and 95% confidence interval for intra- and interexaminer agreement Intraexaminer (F.U.) Landmark Greater wing superior orbit, right Greater wing superior orbit, left Greater wing inferior orbit, right Greater wing inferior orbit, left Zygomatic, right Zygomatic, left Condyle superior, right Condyle superior, left Mastoid process, right Mastoid process, left Nasal cavity, right Nasal cavity, left Maxillary point, right Maxillary point, left Gonion, right Gonion, left Antegonial, right Antegonial, left Crista galli Anterior nasal spine Menton Maxillary first molar, right Maxillary first molar, left Mandibular first molar, right Mandibular first molar, left Upper incisor point Upper incisor edge Lower incısor edge Lower incisor point
Intraexaminer (D.G.C.)
Interexaminer marking 1
Interexaminer marking 2
x-axis 0.9965
y-axis 0.9873
x-axis 0.9924
y-axis 0.9846
x-axis 0.9867
y-axis 0.9715
x-axis 0.9905
y-axis 0.9683
0.9959
0.9643
0.9958
0.9557
0.9944
0.9815
0.9915
0.9681
0.9953
0.9912
0.9919
0.9844
0.9927
0.9632
0.9948
0.9690
0.9971
0.9666
0.9932
0.9558
0.9972
0.9811
0.9949
0.9785
0.9809 0.9960 0.9892 0.9959 0.9892 0.9931 0.9938 0.9964 0.9911 0.9929 0.9861 0.8770 0.9854 0.9919 0.9966 0.9946 0.9901 0.9931
0.9878 0.9600 0.9612 0.9424 0.9984 0.9788 0.9569 0.9708 0.9821 0.9729 0.9972 0.9918 0.9972 0.9911 0.9740 0.9782 0.9977 0.9936
0.9915 0.9962 0.9129 0.9935 0.8834 0.9917 0.9935 0.9954 0.9913 0.9928 0.9854 0.9325 0.9880 0.9931 0.9938 0.9690 0.9874 0.9910
0.9873 0.9525 0.9743 0.9722 0.9863 0.9829 0.9752 0.9654 0.9587 0.9585 0.9410 0.9876 0.9985 0.9878 0.9158 0.9711 0.9975 0.9950
0.8603 0.9988 0.8968 0.9799 0.9847 0.9954 0.9970 0.9972 0.9912 0.9982 0.9882 0.8757 0.9937 0.9979 0.9914 0.9974 0.9774 0.9908
0.9671 0.9777 0.8556 0.8878 0.9978 0.9987 0.9064 0.8975 0.9275 0.9745 0.9855 0.9934 0.9968 0.9967 0.7961 0.9790 0.9994 0.9661
0.8704 0.9960 0.8691 0.9801 0.9178 0.9937 0.9929 0.9954 0.9933 0.9951 0.9900 0.9369 0.9903 0.9935 0.9939 0.9632 0.9850 0.9890
0.9766 0.9666 0.8815 0.9149 0.9899 0.9839 0.9665 0.9493 0.9688 0.9739 0.9338 0.9940 0.9972 0.9941 0.7878 0.9818 0.9986 0.9693
0.9933
0.9856
0.9934
0.9774
0.9977
0.9552
0.9960
0.9551
0.9937
0.9950
0.9887
0.9897
0.9957
0.9784
0.9938
0.9759
0.9929
0.9859
0.9934
0.9825
0.9976
0.9845
0.9961
0.9812
0.9937 0.9876 0.9851 0.9926
0.9929 0.9941 0.9889 0.9950
0.9911 0.9880 0.9866 0.9906
0.9839 0.9897 0.9897 0.9912
0.9992 0.9973 0.9977 0.9993
0.9696 0.9937 0.9857 0.9814
0.9965 0.9971 0.9960 0.9964
0.9797 0.9929 0.9854 0.9793
was saved as a portable document file (PDF). The coordinate system was established with software (AutoCAD, 2004 version; Autodesk, San Rafael, Calif). Statistical calculations were carried out with software (NCSS, Kaysville, Utah) for Windows. In addition to descriptive statistics (means, standard deviations), intraclass correlation coefficients (ICCs) were used for determining interexaminer and intraexaminer agreement for each cephalometric variable. ICC, derived from analysis of variance, assesses rating reliability by comparing the variability of the different ratings of the same subject to the total variation across all ratings and all subjects. It is a measure of the homogeneity of elements within clusters and has a maximum value of 1 when there is complete homogeneity within clusters.10 For the detection of interexaminer error, the mean of the 2
August 2016 Vol 150 Issue 2
measurements done by each examiner was used. To better understand the disagreement seen for controversial points, Bland-Altman plots were used to explore the results in a graphic approach.11 RESULTS
The results showed that each operator was consistent in the repeated measurements (intraexaminer agreement) in most landmarks marked (Table II). ICC values of 0.75 or above are considered good, and those above 0.90 are excellent.12 In this section, landmarks with an ICC less than 0.90 are reported along with the millimeter difference between measurements for ease of comparison with previous studies. One examiner (F.U.) had ICC values of 0.88 and 0.99 for the x and y coordinates of the left gonion. The mean
American Journal of Orthodontics and Dentofacial Orthopedics
Ulkur et al
0.52 6 0.60 0.17 6 0.86 0.34 6 0.73 0.31 6 1.23 1.04 6 0.68
0.76 6 0.85
0.24 6 1.26
0.60 6 0.54
0.72 6 0.32 0.40 6 1.04 0.34 6 0.92 0.55 6 0.97 0.41 6 0.93 0.75 6 0.97 0.20 6 0.58 0.29 6 1.21 0.20 6 0.79 0.03 6 0.60
0.27 6 0.54 0.16 6 1.35
0.70 6 0.61 0.02 6 1.08
0.50 6 0.58 0.71 6 1.04
y-axis, mean difference 6 SD 0.33 6 1.02 0.90 6 0.16 x-axis, mean difference 6 SD 0.84 6 0.40 0.74 6 036 y-axis, mean difference 6 SD 0.84 6 0.31 0.93 6 0.12 x-axis, mean difference 6 SD 0.73 6 1.94 0.90 6 0.12 y-axis, mean difference 6 SD 0.40 6 1.11 0.35 6 1.03 x-axis, mean difference 6 SD 0.29 6 0.52 0.47 6 0.35 y-axis, mean difference 6 SD 0.84 6 0.09 0.65 6 0.36 x-axis, mean difference 6 SD 0.37 6 1.28 0.00 6 0.69
Landmark Gonion, left Mastoid process, right Zygomatic, right Condyle superior, right Crista galli
Interexaminer marking 2 Interexaminer marking 1 Intraexaminer (D.G.C.) Intraexaminer (F.U.)
and 90%
Table III. Mean difference and standard deviations of the landmarks, which have a confidence interval for intra- and interexaminer agreement between 75%
327
Fig 2. Bland-Altman plot of the landmark crista galli. The interexaminer comparison of markings: A, on the x coordinate; B, on the y coordinate.
difference of the 2 markings for this landmark was 0.37 6 1.28 mm for the x coordinate (Table III). The x coordinate of the right mastoid process showed an ICC of 0.88 by 1 examiner (F.U.). The mean difference of the 2 markings for this landmark was 0.47 6 0.35 mm for the x coordinate. The rest of the intraexaminer ICC values were mostly higher than 0.90 or 0.95, showing excellent agreement (Table II). In the interexaminer evaluation, the right zygomatic landmark had ICC values of 0.86 and 0.87 for the x and y coordinates, respectively; these were greater than 0.75 (Table II). The mean difference of the zygomatic landmark for the x coordinate was 0.37 6 0.93 mm (Table III). In our study, the landmarks with low ICC values also exceeded the confidence interval (Figs 2-4) for the Bland-Altman test.13 Bland-Altman plots show scattergrams of the relationship between the means and the differences
American Journal of Orthodontics and Dentofacial Orthopedics
August 2016 Vol 150 Issue 2
Ulkur et al
328
Fig 3. Bland-Altman plot of the landmark gonion. The interexaminer comparison of markings: A, on the x coordinate; B, on the y coordinate.
Fig 4. Bland-Altman plot of the landmark zygomatic. The interexaminer comparison of markings: A, on the x coordinate; B, on the y coordinate.
of the 2 measurements. If the differences have a normal distribution, the scattering of the differences should be marked around zero, and 95% of the marks should be between 1.96 and 11.96. Then we concluded that there is no relationship between the means and the differences; 61.96 is considered the confidence interval, and points marked between these lines are considered within the limits of agreement. Each Bland-Altman diagram describes either the horizontal (Figs 2, A; 3, A; and 4, A) or vertical (Figs 2, B; 3, B; and 4, B) limits of agreement of the points marked. Bland-Altman plots can be described as follows: x-axis defines the average values of measurements obtained from the first and second examiners, (examiner 1 1 examiner 2)/2, across a range of values. The y-axis defines the difference between the values of the 2 measurements of the first and second examiners, (examiner 2 examiner 1)/2. Each dot represents an individual difference. The horizontal line between the dotted lines represents the
bias: ie, the average of the difference values. The dotted lines represent precision: ie, the limits of agreement. The dots outside the limits of agreement (dotted lines) are called outliers. When all differences are not positive, it means that there is no systematic bias. The dotted horizontal lines represent the 95% confidence limits (limits of agreement). For the horizontal change in the position of the zygomatic point, the agreement level is shown in Figure 4 as a Bland-Altman plot. The range of values at the x-axis is between 20 and 55. The differences vary from 0.4 to 3.6; not all differences are positive, meaning that there is no systematic bias. We noted that 2 difference scores exceeded the limits of agreement; one exceeds the lower limit, and the other exceeds the upper limit. When the Bland-Altman plot was examined to determine whether the differences in values depended on the measurement value (the average of the 2 measurements on the x-axis), we found that the differences were scattered around the
August 2016 Vol 150 Issue 2
American Journal of Orthodontics and Dentofacial Orthopedics
Ulkur et al
bias, with no pattern. For the vertical change in the position of the zygomatic point, the x-axis lies across the range of values between 6 and 22. The y-axis represents the difference in values obtained from the 2 examiners. In this case, the differences varied from 1.62 to 1.31; not all differences are positive, which means that there is no systematic bias. We noted that 3 difference scores exceeded the limits of agreement; 1 exceeded the lower limit, and 2 exceeded the upper limit. When the BlandAltman plot was examined to determine whether the differences in values depended on the measurement value (the average of the 2 measurements on the x-axis), we saw that the differences were scattered around the bias, with no pattern. The right condyle superior landmark showed an interexaminer ICC between 0.85 and 0.90 on both the x and y coordinates. The mean difference for interexaminer tests of the left condyle superior landmark on the x coordinate was 0.65 6 0.97 mm, and 0.65 6 1.0 mm was the mean differences on the y coordinate for both time points (Table III). The left gonion landmark had an interexaminer ICC of 0.87 on the x coordinate, and the mean difference of the 2 markings was 0.73 6 1.94 mm. The limits of agreement for gonion are shown in Figure 3 in the Bland-Altman plot. In Figure 3, A, the horizontal deviation in the position of gonion is shown, where the x-axis represents the average values between 20 and 55. The yaxis represents the difference in values obtained from the 2 examiners. The differences vary from 1.61 to 1.10, and not all differences are positive; this means that there was no systematic bias. We noted that 3 of the difference scores exceeded the limits of agreement; 2 exceeded the lower limit, and 1 the upper limit. When the Bland-Altman plot was examined to determine whether the differences in values depended on the measurement value (the average of the 2 measurements on the x-axis), it appeared to have no pattern, with the differences scattered around the bias. In Figure 3, B, when we examined the vertical deviation in the position of gonion, the x-axis represented the average values between 15 and 40. The y-axis represented the differences obtained from the 2 examiners. The differences varied from 4.3 to 2.8; not all differences were positive; this means that there was no systematic bias. We noted that 2 difference scores exceeded the limits of agreement; one exceeded the lower limit, and the other exceeded the upper limit. When the Bland-Altman plot was examined to determine whether the differences in values depended on the measurement value (the average of the 2 measurements on the x-axis), it appeared to have no pattern, with the differences scattered around the bias.
329
The crista galli landmark had an interexaminer ICC of 0.80 on the y coordinate (Table II). The mean difference was 0.25 6 0.84 mm. The limits of agreement for crista galli are shown in the Bland-Altman plot (Fig 2). In Figure 2, A, the horizontal deviation in its position is shown; the standard deviation of the different scores for the x-axis data was approximately 0.77. We noted that 3 difference scores exceeded the lower limit of agreement. When the Bland-Altman plot was examined to determine whether the differences depended on the measurement value (the average of the 2 measurements on the x-axis), we saw that the differences were scattered around the bias, with no pattern (Fig 2, A). In the examination of the vertical deviation in the position of crista galli, the x-axis represented the average values obtained from the 2 examiners across the range of values between 60 and 40. The y-axis represented the difference in values obtained from the 2 examiners. The differences varied from 1.6 to 2.0; not all differences were positive, meaning that there was no systematic bias. We noted that 3 difference scores exceeded the limits of agreement; 1 exceeded the lower limit, and 2 exceeded the upper limit. When the Bland-Altman plot was examined to determine whether the differences depended on the measurement value (the average of the 2 measurements on the x-axis), it appeared that the differences were scattered around the bias, with no pattern. Other than these, the investigated landmarks showed ICC values greater than 0.90, and a few were greater than or equal to 0.95. The greater wing inferior and superior orbit, maxillary point, menton, anterior nasal spine, antegonial notch, mandibular and maxillary molar point, upper and lower incisor point, and upper and lower incisor edge landmarks showed excellent agreement between the 2 examiners in both the x and y coordinates (Table II). DISCUSSION
Cephalometric evaluations of patients with orthodontic malocclusions have been traditionally carried out on lateral and frontal PA cephalographs since the early 20th century when the orthodontist needed to diagnose facial skeletal imbalances to assess growth or response to treatment. Recently, with the popularity of cone-beam computed tomography, orthodontists gather 3D data mostly to evaluate the airway, impacted teeth, and the temporomandibular joint.14,15 When the patient's dose of radiation is reduced substantively, cone-beam computed tomography can be used instead of panoramic, PA, or lateral cephalometric radiographs.16 However, even if there are 3D data, orthodontists usually generate and measure 2D cephalometric radiographs, with which they are more familiar. Until the
American Journal of Orthodontics and Dentofacial Orthopedics
August 2016 Vol 150 Issue 2
Ulkur et al
330
problems with 3D evaluations are solved and the technology is widely used by orthodontists, the problem of landmark identification still exists for 2D evaluations. Precision and reproducibility are important when evaluating a 2D cephalometric radiograph because quantitative, systematic, and objective measurements are needed for diagnosis of a patient's malocclusion so that a satisfactory treatment plan can be formed. The reason for using ICC calculations in this study was to evaluate whether the results for 2 intraexaminer and interexaminer measurements showed agreement. The ICC assesses rating reliability by comparing the variability of the different ratings of the same subject to the total variations across all ratings and all subjects for continuous data. A graphic method is often used as an alternative to the ICC for analyzing interrater reliability data, as proposed by Bland and Altman.13 It combines a graphic approach and a quantitative analysis of the magnitude of the rating differences. Part of the popularity of the Bland-Altman method stems from its graphic nature; t tests that would compare the means of 2 groups were not used because deviations in a few values could affect group means and might cause mathematical errors regarding the results of this study. Correlation was also not used because it mainly shows a relationship, not agreement, between data sets. Therefore, ICC and agreement were used to assess the data. It is recommended to use Bland-Altman plots mainly as an exploratory technique. They allow researchers to have a first glimpse into the interrater reliability results. Ultimately, an ICC based on the appropriate statistical model should be calculated. The mean differences and deviations of the landmarks with ICC values less than 0.90 were tested with Bland-Altman plots in this study; however, evaluation and comparison of data with previous similar studies that mostly used t tests should be valued accordingly and cautiously. Errors in evaluating PA radiographs include positioning the head, reproducing head posture, radiographic errors related to projection, and errors in identifying landmarks.3,17,18 In this study, only identification errors were evaluated. In order to understand the relative horizontal or vertical divergence between landmarks, all landmarks were measured on both the x-axis and the y-axis using a coordinate plane. The results will help orthodontists to be cautious in using a landmark with a large horizontal divergence for a transverse measurement, or a landmark with a large vertical divergence for detection of a cant of the occlusal or maxillary plane. The most challenging landmarks lie on curved lines, in areas of low contrast, and with other structures superimposed.4 In this context, some midline landmarks have low contrast, and most of the time they are
August 2016 Vol 150 Issue 2
superimposed with other structures. In this study, 6 of 7 midline landmarks showed high ICC values; only crista galli, which is the center of a constructed asymmetric quadrilateral, was found to be problematic in the interexaminer agreement test. Crista galli showed only moderate consistency between the 2 examiners in the y coordinates at both time points. The extent of disagreement is shown in the plots (Figs 2-4). The Bland-Altman test showed that the points were scattered, and some points did not lie within the limits of agreement. Our results are consistent with previous reports, which stated that constructed lines might not help to increase the reliability of landmark identification.3,19 The greater wing inferior and superior orbit, maxillary point, menton, antegonial notch, mandibular and maxillary molar point, upper and lower incisor point, and upper and lower incisor edge landmarks had excellent agreement between the 2 examiners at the 2 time points in both the x and y coordinates; this shows that they are reliable landmarks for research and clinical diagnosis. Dental landmark interexaminer agreement was higher than reported in previous research. Leonardi et al4 reported that no information was given about subjects' age in any study that they reviewed, but age is important because superimposition of the dentition may present challenges in locating dental landmarks. The fact that the ages of the patients in this study were between 18 and 45 years might have enabled us to locate dental landmarks more easily. Similarly, Sicurezza et al20 concluded that the superimposition of the third molars could change the ease of identification of dental landmarks in patients between the ages of 11 and 15 years. Most other studies were carried out on skulls, so that locating landmarks would be easier and more consistent.5 On the other hand, points placed on curves with large diameters or superimposed areas (condyle superior, gonion, mastoid process, nasal cavity, zygomatic process) had poorer ICC values with more scattered points in the Bland-Altman graphs and were hard to locate consistently in this study also, similar to other relevant cephalometric studies.17-20 Some landmarks on curves but not superimposed on other structures had higher ICC values: eg, maxillary point and antegonial notch. On the other hand, most of the ICC values for condylar landmarks, which are usually superimposed with the cranial base or the zygomatic arch, and therefore harder to identify, were not excellent but were acceptable (ICC above 0.85) in this study.3,19 In the context of the greater deviation values of the y coordinate than of the x and higher ICC values for interexaminer than for intraexaminer correlations, our
American Journal of Orthodontics and Dentofacial Orthopedics
Ulkur et al
findings were similar to those in the 2 studies of Major et al.7,8 Parallel with our results, Pirttiniemi et al9 tested the interexaminer correlation; the condyles superior right and left were deviated on the x and y coordinates, and the deviations were lower for measurements made on skulls than on patients. The reason for the low deviations of the landmarks, which are marked on the skull cephalograms, was the absence of soft tissue superposition. Landmarks that showed high interexaminer identification errors should be used as approximate measurements.8 As long as the standard deviation on the x-axis of a landmark is low, this landmark can be used safely in transverse measurements. Major et al8 concluded that landmark identification errors greater than 1.5 mm should probably be avoided, and those greater than 2.5 mm should definitely be avoided. The standard deviations of the landmarks evaluated in our study were in the range between these values that were stated, except for the gonion landmark, for which the standard deviation was higher than 1.5 mm on the x coordinate for the intraexaminer tests.7 Although agreement within the examiners was achieved by clearly defining the landmarks before the study, the results showed slightly more interexaminer variability on both the x-axis and y-axis than did intraexaminer landmark identification. The implication of this result may be that in orthodontic and maxillofacial centers, landmark identifications should be carried out using a guide including precise descriptions of the landmarks or having a calibration session at every identification session to obtain more accurate results. Cephalometric landmark identification errors are based not only on difficulty of identification because of localization on broad and flat curves, but also on inaccurate anatomic descriptions of the landmarks. The source of error can be reduced when the definition of a landmark is precisely written.8 Therefore, special attention was given to clearly explain the anatomic locations of the landmarks in written form to the examiners. We believe that the overall high ICC values may be partly explained by the expertise of the examiners, who are experienced orthodontists. Major et al8 mentioned operator experience as an important factor because of the increased familiarity with the anatomic structures and their radiologic appearance. CONCLUSIONS
There are fewer errors in intraexaminer than in interexaminer correlations in landmark identifications on PA radiographs. All landmarks investigated except crista galli showed good agreement between measurements.
331
REFERENCES 1. de Moraes ME, Hollender LG, Chen CS, Moraes LC, Balducci I. Evaluating craniofacial asymmetry with digital cephalometric images and cone-beam computed tomography. Am J Orthod Dentofacial Orthop 2011;139:523-31. 2. van Vlijmen OJ, Kuijpers MA, Berge SJ, Schols JG, Maal TJ, Breuning H, et al. Evidence supporting the use of cone-beam computed tomography in orthodontics. J Am Dent Assoc 2012; 143:241-52. 3. Baumrind S, Frantz RC. The reliability of head film measurements 1. Landmark identification. Am J Orthod 1971;60:111-27. 4. Leonardi R, Annunziata A, Caltabiano M. Landmark identification error in posteroanterior cephalometric radiography. A systematic review. Angle Orthod 2008;78:761-5. 5. H€agg U, Cooke MS, Chan TC, Tng TT, Lau PY. The reproducibility of cephalometric landmarks: an experimental study on skulls. Aust Orthod J 1998;15:177-85. 6. Adenwalla ST, Kronman JH, Attarzadeh F. Porion and condyle as cephalometric landmarks—an error study. Am J Orthod Dentofacial Orthop 1988;94:411-5. 7. Major PW, Johnson DE, Hesse KL, Glover KE. Effect of head orientation on posterior anterior cephalometric landmark identification. Angle Orthod 1996;66:51-60. 8. Major PW, Johnson DE, Hesse KL, Glover KE. Landmark identification error in posterior anterior cephalometrics. Angle Orthod 1994; 64:447-54. 9. Pirttiniemi P, Miettinen J, Kantomaa T. Combined effects of errors in frontal-view asymmetry diagnosis. Eur J Orthod 1996;18: 629-36. 10. Kish L. Sampling organizations and groups of unequal sizes. Am Sociol Rev 1965;30:564-72. 11. Donatelli RE, Lee SJ. How to report reliability in orthodontic research: part 1. Am J Orthod Dentofacial Orthop 2013;144:156-61. 12. Fleiss JL. Reliability of measurement. In: Fleiss JL, editor. The design and analysis of clinical experiments. New York: John Wiley and Sons; 1986. p. 1-32. 13. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;327: 307-10. 14. Lou L, Lagravere MO, Compton S, Major PW, Flores-Mir C. Accuracy of measurements and reliability of landmark identification with computed tomography (CT) techniques in the maxillofacial area: a systematic review. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2007;104:402-11. 15. Danforth RA, Dus I, Mah J. 3-D volume imaging for dentistry: a new dimension. J Calif Dent Assoc 2003;31:817-23. 16. Ludlow JB, Ludlow-Davies LE, White SC. Patient risk related to common dental radiographic examinations: the impact of 2007 International Commission on Radiological Protection recommendations regarding dose calculation. J Am Dent Assoc 2008;139: 1237-43. 17. Vincent A, West VC. Cephalometric landmark identification error. Aust Orthod J 1987;10:98-104. 18. El-Mangory EH, Shaheen SI, Mostafa YA. Landmark identification in computerized posterior-anterior cephalometrics. Am J Orthod Dentofacial Orthop 1987;91:57-61. 19. Oshagh M, Shahidi SH, Danaei SH. Effects of image enhancement on reliability of landmark identification in digital cephalometry. Indian J Dent Res 2013;24:98-103. 20. Sicurezza E, Greco M, Giordano D, Maiorana F, Leonardi R. Accuracy of landmark identification on postero-anterior cephalograms. Prog Orthod 2012;13:132-40.
American Journal of Orthodontics and Dentofacial Orthopedics
August 2016 Vol 150 Issue 2