- Email: [email protected]
Validation of post mortem dental CT for disaster victim identification
Journal of Forensic Radiology and Imaging 5 (2016) 25–30
Contents lists available at ScienceDirect
Journal of Forensic Radiology and Imaging journal homepage: www.elsevier.com/locate/jofri
Validation of post mortem dental CT for disaster victim identification Thomas D. Ruder a,b,c,n, Yannick A. Thali a,d, Saiful N.A. Rashid e,f, Michael T. Mund a,g, Michael J. Thali a, Gary M. Hatch h, Angi M. Christensen i, Sandra Somaini j, Garyfalia Ampanozi a,c a
Institute of Forensic Medicine, Department of Forensic Medicine and Imaging, University of Zurich, CH-8057 Zurich, Switzerland Institute of Diagnostic, Interventional, and Pediatric Radiology, University Hospital Bern, CH-3010 Bern, Switzerland c Center of Forensic Imaging and Virtopsy, Institute of Forensic Medicine, University of Bern, CH-3012 Bern, Switzerland d Institute of Radiology, Cantonal Hospital Lucerne, CH-6000 Lucerne, Switzerland e Department of Imaging, Faculty of Medicine & Health Sciences, Universiti Putra Malaysia, Malaysia f National Institute of Forensic Medicine, Hospital Kuala Lumpur, Malaysia g JDMT Medical Services AG, CH-8330 Pfäffikon, Switzerland h Radiology-Pathology Center for Forensic Imaging, Departments of Radiology and Pathology, University of New Mexico School of Medicine, Albuquerque, NM 87102, USA i Federal Bureau of Investigation Laboratory, Quantico, VA 22135, USA j Institute of Diagnostic and Interventional Neuroradiology, University Hospital Bern, CH-3010 Bern, Switzerland b
art ic l e i nf o
a b s t r a c t
Article history: Received 27 November 2015 Accepted 25 January 2016 Available online 31 January 2016
The objective of this study was to test the accuracy and inter-reader variability of comparative radiologic identification based on dental post mortem computed tomography (PMCT) and ante mortem (AM) dental radiographs. Five raters with varying degrees of expertize and experience independently compared 115 dental PMCT images to 114 AM dental radiographs to identify matching pairs (n ¼98), unmatched PMCT images (n ¼17), and unmatched AM radiographs (n ¼ 16). Levels of confidence (LOC) and number of concordant features (NOCF) of matched pairs were documented. Accuracy of matches/exclusions, interrater correlation coefficient and correlation between correct matches/exclusions, LOC and NOCF were calculated for all raters. Mean accuracy was 92% for matches and 80% for exclusions. Interrater correlation coefficient regarding LOC and NOCF were 0.623 and 0.907 respectively. LOC were correlated with NOCF of matched pairs but accuracy of matches/exclusions was neither correlated to LOC nor to NOCF. This study shows that visual comparison of PMCT images with AM dental radiographs is a reliable method for identification. Accuracy of identification using PMCT/AM dental radiographs was as high as in comparable studies using post mortem (PM) dental radiographs/AM dental radiographs. Raters with practical experience in forensic identification and experience with the imaging modality (in this case: dental PMCT) achieved higher accuracy than inexperienced raters. Match accuracy did not correlate with subjective confidence or number of concordant features. It is advised to work in teams rather than individually when dealing with real cases in forensic identification, to minimize subjective interpretation and avoid confirmation bias. & 2016 Elsevier Ltd. All rights reserved.
Keywords: Forensic radiology Forensic identification Dental identification Post mortem CT Post mortem dental-CT
1. Introduction Identification of the dead is a fundamental part of forensic investigation [1]. Individual identification relies on comparison of
n Correspondence to: Department of Forensic Medicine and Imaging, Institute of Forensic Medicine, University of Zurich, Winterthurerstrasse 190/52, CH-8057 Zurich, Switzerland. E-mail addresses: [email protected], [email protected] (T.D. Ruder).
http://dx.doi.org/10.1016/j.jofri.2016.01.006 2212-4780/& 2016 Elsevier Ltd. All rights reserved.
ante mortem (AM) and post mortem (PM) data, typically fingerprints, DNA samples, or dental records [2]. Interpol offers general guidelines for victim identification in mass disasters [2]. However there are no internationally recognized guidelines in isolated cases and the procedure differs as a function of local conventions, availability of experts, and the scenario [3]. In the past 15 years, the use of post mortem computed tomography (PMCT) in forensic death investigations increased [4] and its potential for disaster victim identification was put to the test both in actual and simulated mass disaster scenarios [5–7]. Today,
26
T.D. Ruder et al. / Journal of Forensic Radiology and Imaging 5 (2016) 25–30
Fig. 1. Four sample images of corresponding ante mortem radiographs and reconstructed post mortem computed tomography (PMCT) of one quadrant. A–C feature cases where restorative materials are present. D features a case without restorative materials. PMCT has several advantages over plain film dental images, including threedimensional image reformatting, acquisition in a non-invasive fashion, and anatomical and pathological information not only of the teeth but of the surrounding connective tissue including the mandibular and the paranasal and maxillary sinuses. The main limitation of dental PMCT is the occurrence of metal artifacts (beam hardening artifacts) from restorative materials with a high radiologic opacity which obscure the edge detail of restorative materials. (e.g. A and B). This study found no evidence that the accuracy of this approach suffers from the presence of CT image artifacts.
PMCT has an established role in disaster victim identification (DVI) [8]. One of the principal reasons for the rapid adoption of PMCT for identification is that data from whole-body PMCT scans can be reformatted and rendered to match almost any AM imaging examination, including dental radiographs [4]. Such individually reformatted PMCT images of the dentition (sometimes referred to as “dental CT” or “dental PMCT”) for comparative identification were introduced to forensic sciences a decade ago [9]. Today, radiologic identification through comparison of dental PMCT with AM dental radiographs is routinely used for personal identification by various institutions (including those of several of the authors). Principal advantages of dental PMCT over normal post mortem dental radiographs are: (1) three-dimensional image reformatting (e.g. dental PMCT images may be matched to any type of AM dental radiographs, including panorex/orthopantograms and bite-wing radiographs); (2) PMCT images are acquired in an non-invasive fashion (e.g. no need to incise the masseter muscles or extract the mandible for adequately positioned PM dental radiograph); (3) PMCT provides detailed anatomical and pathological information not only of the teeth but of the surrounding connective tissue including the mandibular and the paranasal and maxillary sinuses (which may contribute to identification) [5,9–13]. There is further consensus in the literature that the main
limitation of dental PMCT is the occurrence of metal artifacts (beam hardening artifacts) from restorative materials with a high radiologic opacity, such as gold or amalgam [5,9–11,13,14]. These metal artifacts obscure the edge detail of restorative materials. Several authors speculated that this type of artifact might potentially affect the reliability of comparative radiologic identification based on dental PCMT and AM dental radiographs, although no evidence has been published to confirm this hypothesis [5,9– 11,14]. Nevertheless, several methods were developed to reduce streak artifact [13–15] and overall, most authors are confident that PMCT is a useful and reliable tool for dental identification both in single identification cases and mass fatality events [3,5,10,12]. The hypothesis is that comparing dental PMCT to AM dental radiographs is (at least) as valid for identification as the traditional method of comparing PM dental radiographs to AM dental radiographs [16–19]. The accuracy of comparative identification using dental PMCT and AM dental radiographs has not yet been directly evaluated. The objective of this study was to test the accuracy and interreader variability of comparative radiologic identification based on dental PMCT and AM dental radiographs in a fictitious mass fatality scenario using multiple readers with varying degrees of expertize and experience.
T.D. Ruder et al. / Journal of Forensic Radiology and Imaging 5 (2016) 25–30
2. Materials and methods 2.1. Study population The department of the public prosecutor approved the study. The study supervisor (who did not act as reader) reviewed the institutional archive for all forensic reports where identification of individuals was performed through comparison of dental PMCT and AM dental bite-wing radiographs between January 2007 and December 2010. Each of these reports featured a side-by-side comparison of AM bitewing radiographs and the corresponding reformatted volume rendering images from dental PMCT. From these reports, the study supervisor extracted 115 individual dental PMCT images and 114 AM bite-wing radiographs consisting of 98 pairs of matching PMCT and AM radiographs (Fig. 1), 17 extra PMCT images without matching AM radiographs, and 16 extra AM radiographs without matching PMCT images. Each image was selected at the discretion of the study supervisor with the intention to create a realistic experimental set of images, representing a simulated maritime mass disaster scenario in which the post mortem data of 115 unidentified decedents had to be compared to the ante mortem data of 114 missing individuals. The 17 extra PMCT images without matching AM radiographs corresponded to 17 recovered bodies for which AM data was not yet available and the 16 extra AM radiographs without matching PMCT image corresponded to 16 missing individuals whose remains had not yet been recovered from the water. All 229 images were individually printed on photographic paper measuring 60 95 mm. Each image was anonymized and consisted of either a PMCT image or an AM radiograph of one individual dental quadrant. Each radiograph was labeled with an individual, fictitious first name; each PMCT was labeled with a number from one to 115. There was no further information on the printouts (including specifications of left or right and upper or lower jaw).
27
identification; rater 5 (MSt) was a fifth year medical student, with no experience in either forensic radiology or PMCT and dental identification. Each rater was asked to compare all 115 PMCT images individually to the 114 AM radiographs and classify the images either as match (out of 98 cases with matching AM radiographs and PMCT) or exclusions (out of 33 exclusions, 17 cases with PMCT but without corresponding AM radiographs and 16 cases with AM radiographs but without corresponding PMCT). In a total, there were 131 possible correct classifications (98 þ17 þ 16). Raters were unaware of the total number of possible matches, number of PMCT images without corresponding AM radiographs, number of AM radiographs without corresponding PMCT, and were blinded to any personal data or case related data of the subjects enrolled in the study. Raters indicated their level of confidence for each decision using the categories of the Interpol Disaster Victim Identification Guide: (1) positive identification, (2) probable identification, (3) possible identification, and (4) exclusion. In addition, raters counted concordant features in every matched PMCT image/AM radiograph-pair. Concordance was determined by each rater individually based his or her judgement of a visual comparison of the following predefined features on AM radiographs and PMCT images: (1) concordant number of teeth (one point); (2) concordant number of teeth with dental restorations (one point); (3) concordant position and shape of restorations (one point per concordant restoration); (4) and concordant relative position of teeth (one point). The total sum of these features was used to objectively categorize each match as: moderate match (1-3 concordant features); fair match (4–6 concordant features); excellent match ( 47 concordant features). It is noted that previous work has shown that there is no basis for defining a minimum number of concordant points for dental identifications [20]. Moreover, population frequencies have been argued to be much more important than the number of concordant points in forensic identification comparisons [21]. Nonetheless, here we evaluate these concepts in a DVI scenario.
2.2. Imaging protocol 2.4. Statistical analysis Dental PMCT was performed using a helical, multi-detector CT scanner (Somatom Emotion 6, Siemens Healthcare, Erlangen, Germany) with typical raw data acquisition at 110 kV, 160 mAs, and 6 mm 1 mm collimation. CT image reconstruction was performed with a slice thickness of 1.25 mm, in increments of 0.7 mm, using soft tissue and bone-weighted tissue kernels. Individual volume rendered images were reconstructed from soft tissue kernel reconstructions using a preset volume rendering algorithm (Classic Metal 1) on a dedicated work station (InSpace, Leonardo, Siemens Healthcare, Erlangen, Germany). AM dental radiographs had been performed on several different x-ray machines by different manufacturers, using different protocols, each according to the individual dentist's specification. 2.3. Radiologic identification Five raters with varying degrees of expertize and experience were asked to compare the PMCT images to the AM radiographs. Rater 1 (FRx) was a radiologist with three years of experience in forensic radiology including PMCT and dental identification; rater 2 (FPa) was a forensic pathologist with three years of experience in forensic radiology including PMCT and dental identification; rater 3 (FOd) was a forensic pathologist and odontologist with eight years of experience in forensic odontology including dental identification (but little experience with PMCT); rater 4 (CRx) was a radiologist with three years of experience in clinical radiology but little experience with PMCT and no experience in dental
Data analysis was performed with IBMs SPSSs Statistics (Version 20, release 20.0.0, 2011, IMB Corp., Armonk, NY, USA). Shapiro-Wilk Test was used to test normality of distribution. Nonparametric Levene-Test was used to test for homoscedasticity of variables. Interrater correlation coefficient (ICC) was used to assess interrater agreement regarding levels of confidence (positive identification, probable identification, possible identification, and exclusion) as well as interrater agreement regarding number of concordant radiologic features. ICC o0.40 indicates poor reproducibility, ICC values in the range 0.40–0.75 indicate fair to good reproducibility, and an ICC value of greater than 0.75 shows excellent reproducibility [22]. Games-Howell Test was used to assess significance levels between level of confidence of matched pairs and the number of concordant radiologic features. Mann– Whitney U-Test was used to assess significance levels between number of concordant radiologic features and accuracy of matched pairs. Fischer's Exact Test was used to determine the correlation between the level of confidence and accuracy of a match/exclusion. A p-value of o0.05 indicates statistical significance.
3. Results Mean number of correctly matched pairs of PMCT and AM radiographs for all five raters was 90.2/98 (92%, range: 78–95 matches). Mean number of correctly excluded extra PMCT images
28
T.D. Ruder et al. / Journal of Forensic Radiology and Imaging 5 (2016) 25–30
Table 1 Detailed analysis of rater performance. Rater
MSt FRx FPa CRx FOd Mean Summ
Positive ID
Probable ID
Possible ID
Matched pairs (total)
Exclusion
True
False
True
False
True
False
True
%
False
%
True
%
False
%
65 72 30 52 53 54.4 272
0 0 0 4 0 0.8 4
16 15 32 20 33 23.2 116
3 1 0 0 1 1 5
9 7 33 6 8 12.6 63
4 2 0 7 8 4.2 21
90 94 95 78 94 90.2 451
91.8 95.9 96.9 79.6 95.9 92.0
7 3 0 11 9 6 30
7.1 3.1 0.0 11.2 9.2 6.1
24 28 32 28 20 26.4 132
72.7 84.8 97.0 84.8 60.6 80.0
10 6 4 14 8 8.4 42
30.3 18.2 12.1 42.4 24.2 25.5
Table 2 Sensitivity, specificity, positive predictive value, and negative predictive value for each rater. Rater
Sensitivity
Specificity
PPV
NPV
MSt FRx FPa CRx FOd Mean
90.0 94.0 96.0 84.8 92.2 91.5
77.4 90.3 100.0 71.8 69.0 81.5
92.8 96.6 100.0 87.6 91.3 93.8
70.6 82.4 88.9 66.7 71.4 75.9
or AM radiographs with no corresponding AM radiograph or PMCT image was 24/33 (80%, range: 20–32 exclusions). This corresponds to a mean sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of 91.5% (range: 84.8–98.0%), 81.5% (range: 69.0–100%), 93.6% (range: 87.6–100%) and 75.9% (range: 66.7–88.9%) respectively. Tables 1 and 2 provide the results for all five raters individually. Altogether, the five raters matched a total of 481 pairs of PMCT and AM radiographs. Of these, 30 matches were false. Of these, four were rated as “positive identification”, five as “probable identification” and 21 as “possible identification”. In addition, in 24 cases of these 30 cases, the false match was made by only one (but not always the same) of the five raters. In two cases, the same false pairs were matched by two different raters and in only one case the same false pair was matched by three different raters. In other words: in all cases except one, every false positive match conflicted with the judgement of at least three raters. Interrater agreement regarding levels of confidence (positive identification, probable identification, possible identification, and exclusion) was fair to good (ICC 0.623). Interrater agreement regarding the number of concordant radiologic features was excellent (ICC 0.907). Games-Howell test revealed that matches with a high level of subjective confidence were significantly associated with higher numbers of concordant radiologic features whereas matches with a low level of subjective confidence were significantly associated with lower numbers of concordant radiologic features (Fig. 2). Mann–Whitney U-Test revealed that the number of concordant radiologic features was not significantly correlated to the accuracy of a match. In addition, Fischer's exact test revealed that the level of confidence was also not correlated to the accuracy of a match/ exclusion (r ¼0.333). In other words: raters agree on both the level of confidence and the number of concordant features of matched pairs, but neither the level of confidence nor the number of concordant features are statistically linked to the accuracy of a match/ exclusion.
Fig. 2. Box-plot graph visualizing the relationship between levels of confidence for matched PMCT/AM radiographs (positive ID, probable ID, and possible ID) and the number of concordant radiologic features. Bottom and top of boxes represent the 25th and 75th percentile, respectively; black band in the boxes represents median, whiskers indicate minimum and maximum of all data. Games-Howell test revealed that matches with a high level of subjective confidence were significantly associated with higher numbers of concordant radiologic features whereas matches with a low level of subjective confidence were significantly associated with lower numbers of concordant radiologic features.
4. Discussion This study shows that visual comparison of PMCT images and AM dental radiographs is a reliable method for identification with an excellent accuracy, if performed by an experienced rater. In this study, positive identification of matching pairs of PMCT and AM dental radiographs showed an accuracy of 92% (90.2/98) across all raters. This result stands in agreement with published data on accuracy of identification through visual comparison of plain films of PM and AM dental radiographs which ranges from 85.5% to 93% [16–18]. This confirms the general hypothesis that comparison of PMCT images to AM dental radiographs is as reliable for identification as comparison of PM dental radiographs to AM dental radiographs [9,10,14,15]. The results of this study indicate that experience is essential for successful identification. This finding is in accordance with previous studies on comparative identification using dental radiographs [17,18]. Scholl et al. revealed that rater experience is more important than formal training: in their study trained forensic odontologists who had no practical experience in forensic identification performed worse than dentists without formal training in forensic odontology but with practical experience in identification (91% correct matches vs. 100% correct matches) [17]. Pretty et al. investigated this aspect further and found that raters with substantial practical experience (i.e. performed more than 5 forensic
T.D. Ruder et al. / Journal of Forensic Radiology and Imaging 5 (2016) 25–30
identifications) achieved higher numbers of correct matches (91%) than raters with moderate practical experience (i.e. performed between 1 and 5 forensic identification), raters with no practical experience, and untrained raters (81%, 86%, and 84% respectively) [18]. The results of our study stand in agreement with these findings; all three raters with substantial experience, as defined by Pretty et al. (forensic odontologist, forensic pathologist, and forensic radiologist) achieved higher numbers of correct matches (95.9%, 96.9%, and 95.9%, respectively) than the two raters with no practical experience in forensic identification (medical student 91.8%, clinical radiologist 79.6%). In addition, the inexperienced raters made more false positive matches (medical student 7.1%, and clinical radiologist 11.2%) than two of the experienced raters (forensic pathologist: 0% and forensic radiologist: 3.1%). Unexpectedly, the forensic odontologist made as many false positive identifications as the inexperienced raters (9.2%). Although the number of false positive matches may appear to be relatively high (30 false positive matches vs. 451 correct matches) one has to keep in mind that the vast majority of these (21/30) were rated as “possible identification”. In a real DVI scenario, these cases would certainly have been further evaluated. Nevertheless, this finding seems to provide insight into another aspect of experience: experience with the imaging modality used for identification. The forensic odontologist had gained extensive practical experience in identification during his deployment to Thailand after the 2004 Tsunami, but had never worked with dental PMCT images. His inexperience with dental PMCT may be an explanation for his high rate of false positive matches. A similar observation was also made by Wenzel et al. [19]. In their study, raters who were inexperienced in both dental radiography and inexperienced in forensic identification made more false positive matches than raters with experience in the imaging modality and the process of identification [19]. Overall, the outcome of this and previous work indicates that practical experience in forensic identification is crucial to achieve high numbers of positive matches whereas experience with the imaging modality (in this case: dental PMCT) affects the number of false positive matches. It was interesting to observe that the level of confidence was not correlated to the accuracy of a match or the ability to correctly exclude unmatched PMCT images or dental radiographs. Today, both Interpol and the ABFO (American Board of Forensic Odontologists) use four slightly different levels of confidence to rate a pair of matching dental images (Interpol: positive match, probable match, possible match, exclusion; ABFO: positive match, possible match, insufficient evidence, exclusion) [2,23]. Although Interpol provides an explanation on how to distinguish between these subcategories, raters in our study found the differentiation between subcategories that are neither a certain match nor a certain exclusion difficult. This difficulty is reflected in the finding that levels of confidence did not have a significant impact on the accuracy of a match in this study. It is the opinion of these authors that a simplified classification system (positive match, uncertain, exclusion) would be more practical for routine identification cases (since any subcategory that is neither a certain match nor a certain exclusion invariably demands a search for additional AM or PM data) and more scientific for research purposes. This approach was also chosen by Wenzel et al. in their study and proved to be very successful [19]. Also interesting was the lack of significant correlation between the number of concordant radiologic features and the accuracy of a match. This is likely due in part to the fact that “concordance” (which essentially means that the rater believes two areas or features look the same) is itself subjective, and moreover that an association between the number of concordant points and a correct identification has never been shown. In forensic identifications, however, the population frequency of the trait is more
29
important than the number of shared features. Numerous concordant features, if many people share them, are of little use for making a conclusion regarding identification. Conversely, even one feature or configuration can be enough for identification, provided that the configuration is sufficiently rare. The study has several limitations, which deserve discussion: First, the use of printed images instead of digital images may be criticized. The authors concur that tools to magnify images or change image contrast might improve outcomes. However, the use of print-outs proved to be more practical for raters to display several different images side-by-side during the process of matching/exclusion. Second: it may be questioned that dental PMCT data was limited to single VRT images. The authors agree that for individual identification, the use of a whole PMCT dataset is a prerequisite since the PMCT images can (and must) be individually adapted to the AM bite-wing or panorex images. In this scenario, we used PMCT images that had already been individually reconstructed and adapted to the AM radiographs when they had been used for real forensic identification. It was part of the design of the study to only use images from cases which had been positively identified using PMCT images and dental radiographs. Moreover, the use of printed single VRT images in this study, would likely underestimate the rate of correct identifications. Therefore it is quite possible, that the use of original PMCT data would have increased the accuracy of identification in ambiguous cases. Finally, the fact that raters were asked to first indicate their level of confidence regarding a match/exclusion using the classification system of Interpol and second assess the number of concordant radiologic features on the PMCT image and the dental radiograph may have introduced a confirmation bias [24]. Confirmation bias is a cognitive bias “by which people tend to seek, perceive, interpret, and create new evidence in ways that verify preexisting beliefs” [24]. A number of publications have revealed how confirmation bias can have a significant impact on several forensic disciplines including fingerprint analysis, anthropology, bullet analysis, bite mark analysis, and even interpretation of DNA evidence [24–29]. Confirmation bias is especially frequent in situations where experts have to provide a subjective opinion (like indicating a level of confidence regarding a match in identification). It is important to state that this study was not designed to search for confirmation bias in dental identification, but it is certainly worth considering its occurrence in this study. The correlation found between the level of confidence and the number of concordant features of matched pairs suggests that the more confident raters are about the accuracy of an identification, the more concordant points will they believe to see (and vice versa). It is beyond the scope of this manuscript (and also beyond the design of the study) to discuss this effect in detail, but it is safe to say that single-rater identifications are susceptible to subjective bias. A very simple way to reduce confirmation bias is to involve multiple independent raters for identifications and reevaluate every case of rater disagreement with a third, independent rater or in a consensus reading. This approach would certainly have significantly reduced the number of false positive identifications in our study, since 80% (24/30) false matches were made by one of all five raters only. It is certainly no coincidence that this conclusion stands in agreement with the Interpol DVI guide which heavily emphasizes the importance of (interdisciplinary) team work in disaster victim identification.
5. Conclusions This study shows that visual comparison of PMCT images with AM dental radiographs is a reliable method for identification. Accuracy of identification using PMCT/AM dental radiographs was as
30
T.D. Ruder et al. / Journal of Forensic Radiology and Imaging 5 (2016) 25–30
high in this study as in comparable studies using PM dental radiographs/AM dental radiographs. There is no evidence that the accuracy of this approach suffers from the presence of CT image artifacts. These results provide evidence that practical experience in forensic identification is crucial to accurately assign positive matches, whereas experience with the imaging modality (in this case: dental PMCT) is inversely related to the number of false positive matches. Match accuracy did not correlate with subjective confidence or number of concordant points. It is advised to work in teams rather than individually when dealing with real cases in forensic identification, to minimize subjective interpretation and avoid confirmation bias.
Conflict of interest The authors have no conflict of interest to report.
References [1] P. Saukko, B. Knight, The forensic autopsy, in: P. Saukko, B. Knight (Eds.), Knight's Forensic Pathology, Edward Arnold Ltd., London, 2004, pp. 1–52. [2] Interpol. Disaster Victim Identification Guide (DVI Guide) (version 2014). Available from: 〈http://www.interpol.int/INTERPOL-expertise/Forensics/DVI-Pa ges/DVI-guide〉. [3] G.M. Hatch, F. Dedouit, A.M. Christensen, M.J. Thali, T.D. Ruder, RADid: a pictorial review of radiologic identification using post mortem CT, J. Forensic Radiol. Imag. 2 (2014) 52–59. [4] M. Baglivo, S. Winklhofer, G.M. Hatch, G. Ampanozi, M.J. Thali, T.D. Ruder, The rise of forensic and post mortem radiology – analysis of the literature between the year 2000 and 2011, J. Forensic Radiol. Imag. 1 (2013) 3–9. [5] M. Sidler, C. Jackowski, R. Dirnhofer, P. Vock, M. Thali, Use of multislice computed tomography in disaster victim identification—advantages and limitations, Forensic Sci. Int. 169 (2/3) (2006) 118–128. [6] C. O'Donnell, M. Iino, K. Mansharan, J. Leditscke, N. Woodford, Contribution of post mortem multidetector CT scanning to identification of the deceased in a mass disaster: experience gained from the 2009 Victorian bushfires, Forensic Sci. Int. 205 (2011) 15–28. [7] T.D. Ruder, M. Kraehenbuehl, W.F. Gotsmy, S. Mathier, L.C. Ebert, M.J. Thali, G. M. Hatch, Radiologic identification of disaster victims: a simple and reliable method using CT of the paranasal sinuses, Eur. J. Radiol. 81 (2012) e132–e138. [8] B. Morgan, A. Alminyah, A. Cala, C. O'Donnell, D. Elliott, G. Gorincour, P. Hofman, M. Iino, Y. Makino, A. Moskata, C. Robinson, G.N. Rutty, A. Sajantila, J. Vallis, N. Woodford, K. Woźniak, M. Viner, Use of post mortem computed tomography in Disaster Victim Identification-Positional statement of the members of the Disaster Victim Identification working group of the International Society of Forensic Radiology and Imaging, J. Forensic Radiol. Imag. 2 (2014) 114–116. [9] M.J. Thali, T. Markwalder, C. Jackowski, M. Sonnenschein, R. Dirnhofer, Dental CT imaging as a screening tool for dental profiling: advantages and limitations, J. Forensic Sci. 51 (2006) 113–119.
[10] C. Jackowski, E. Aghayev, M. Sonnenschein, R. Dirnhofer, M.J. Thali, Maximum intensity projection of cranial computed tomography data for dental identification, Int. J. Legal Med. 120 (2006) 165–167. [11] S. Kirchhoff, F. Fischer, G. Lindemaier, P. Herzog, C. Kirchhoff, C. Becker, J. Bark, M.F. Reiser, W. Eisenmenger, Is post mortem CT of the dentition adequate for correct forensic identification? comparison of dental computed tomograpy and visual dental record, Int. J. Legal Med. 122 (2008) 471–479. [12] A. Franco, P. Thevissen, W. Coudyzer, W. Develter, W. Van de Voorde, R. Oyen, D. Vandermeulen, R. Jacobs, G. Willems, Feasibility and validation of virtual autopsy for dental identification using the Interpol dental codes, J. Forensic Legal Med. 20 (2013) 248–254. [13] P. Stolzmann, S. Winklhofer, N. Schwendener, H. Alkadhi, M.J. Thali, T.D. Ruder, Monoenergetic computed tomography reconstructions reduce beam hardening artifacts from dental restorations, Forensic Sci. Med. Pathol. 9 (2013) 327–332. [14] C. Jackowski, A. Lussi, M. Classens, T. Kilchoer, S. Bolliger, E. Aghayev, A. Criste, R. Dirnhofer, M.J. Thali, Extended CT scale overcomes restoration caused streak artifacts for dental identification in CT – 3D color encoded automatic discrimination of dental restorations, J. Comput. Assist. Tomogr. 30 (2006) 510–513. [15] C. Jackowski, M. Wyss, A. Persson, M. Classens, M.J. Thali, A. Lussi, Ultra-highresolution dual-source CT for forensic dental visualization-discrimination of ceramic and composite fillings, Int. J. Legal Med. 122 (2008) 301–307. [16] D.F. MacLean, S.L. Kogon, L.W. Stitt, Validation of dental radiographs for human identification, J. Forensic Sci. 39 (1994) 1195–1200. [17] S.A. Sholl, G.H. Moody, Evaluation of dental radiographic identification: an experimental study, Forensic Sci. Int. 115 (2001) 165–169. [18] I.A. Pretty, R.J. Pretty, B.R. Rothwell, D. Sweet, The reliability of digitized radiographs for dental identification: a web-based study, J. Forensic Sci. 48 (2003) 1325–1330. [19] A. Wenzel, A. Richards, J. Heidmann, Matching simulated antemortem and post mortem dental radiographs from human skulls by dental students and experts: testing skills for pattern recognition, J. Forensic Odontostomatol. 28 (2010) 5–12. [20] A.B. Acharya, J.A. Taylor, Are a minimum number of concordant matches needed to establish identity in forensic odontology? J. Forensic Odontostomatol. 21 (1) (2003) 6–13. [21] A.M. Christensen, G.M. Hatch, B.G. Brogdon, A current perspective on forensic radiology, J. Forensic Radiol. Imag. 2 (3) (2014) 111–113. [22] B. Rosner, Fundamentals of Biostatistics, Duxbury Press, Belmont, CA, 2005. [23] American Board of Forensic Odontology, Diplomates Reference Manual (January 2013 edition). Available from: 〈http://www.abfo.org/wp-content/up loads/2012/08/ABFO-Reference-Manual-1-22-2013-revision.pdf〉. [24] S.M. Kassin, I.E. Dror, J. Kukucka, The forensic confirmation bias: problems, perspectives, and proposed solutions, J. Appl. Res. Mem. Cogn. 2 (2013) 42–52. [25] I.E. Dror, D. Charlton, Why experts make errors, J. Forensic Identif. (2006) 56600–56616. [26] S. Nakhaeizadeh, I.E. Dror, R.M. Morgan, Cognitive bias in forensic anthropology: visual assessment of skeletal remains is susceptible to confirmation bias, Sci. Justice 54 (2014) 208–214. [27] J. Kerstholt, A. Eikelboom, T. Dijkman, R. Stoel, R. Hermsen, B. van Leuven, Does suggestive information cause a confirmation bias in bullet comparisons? Forensic Sci Int. 198 (2010) 138–142. [28] M. Page, J. Taylor, M. Blenkin, Context effects and observer bias–implications for forensic odontology, J. Forensic Sci. 57 (2012) 108–112. [29] I.E. Dror, G. Hampikian, Subjectivity and bias in forensic DNA mixture interpretation, Sci. Justice 51 (2011) 204–208.
Contents lists available at ScienceDirect
Journal of Forensic Radiology and Imaging journal homepage: www.elsevier.com/locate/jofri
Validation of post mortem dental CT for disaster victim identification Thomas D. Ruder a,b,c,n, Yannick A. Thali a,d, Saiful N.A. Rashid e,f, Michael T. Mund a,g, Michael J. Thali a, Gary M. Hatch h, Angi M. Christensen i, Sandra Somaini j, Garyfalia Ampanozi a,c a
Institute of Forensic Medicine, Department of Forensic Medicine and Imaging, University of Zurich, CH-8057 Zurich, Switzerland Institute of Diagnostic, Interventional, and Pediatric Radiology, University Hospital Bern, CH-3010 Bern, Switzerland c Center of Forensic Imaging and Virtopsy, Institute of Forensic Medicine, University of Bern, CH-3012 Bern, Switzerland d Institute of Radiology, Cantonal Hospital Lucerne, CH-6000 Lucerne, Switzerland e Department of Imaging, Faculty of Medicine & Health Sciences, Universiti Putra Malaysia, Malaysia f National Institute of Forensic Medicine, Hospital Kuala Lumpur, Malaysia g JDMT Medical Services AG, CH-8330 Pfäffikon, Switzerland h Radiology-Pathology Center for Forensic Imaging, Departments of Radiology and Pathology, University of New Mexico School of Medicine, Albuquerque, NM 87102, USA i Federal Bureau of Investigation Laboratory, Quantico, VA 22135, USA j Institute of Diagnostic and Interventional Neuroradiology, University Hospital Bern, CH-3010 Bern, Switzerland b
art ic l e i nf o
a b s t r a c t
Article history: Received 27 November 2015 Accepted 25 January 2016 Available online 31 January 2016
The objective of this study was to test the accuracy and inter-reader variability of comparative radiologic identification based on dental post mortem computed tomography (PMCT) and ante mortem (AM) dental radiographs. Five raters with varying degrees of expertize and experience independently compared 115 dental PMCT images to 114 AM dental radiographs to identify matching pairs (n ¼98), unmatched PMCT images (n ¼17), and unmatched AM radiographs (n ¼ 16). Levels of confidence (LOC) and number of concordant features (NOCF) of matched pairs were documented. Accuracy of matches/exclusions, interrater correlation coefficient and correlation between correct matches/exclusions, LOC and NOCF were calculated for all raters. Mean accuracy was 92% for matches and 80% for exclusions. Interrater correlation coefficient regarding LOC and NOCF were 0.623 and 0.907 respectively. LOC were correlated with NOCF of matched pairs but accuracy of matches/exclusions was neither correlated to LOC nor to NOCF. This study shows that visual comparison of PMCT images with AM dental radiographs is a reliable method for identification. Accuracy of identification using PMCT/AM dental radiographs was as high as in comparable studies using post mortem (PM) dental radiographs/AM dental radiographs. Raters with practical experience in forensic identification and experience with the imaging modality (in this case: dental PMCT) achieved higher accuracy than inexperienced raters. Match accuracy did not correlate with subjective confidence or number of concordant features. It is advised to work in teams rather than individually when dealing with real cases in forensic identification, to minimize subjective interpretation and avoid confirmation bias. & 2016 Elsevier Ltd. All rights reserved.
Keywords: Forensic radiology Forensic identification Dental identification Post mortem CT Post mortem dental-CT
1. Introduction Identification of the dead is a fundamental part of forensic investigation [1]. Individual identification relies on comparison of
n Correspondence to: Department of Forensic Medicine and Imaging, Institute of Forensic Medicine, University of Zurich, Winterthurerstrasse 190/52, CH-8057 Zurich, Switzerland. E-mail addresses: [email protected], [email protected] (T.D. Ruder).
http://dx.doi.org/10.1016/j.jofri.2016.01.006 2212-4780/& 2016 Elsevier Ltd. All rights reserved.
ante mortem (AM) and post mortem (PM) data, typically fingerprints, DNA samples, or dental records [2]. Interpol offers general guidelines for victim identification in mass disasters [2]. However there are no internationally recognized guidelines in isolated cases and the procedure differs as a function of local conventions, availability of experts, and the scenario [3]. In the past 15 years, the use of post mortem computed tomography (PMCT) in forensic death investigations increased [4] and its potential for disaster victim identification was put to the test both in actual and simulated mass disaster scenarios [5–7]. Today,
26
T.D. Ruder et al. / Journal of Forensic Radiology and Imaging 5 (2016) 25–30
Fig. 1. Four sample images of corresponding ante mortem radiographs and reconstructed post mortem computed tomography (PMCT) of one quadrant. A–C feature cases where restorative materials are present. D features a case without restorative materials. PMCT has several advantages over plain film dental images, including threedimensional image reformatting, acquisition in a non-invasive fashion, and anatomical and pathological information not only of the teeth but of the surrounding connective tissue including the mandibular and the paranasal and maxillary sinuses. The main limitation of dental PMCT is the occurrence of metal artifacts (beam hardening artifacts) from restorative materials with a high radiologic opacity which obscure the edge detail of restorative materials. (e.g. A and B). This study found no evidence that the accuracy of this approach suffers from the presence of CT image artifacts.
PMCT has an established role in disaster victim identification (DVI) [8]. One of the principal reasons for the rapid adoption of PMCT for identification is that data from whole-body PMCT scans can be reformatted and rendered to match almost any AM imaging examination, including dental radiographs [4]. Such individually reformatted PMCT images of the dentition (sometimes referred to as “dental CT” or “dental PMCT”) for comparative identification were introduced to forensic sciences a decade ago [9]. Today, radiologic identification through comparison of dental PMCT with AM dental radiographs is routinely used for personal identification by various institutions (including those of several of the authors). Principal advantages of dental PMCT over normal post mortem dental radiographs are: (1) three-dimensional image reformatting (e.g. dental PMCT images may be matched to any type of AM dental radiographs, including panorex/orthopantograms and bite-wing radiographs); (2) PMCT images are acquired in an non-invasive fashion (e.g. no need to incise the masseter muscles or extract the mandible for adequately positioned PM dental radiograph); (3) PMCT provides detailed anatomical and pathological information not only of the teeth but of the surrounding connective tissue including the mandibular and the paranasal and maxillary sinuses (which may contribute to identification) [5,9–13]. There is further consensus in the literature that the main
limitation of dental PMCT is the occurrence of metal artifacts (beam hardening artifacts) from restorative materials with a high radiologic opacity, such as gold or amalgam [5,9–11,13,14]. These metal artifacts obscure the edge detail of restorative materials. Several authors speculated that this type of artifact might potentially affect the reliability of comparative radiologic identification based on dental PCMT and AM dental radiographs, although no evidence has been published to confirm this hypothesis [5,9– 11,14]. Nevertheless, several methods were developed to reduce streak artifact [13–15] and overall, most authors are confident that PMCT is a useful and reliable tool for dental identification both in single identification cases and mass fatality events [3,5,10,12]. The hypothesis is that comparing dental PMCT to AM dental radiographs is (at least) as valid for identification as the traditional method of comparing PM dental radiographs to AM dental radiographs [16–19]. The accuracy of comparative identification using dental PMCT and AM dental radiographs has not yet been directly evaluated. The objective of this study was to test the accuracy and interreader variability of comparative radiologic identification based on dental PMCT and AM dental radiographs in a fictitious mass fatality scenario using multiple readers with varying degrees of expertize and experience.
T.D. Ruder et al. / Journal of Forensic Radiology and Imaging 5 (2016) 25–30
2. Materials and methods 2.1. Study population The department of the public prosecutor approved the study. The study supervisor (who did not act as reader) reviewed the institutional archive for all forensic reports where identification of individuals was performed through comparison of dental PMCT and AM dental bite-wing radiographs between January 2007 and December 2010. Each of these reports featured a side-by-side comparison of AM bitewing radiographs and the corresponding reformatted volume rendering images from dental PMCT. From these reports, the study supervisor extracted 115 individual dental PMCT images and 114 AM bite-wing radiographs consisting of 98 pairs of matching PMCT and AM radiographs (Fig. 1), 17 extra PMCT images without matching AM radiographs, and 16 extra AM radiographs without matching PMCT images. Each image was selected at the discretion of the study supervisor with the intention to create a realistic experimental set of images, representing a simulated maritime mass disaster scenario in which the post mortem data of 115 unidentified decedents had to be compared to the ante mortem data of 114 missing individuals. The 17 extra PMCT images without matching AM radiographs corresponded to 17 recovered bodies for which AM data was not yet available and the 16 extra AM radiographs without matching PMCT image corresponded to 16 missing individuals whose remains had not yet been recovered from the water. All 229 images were individually printed on photographic paper measuring 60 95 mm. Each image was anonymized and consisted of either a PMCT image or an AM radiograph of one individual dental quadrant. Each radiograph was labeled with an individual, fictitious first name; each PMCT was labeled with a number from one to 115. There was no further information on the printouts (including specifications of left or right and upper or lower jaw).
27
identification; rater 5 (MSt) was a fifth year medical student, with no experience in either forensic radiology or PMCT and dental identification. Each rater was asked to compare all 115 PMCT images individually to the 114 AM radiographs and classify the images either as match (out of 98 cases with matching AM radiographs and PMCT) or exclusions (out of 33 exclusions, 17 cases with PMCT but without corresponding AM radiographs and 16 cases with AM radiographs but without corresponding PMCT). In a total, there were 131 possible correct classifications (98 þ17 þ 16). Raters were unaware of the total number of possible matches, number of PMCT images without corresponding AM radiographs, number of AM radiographs without corresponding PMCT, and were blinded to any personal data or case related data of the subjects enrolled in the study. Raters indicated their level of confidence for each decision using the categories of the Interpol Disaster Victim Identification Guide: (1) positive identification, (2) probable identification, (3) possible identification, and (4) exclusion. In addition, raters counted concordant features in every matched PMCT image/AM radiograph-pair. Concordance was determined by each rater individually based his or her judgement of a visual comparison of the following predefined features on AM radiographs and PMCT images: (1) concordant number of teeth (one point); (2) concordant number of teeth with dental restorations (one point); (3) concordant position and shape of restorations (one point per concordant restoration); (4) and concordant relative position of teeth (one point). The total sum of these features was used to objectively categorize each match as: moderate match (1-3 concordant features); fair match (4–6 concordant features); excellent match ( 47 concordant features). It is noted that previous work has shown that there is no basis for defining a minimum number of concordant points for dental identifications [20]. Moreover, population frequencies have been argued to be much more important than the number of concordant points in forensic identification comparisons [21]. Nonetheless, here we evaluate these concepts in a DVI scenario.
2.2. Imaging protocol 2.4. Statistical analysis Dental PMCT was performed using a helical, multi-detector CT scanner (Somatom Emotion 6, Siemens Healthcare, Erlangen, Germany) with typical raw data acquisition at 110 kV, 160 mAs, and 6 mm 1 mm collimation. CT image reconstruction was performed with a slice thickness of 1.25 mm, in increments of 0.7 mm, using soft tissue and bone-weighted tissue kernels. Individual volume rendered images were reconstructed from soft tissue kernel reconstructions using a preset volume rendering algorithm (Classic Metal 1) on a dedicated work station (InSpace, Leonardo, Siemens Healthcare, Erlangen, Germany). AM dental radiographs had been performed on several different x-ray machines by different manufacturers, using different protocols, each according to the individual dentist's specification. 2.3. Radiologic identification Five raters with varying degrees of expertize and experience were asked to compare the PMCT images to the AM radiographs. Rater 1 (FRx) was a radiologist with three years of experience in forensic radiology including PMCT and dental identification; rater 2 (FPa) was a forensic pathologist with three years of experience in forensic radiology including PMCT and dental identification; rater 3 (FOd) was a forensic pathologist and odontologist with eight years of experience in forensic odontology including dental identification (but little experience with PMCT); rater 4 (CRx) was a radiologist with three years of experience in clinical radiology but little experience with PMCT and no experience in dental
Data analysis was performed with IBMs SPSSs Statistics (Version 20, release 20.0.0, 2011, IMB Corp., Armonk, NY, USA). Shapiro-Wilk Test was used to test normality of distribution. Nonparametric Levene-Test was used to test for homoscedasticity of variables. Interrater correlation coefficient (ICC) was used to assess interrater agreement regarding levels of confidence (positive identification, probable identification, possible identification, and exclusion) as well as interrater agreement regarding number of concordant radiologic features. ICC o0.40 indicates poor reproducibility, ICC values in the range 0.40–0.75 indicate fair to good reproducibility, and an ICC value of greater than 0.75 shows excellent reproducibility [22]. Games-Howell Test was used to assess significance levels between level of confidence of matched pairs and the number of concordant radiologic features. Mann– Whitney U-Test was used to assess significance levels between number of concordant radiologic features and accuracy of matched pairs. Fischer's Exact Test was used to determine the correlation between the level of confidence and accuracy of a match/exclusion. A p-value of o0.05 indicates statistical significance.
3. Results Mean number of correctly matched pairs of PMCT and AM radiographs for all five raters was 90.2/98 (92%, range: 78–95 matches). Mean number of correctly excluded extra PMCT images
28
T.D. Ruder et al. / Journal of Forensic Radiology and Imaging 5 (2016) 25–30
Table 1 Detailed analysis of rater performance. Rater
MSt FRx FPa CRx FOd Mean Summ
Positive ID
Probable ID
Possible ID
Matched pairs (total)
Exclusion
True
False
True
False
True
False
True
%
False
%
True
%
False
%
65 72 30 52 53 54.4 272
0 0 0 4 0 0.8 4
16 15 32 20 33 23.2 116
3 1 0 0 1 1 5
9 7 33 6 8 12.6 63
4 2 0 7 8 4.2 21
90 94 95 78 94 90.2 451
91.8 95.9 96.9 79.6 95.9 92.0
7 3 0 11 9 6 30
7.1 3.1 0.0 11.2 9.2 6.1
24 28 32 28 20 26.4 132
72.7 84.8 97.0 84.8 60.6 80.0
10 6 4 14 8 8.4 42
30.3 18.2 12.1 42.4 24.2 25.5
Table 2 Sensitivity, specificity, positive predictive value, and negative predictive value for each rater. Rater
Sensitivity
Specificity
PPV
NPV
MSt FRx FPa CRx FOd Mean
90.0 94.0 96.0 84.8 92.2 91.5
77.4 90.3 100.0 71.8 69.0 81.5
92.8 96.6 100.0 87.6 91.3 93.8
70.6 82.4 88.9 66.7 71.4 75.9
or AM radiographs with no corresponding AM radiograph or PMCT image was 24/33 (80%, range: 20–32 exclusions). This corresponds to a mean sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of 91.5% (range: 84.8–98.0%), 81.5% (range: 69.0–100%), 93.6% (range: 87.6–100%) and 75.9% (range: 66.7–88.9%) respectively. Tables 1 and 2 provide the results for all five raters individually. Altogether, the five raters matched a total of 481 pairs of PMCT and AM radiographs. Of these, 30 matches were false. Of these, four were rated as “positive identification”, five as “probable identification” and 21 as “possible identification”. In addition, in 24 cases of these 30 cases, the false match was made by only one (but not always the same) of the five raters. In two cases, the same false pairs were matched by two different raters and in only one case the same false pair was matched by three different raters. In other words: in all cases except one, every false positive match conflicted with the judgement of at least three raters. Interrater agreement regarding levels of confidence (positive identification, probable identification, possible identification, and exclusion) was fair to good (ICC 0.623). Interrater agreement regarding the number of concordant radiologic features was excellent (ICC 0.907). Games-Howell test revealed that matches with a high level of subjective confidence were significantly associated with higher numbers of concordant radiologic features whereas matches with a low level of subjective confidence were significantly associated with lower numbers of concordant radiologic features (Fig. 2). Mann–Whitney U-Test revealed that the number of concordant radiologic features was not significantly correlated to the accuracy of a match. In addition, Fischer's exact test revealed that the level of confidence was also not correlated to the accuracy of a match/ exclusion (r ¼0.333). In other words: raters agree on both the level of confidence and the number of concordant features of matched pairs, but neither the level of confidence nor the number of concordant features are statistically linked to the accuracy of a match/ exclusion.
Fig. 2. Box-plot graph visualizing the relationship between levels of confidence for matched PMCT/AM radiographs (positive ID, probable ID, and possible ID) and the number of concordant radiologic features. Bottom and top of boxes represent the 25th and 75th percentile, respectively; black band in the boxes represents median, whiskers indicate minimum and maximum of all data. Games-Howell test revealed that matches with a high level of subjective confidence were significantly associated with higher numbers of concordant radiologic features whereas matches with a low level of subjective confidence were significantly associated with lower numbers of concordant radiologic features.
4. Discussion This study shows that visual comparison of PMCT images and AM dental radiographs is a reliable method for identification with an excellent accuracy, if performed by an experienced rater. In this study, positive identification of matching pairs of PMCT and AM dental radiographs showed an accuracy of 92% (90.2/98) across all raters. This result stands in agreement with published data on accuracy of identification through visual comparison of plain films of PM and AM dental radiographs which ranges from 85.5% to 93% [16–18]. This confirms the general hypothesis that comparison of PMCT images to AM dental radiographs is as reliable for identification as comparison of PM dental radiographs to AM dental radiographs [9,10,14,15]. The results of this study indicate that experience is essential for successful identification. This finding is in accordance with previous studies on comparative identification using dental radiographs [17,18]. Scholl et al. revealed that rater experience is more important than formal training: in their study trained forensic odontologists who had no practical experience in forensic identification performed worse than dentists without formal training in forensic odontology but with practical experience in identification (91% correct matches vs. 100% correct matches) [17]. Pretty et al. investigated this aspect further and found that raters with substantial practical experience (i.e. performed more than 5 forensic
T.D. Ruder et al. / Journal of Forensic Radiology and Imaging 5 (2016) 25–30
identifications) achieved higher numbers of correct matches (91%) than raters with moderate practical experience (i.e. performed between 1 and 5 forensic identification), raters with no practical experience, and untrained raters (81%, 86%, and 84% respectively) [18]. The results of our study stand in agreement with these findings; all three raters with substantial experience, as defined by Pretty et al. (forensic odontologist, forensic pathologist, and forensic radiologist) achieved higher numbers of correct matches (95.9%, 96.9%, and 95.9%, respectively) than the two raters with no practical experience in forensic identification (medical student 91.8%, clinical radiologist 79.6%). In addition, the inexperienced raters made more false positive matches (medical student 7.1%, and clinical radiologist 11.2%) than two of the experienced raters (forensic pathologist: 0% and forensic radiologist: 3.1%). Unexpectedly, the forensic odontologist made as many false positive identifications as the inexperienced raters (9.2%). Although the number of false positive matches may appear to be relatively high (30 false positive matches vs. 451 correct matches) one has to keep in mind that the vast majority of these (21/30) were rated as “possible identification”. In a real DVI scenario, these cases would certainly have been further evaluated. Nevertheless, this finding seems to provide insight into another aspect of experience: experience with the imaging modality used for identification. The forensic odontologist had gained extensive practical experience in identification during his deployment to Thailand after the 2004 Tsunami, but had never worked with dental PMCT images. His inexperience with dental PMCT may be an explanation for his high rate of false positive matches. A similar observation was also made by Wenzel et al. [19]. In their study, raters who were inexperienced in both dental radiography and inexperienced in forensic identification made more false positive matches than raters with experience in the imaging modality and the process of identification [19]. Overall, the outcome of this and previous work indicates that practical experience in forensic identification is crucial to achieve high numbers of positive matches whereas experience with the imaging modality (in this case: dental PMCT) affects the number of false positive matches. It was interesting to observe that the level of confidence was not correlated to the accuracy of a match or the ability to correctly exclude unmatched PMCT images or dental radiographs. Today, both Interpol and the ABFO (American Board of Forensic Odontologists) use four slightly different levels of confidence to rate a pair of matching dental images (Interpol: positive match, probable match, possible match, exclusion; ABFO: positive match, possible match, insufficient evidence, exclusion) [2,23]. Although Interpol provides an explanation on how to distinguish between these subcategories, raters in our study found the differentiation between subcategories that are neither a certain match nor a certain exclusion difficult. This difficulty is reflected in the finding that levels of confidence did not have a significant impact on the accuracy of a match in this study. It is the opinion of these authors that a simplified classification system (positive match, uncertain, exclusion) would be more practical for routine identification cases (since any subcategory that is neither a certain match nor a certain exclusion invariably demands a search for additional AM or PM data) and more scientific for research purposes. This approach was also chosen by Wenzel et al. in their study and proved to be very successful [19]. Also interesting was the lack of significant correlation between the number of concordant radiologic features and the accuracy of a match. This is likely due in part to the fact that “concordance” (which essentially means that the rater believes two areas or features look the same) is itself subjective, and moreover that an association between the number of concordant points and a correct identification has never been shown. In forensic identifications, however, the population frequency of the trait is more
29
important than the number of shared features. Numerous concordant features, if many people share them, are of little use for making a conclusion regarding identification. Conversely, even one feature or configuration can be enough for identification, provided that the configuration is sufficiently rare. The study has several limitations, which deserve discussion: First, the use of printed images instead of digital images may be criticized. The authors concur that tools to magnify images or change image contrast might improve outcomes. However, the use of print-outs proved to be more practical for raters to display several different images side-by-side during the process of matching/exclusion. Second: it may be questioned that dental PMCT data was limited to single VRT images. The authors agree that for individual identification, the use of a whole PMCT dataset is a prerequisite since the PMCT images can (and must) be individually adapted to the AM bite-wing or panorex images. In this scenario, we used PMCT images that had already been individually reconstructed and adapted to the AM radiographs when they had been used for real forensic identification. It was part of the design of the study to only use images from cases which had been positively identified using PMCT images and dental radiographs. Moreover, the use of printed single VRT images in this study, would likely underestimate the rate of correct identifications. Therefore it is quite possible, that the use of original PMCT data would have increased the accuracy of identification in ambiguous cases. Finally, the fact that raters were asked to first indicate their level of confidence regarding a match/exclusion using the classification system of Interpol and second assess the number of concordant radiologic features on the PMCT image and the dental radiograph may have introduced a confirmation bias [24]. Confirmation bias is a cognitive bias “by which people tend to seek, perceive, interpret, and create new evidence in ways that verify preexisting beliefs” [24]. A number of publications have revealed how confirmation bias can have a significant impact on several forensic disciplines including fingerprint analysis, anthropology, bullet analysis, bite mark analysis, and even interpretation of DNA evidence [24–29]. Confirmation bias is especially frequent in situations where experts have to provide a subjective opinion (like indicating a level of confidence regarding a match in identification). It is important to state that this study was not designed to search for confirmation bias in dental identification, but it is certainly worth considering its occurrence in this study. The correlation found between the level of confidence and the number of concordant features of matched pairs suggests that the more confident raters are about the accuracy of an identification, the more concordant points will they believe to see (and vice versa). It is beyond the scope of this manuscript (and also beyond the design of the study) to discuss this effect in detail, but it is safe to say that single-rater identifications are susceptible to subjective bias. A very simple way to reduce confirmation bias is to involve multiple independent raters for identifications and reevaluate every case of rater disagreement with a third, independent rater or in a consensus reading. This approach would certainly have significantly reduced the number of false positive identifications in our study, since 80% (24/30) false matches were made by one of all five raters only. It is certainly no coincidence that this conclusion stands in agreement with the Interpol DVI guide which heavily emphasizes the importance of (interdisciplinary) team work in disaster victim identification.
5. Conclusions This study shows that visual comparison of PMCT images with AM dental radiographs is a reliable method for identification. Accuracy of identification using PMCT/AM dental radiographs was as
30
T.D. Ruder et al. / Journal of Forensic Radiology and Imaging 5 (2016) 25–30
high in this study as in comparable studies using PM dental radiographs/AM dental radiographs. There is no evidence that the accuracy of this approach suffers from the presence of CT image artifacts. These results provide evidence that practical experience in forensic identification is crucial to accurately assign positive matches, whereas experience with the imaging modality (in this case: dental PMCT) is inversely related to the number of false positive matches. Match accuracy did not correlate with subjective confidence or number of concordant points. It is advised to work in teams rather than individually when dealing with real cases in forensic identification, to minimize subjective interpretation and avoid confirmation bias.
Conflict of interest The authors have no conflict of interest to report.
References [1] P. Saukko, B. Knight, The forensic autopsy, in: P. Saukko, B. Knight (Eds.), Knight's Forensic Pathology, Edward Arnold Ltd., London, 2004, pp. 1–52. [2] Interpol. Disaster Victim Identification Guide (DVI Guide) (version 2014). Available from: 〈http://www.interpol.int/INTERPOL-expertise/Forensics/DVI-Pa ges/DVI-guide〉. [3] G.M. Hatch, F. Dedouit, A.M. Christensen, M.J. Thali, T.D. Ruder, RADid: a pictorial review of radiologic identification using post mortem CT, J. Forensic Radiol. Imag. 2 (2014) 52–59. [4] M. Baglivo, S. Winklhofer, G.M. Hatch, G. Ampanozi, M.J. Thali, T.D. Ruder, The rise of forensic and post mortem radiology – analysis of the literature between the year 2000 and 2011, J. Forensic Radiol. Imag. 1 (2013) 3–9. [5] M. Sidler, C. Jackowski, R. Dirnhofer, P. Vock, M. Thali, Use of multislice computed tomography in disaster victim identification—advantages and limitations, Forensic Sci. Int. 169 (2/3) (2006) 118–128. [6] C. O'Donnell, M. Iino, K. Mansharan, J. Leditscke, N. Woodford, Contribution of post mortem multidetector CT scanning to identification of the deceased in a mass disaster: experience gained from the 2009 Victorian bushfires, Forensic Sci. Int. 205 (2011) 15–28. [7] T.D. Ruder, M. Kraehenbuehl, W.F. Gotsmy, S. Mathier, L.C. Ebert, M.J. Thali, G. M. Hatch, Radiologic identification of disaster victims: a simple and reliable method using CT of the paranasal sinuses, Eur. J. Radiol. 81 (2012) e132–e138. [8] B. Morgan, A. Alminyah, A. Cala, C. O'Donnell, D. Elliott, G. Gorincour, P. Hofman, M. Iino, Y. Makino, A. Moskata, C. Robinson, G.N. Rutty, A. Sajantila, J. Vallis, N. Woodford, K. Woźniak, M. Viner, Use of post mortem computed tomography in Disaster Victim Identification-Positional statement of the members of the Disaster Victim Identification working group of the International Society of Forensic Radiology and Imaging, J. Forensic Radiol. Imag. 2 (2014) 114–116. [9] M.J. Thali, T. Markwalder, C. Jackowski, M. Sonnenschein, R. Dirnhofer, Dental CT imaging as a screening tool for dental profiling: advantages and limitations, J. Forensic Sci. 51 (2006) 113–119.
[10] C. Jackowski, E. Aghayev, M. Sonnenschein, R. Dirnhofer, M.J. Thali, Maximum intensity projection of cranial computed tomography data for dental identification, Int. J. Legal Med. 120 (2006) 165–167. [11] S. Kirchhoff, F. Fischer, G. Lindemaier, P. Herzog, C. Kirchhoff, C. Becker, J. Bark, M.F. Reiser, W. Eisenmenger, Is post mortem CT of the dentition adequate for correct forensic identification? comparison of dental computed tomograpy and visual dental record, Int. J. Legal Med. 122 (2008) 471–479. [12] A. Franco, P. Thevissen, W. Coudyzer, W. Develter, W. Van de Voorde, R. Oyen, D. Vandermeulen, R. Jacobs, G. Willems, Feasibility and validation of virtual autopsy for dental identification using the Interpol dental codes, J. Forensic Legal Med. 20 (2013) 248–254. [13] P. Stolzmann, S. Winklhofer, N. Schwendener, H. Alkadhi, M.J. Thali, T.D. Ruder, Monoenergetic computed tomography reconstructions reduce beam hardening artifacts from dental restorations, Forensic Sci. Med. Pathol. 9 (2013) 327–332. [14] C. Jackowski, A. Lussi, M. Classens, T. Kilchoer, S. Bolliger, E. Aghayev, A. Criste, R. Dirnhofer, M.J. Thali, Extended CT scale overcomes restoration caused streak artifacts for dental identification in CT – 3D color encoded automatic discrimination of dental restorations, J. Comput. Assist. Tomogr. 30 (2006) 510–513. [15] C. Jackowski, M. Wyss, A. Persson, M. Classens, M.J. Thali, A. Lussi, Ultra-highresolution dual-source CT for forensic dental visualization-discrimination of ceramic and composite fillings, Int. J. Legal Med. 122 (2008) 301–307. [16] D.F. MacLean, S.L. Kogon, L.W. Stitt, Validation of dental radiographs for human identification, J. Forensic Sci. 39 (1994) 1195–1200. [17] S.A. Sholl, G.H. Moody, Evaluation of dental radiographic identification: an experimental study, Forensic Sci. Int. 115 (2001) 165–169. [18] I.A. Pretty, R.J. Pretty, B.R. Rothwell, D. Sweet, The reliability of digitized radiographs for dental identification: a web-based study, J. Forensic Sci. 48 (2003) 1325–1330. [19] A. Wenzel, A. Richards, J. Heidmann, Matching simulated antemortem and post mortem dental radiographs from human skulls by dental students and experts: testing skills for pattern recognition, J. Forensic Odontostomatol. 28 (2010) 5–12. [20] A.B. Acharya, J.A. Taylor, Are a minimum number of concordant matches needed to establish identity in forensic odontology? J. Forensic Odontostomatol. 21 (1) (2003) 6–13. [21] A.M. Christensen, G.M. Hatch, B.G. Brogdon, A current perspective on forensic radiology, J. Forensic Radiol. Imag. 2 (3) (2014) 111–113. [22] B. Rosner, Fundamentals of Biostatistics, Duxbury Press, Belmont, CA, 2005. [23] American Board of Forensic Odontology, Diplomates Reference Manual (January 2013 edition). Available from: 〈http://www.abfo.org/wp-content/up loads/2012/08/ABFO-Reference-Manual-1-22-2013-revision.pdf〉. [24] S.M. Kassin, I.E. Dror, J. Kukucka, The forensic confirmation bias: problems, perspectives, and proposed solutions, J. Appl. Res. Mem. Cogn. 2 (2013) 42–52. [25] I.E. Dror, D. Charlton, Why experts make errors, J. Forensic Identif. (2006) 56600–56616. [26] S. Nakhaeizadeh, I.E. Dror, R.M. Morgan, Cognitive bias in forensic anthropology: visual assessment of skeletal remains is susceptible to confirmation bias, Sci. Justice 54 (2014) 208–214. [27] J. Kerstholt, A. Eikelboom, T. Dijkman, R. Stoel, R. Hermsen, B. van Leuven, Does suggestive information cause a confirmation bias in bullet comparisons? Forensic Sci Int. 198 (2010) 138–142. [28] M. Page, J. Taylor, M. Blenkin, Context effects and observer bias–implications for forensic odontology, J. Forensic Sci. 57 (2012) 108–112. [29] I.E. Dror, G. Hampikian, Subjectivity and bias in forensic DNA mixture interpretation, Sci. Justice 51 (2011) 204–208.