Post-mortem iris recognition with deep-learning-based image segmentation

Journal Pre-proof Post-mortem Iris Recognition with Deep-Learning-based Image Segmentation

Mateusz Trokielewicz, Adam Czajka, Piotr Maciejewicz PII:

S0262-8856(19)30459-7

DOI:

https://doi.org/10.1016/j.imavis.2019.103866

Reference:

IMAVIS 103866

To appear in:

Image and Vision Computing

Received date:

5 January 2019

Revised date:

28 June 2019

Accepted date:

3 December 2019

Please cite this article as: M. Trokielewicz, A. Czajka and P. Maciejewicz, Post-mortem Iris Recognition with Deep-Learning-based Image Segmentation, Image and Vision Computing(2019), https://doi.org/10.1016/j.imavis.2019.103866

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

© 2019 Published by Elsevier.

Journal Pre-proof

Post-mortem Iris Recognition with Deep-Learning-based Image Segmentation Mateusz Trokielewicza,∗, Adam Czajkab , Piotr Maciejewiczc and Machine Intelligence Laboratory, Research and Academic Computer Network Kolska 12, 01-045 Warsaw, Poland b Department of Computer Science, University of Notre Dame 46556 IN, USA c Department of Ophthalmology, Medical University of Warsaw Lindleya 4, 02-005 Warsaw, Poland

ro

of

a Biometrics

-p

Abstract

This paper proposes the first known to us iris recognition methodology designed specif-

re

ically for post-mortem samples. We propose to use deep learning-based iris segmentation models to extract highly irregular iris texture areas in post-mortem iris images. We

lP

show how to use segmentation masks predicted by neural networks in conventional, Gabor-based iris recognition method, which employs circular approximations of the

na

pupillary and limbic iris boundaries. As a whole, this method allows for a significant improvement in post-mortem iris recognition accuracy over the methods designed only for ante-mortem irises, including the academic OSIRIS and commercial IriCore imple-

ur

mentations. The proposed method reaches the EER less than 1% for samples collected up to 10 hours after death, when compared to 16.89% and 5.37% of EER observed for

Jo

OSIRIS and IriCore, respectively. For samples collected up to 369 hours post-mortem, the proposed method achieves the EER 21.45%, while 33.59% and 25.38% are observed for OSIRIS and IriCore, respectively. Additionally, the method is tested on a database of iris images collected from ophthalmology clinic patients, for which it also offers an advantage over the two other algorithms. This work is the first step towards post-mortem-specific iris recognition, which increases the chances of identification of deceased subjects in forensic investigations. The new database of post-mortem iris ∗ Corresponding

author Email address: [email protected] (Mateusz Trokielewicz)

Preprint submitted to Image and Vision Computing Journal

June 28, 2019

Journal Pre-proof

images acquired from 42 subjects, as well as the deep learning-based segmentation models are made available along with the paper, to ensure all the results presented in this manuscript are reproducible. Keywords: biometrics, iris recognition, post-mortem, image segmentation

1. Introduction

of

1.1. Post-mortem iris recognition

ro

Post-mortem iris recognition has recently gained attention from the biometrics community, starting with a Master’s thesis from Boston University [1], and studies on ex vivo pig eyes in 2015 [2]. Since then, some research groups have been hard at work

-p

5

evaluating how this biometrics works with irises of deceased subjects [3, 4, 5, 6, 7], and

re

proposing PAD (Presentation Attack Detection) techniques aimed at detecting cadaver iris presentations to the sensor [8, 9]. Post-mortem iris recognition is also discussed

10

lP

as a potentially useful method for forensic proceedings in addition to typical forensic biometric characteristics such as fingerprints, if the method could be applied reliably

na

[7]. Despite this attention, concluded with many claims that post-mortem iris biometrics, however challenging, may still be possible in favorable conditions, no specific, cadaver-aware methods for improving the recognition reliability have been proposed,

15

ur

apart from the work of Trokielewicz et al. [10], who introduced a data-driven semantic segmentation model for iris localization within images collected from cadaver eyes,

Jo

yet without evaluating how the new segmentation algorithm improves the recognition accuracy.

1.2. Forensic motivation There are several scenarios in which post-mortem iris recognition can be useful due 20

to its speed (when compared to usually slower DNA analysis), and which have been already explicitly appreciated by forensics community. One of them is more accurate matching of ante-mortem samples (if available) with post-mortem data obtained at crime scenes and mass fatality incidents. One of the United States Department of Justice’s operational requirements is defined as:

2

Journal Pre-proof

25

“Enhancement of unidentified decedent system(s) with weighting capability for antemortem and postmortem comparisons with the goal of providing a ranked list of best matches to effectively and efficiently identify potential candidates or hits.”1 . The work presented in this paper directly addresses this requirement. The second, operational practice-driven application, is to rapidly register the body

30

at the scene, to track it later and dispatch it correctly to family or mortuary. In case of

of

mass fatality accidents, DNA is too slow and forensic practitioners already decided to use iris recognition, which has been demonstrated for the first time in 2015 in Sansola’s

ro

thesis defended at the Boston Medical School2 .

-p

1.3. Contributions of this work

In this work, we build upon the findings presented in [10], by training several addi-

35

re

tional models with larger amounts of post-mortem data and different ground truth mask creation rules, together with constructing an iris image normalization method based on

lP

the circular Hough transform (CHT). This end-to-end segmentation algorithm is then coupled with the well known open source iris recognition method OSIRIS [11], and evaluated against the baseline OSIRIS performance and against the commercial Iri-

na

40

Core method, which presented the best performance of post-mortem iris matching in past studies [7]. The contributions that this study makes towards the state-of-the-

ur

art in post-mortem iris recognition are thus the following:

Jo

• introduction of two new, deep convolutional neural network-based (DCNN-based) post-mortem iris image segmentation methods, trained with different ground

45

truths, namely coarse iris approximation, and fine-grained, post-mortem specific masking of the regions affected by tissue decay,

• an end-to-end iris localization and image segmentation model that can be used as a drop-in replacement for any iris recognition methodology that uses normalized images with iris boundary approximated by circles,

50

1 https://www.nij.gov/topics/forensics/documents/2018-2-forensic-twg-table.

pdf 2 https://open.bu.edu/handle/2144/13975

3

Journal Pre-proof

• a new dataset of cadaver iris images collected from 42 subjects over a time period of up to 369 hours post-mortem3 , • experiments showing a considerable improvement in the matching performance of a method employing the proposed segmentation and Gabor wavelet-based en55

coding,

of

• source codes and neural network models’ weights, which allow for full reproducibility of the results, as well as incentive for further research.

ro

This article is laid out as follows. After the introductory part, Sec. 2 discusses the historical assumptions and the current course of post-mortem iris recognition research. Biometric databases used for training and estimation, as well as evaluation

-p

60

re

of the proposed approach are introduced in Sec. 3, whereas in Sec. 4 we describe the methodology of our solution and ways for its evaluation, which is performed and

lP

reported in Sec. 5. Finally, Sec. 6 contains relevant conclusions.

65

na

2. Related work

Even before any meaningful experimental studies regarding post-mortem iris recognition had come to fruition, both the research community and the industry were skep-

ur

tical about the feasibility of post-mortem iris recognition [12, 13, 14]. The first systematic study in this field was Sansola’s MSc thesis at Boston Univer-

70

Jo

sity [1], in which the IriShield M2120U iris recognition camera and IriCore matching software were employed to photograph irises of 43 deceased subjects at different postmortem time intervals. The method yielded 19-30% of false non-matches and no false matches, depending on the time since death involved. Later on, Saripalle et al. [2] studied ex-vivo eyes of domestic pigs and their biometric capability during the degradation of tissues after they are taken out of the cadaver. They found that the irises lose 75

their biometric capabilities 6 to 8 hours after death, proving the fast rate of ex-vivo eye degradation. 3 Release

agreement for this dataset is available at http://zbum.ia.pw.edu.pl/EN/node/46

4

Journal Pre-proof

In our previous works, we have refuted most of the popular claims, showing that the iris can still successfully serve as a biometric identifier for 27 hours after death [3], with the pupil typically remaining mid-sized without any excessive dilation or constriction, 80

with well visible iris structure. More than 90% of the irises were correctly recognized when photographed a few hours after death, including 100% accuracy obtained for the IriCore method (which comes from the same vendor as the camera used in this study).

of

As time after death increases, the equal error rate increases to 13.3% when images captured approximately 27 hours after death are compared against those obtained 5h after demise. Later, we showed that correct matches can still be expected even after 17

ro

85

iris images acquired from 17 cadavers [15].

-p

days [4] and offered the first known to us database of 1330 NIR and VIS post-mortem

A study tracking biometric capabilities of both the typically used forensic modali-

90

re

ties, namely fingerprints and faces, as well as iris is presented by Bolme et al. [5]. The authors placed twelve deceased subjects in outdoor conditions to assess how the envi-

lP

ronment and time affect the biometric performance. Irises were found to be degrading very fast and become unusable as shortly as few days after the body placement. When

na

the bodies were kept outside for 14 days, the correct verification rate reported by the authors decreased to 0.6%, but the real-life chance of recognizing an iris is estimated 95

to be less than 0.1%. Fingerprints and faces, on the other hand, remained readable for

ur

much longer periods of time.

Jo

Sauerwein et al. [6] showed that irises stay readable for up to 34 days after death, when cadavers were kept in outdoor conditions during winter. This readability parameter, however, was assessed by human experts acquiring the samples and no automatic 100

iris recognition algorithms were used. The most recent paper in this study by Trokielewicz et al. [7] included the most comprehensive experiments in this field to date, and introduces the new dataset of cadaver iris images [16], which in combination with the first part of the Warsaw corpus aggregates 1,200 near-infrared and 1,787 visible light samples collected from 37 de-

105

ceased subjects kept in mortuary conditions. In the foretold paper, we employed four iris recognition methods to analyze the genuine and impostor comparison scores and evaluate the dynamics of iris decay over a period of up to 814 hours. The findings show 5

Journal Pre-proof

that iris recognition may be close-to-perfect 5 to 7 hours after death and occasionally is still viable even 21 days after death. We also demonstrated that false-match probability 110

is significantly higher when live iris images are compared with post-mortem samples than when only live samples are used in comparisons.

3. Databases of iris images

of

For the purpose of this work, we use four different databases of iris images, includ-

115

ro

ing three subject-disjoint sets of post-mortem iris images, namely the:

• Warsaw-BioBase-Postmortem-Iris-v1.1, comprising 574 near-infrared (NIR)

-p

and 1023 visible light (VIS) images collected from 17 cadavers over a period of

re

up to 34 days [15, 4],

• Warsaw-BioBase-Postmortem-Iris-v2, a subject-disjoint extension of the v1.1

120

lP

of the database, adding 626 NIR and 764 VIS data from 20 more cadavers [16, 7], • Warsaw-BioBase-Postmortem-Iris-v3, a new set of images collected for the

na

purpose of this study, adding data from 42 more subjects with a total of 1094 NIR images and 785 VIS images, collected over a time horizon of up to 369

ur

hours since demise.

The first two datasets are aggregated together and used as training data, whereas the third set is employed for evaluation purposes. This way, the training and testing

Jo

125

data are subject-disjoint, i.e., data from subjects ‘participating’ in the training of the segmentation models and estimation of the normalization algorithm’s parameters is not repeated in the evaluation experiments. In addition to the post-mortem data, we use another database of challenging iris 130

images for supplemental testing of the proposed approach: • Warsaw-BioBase-Disease-Iris-v1, comprising iris images collected mostly from elderly patients of an ophthalmology clinic, including subjects with various ophthalmic conditions; we use a subset of 551 NIR images from 76 randomly chosen eyes [17, 18].

6

Journal Pre-proof

135

Following the findings reported in [7] that strongly favor the NIR images for postmortem iris recognition purposes, we only employ this type of images in all experiments conducted in this work. Example images from each of the evaluation databases

-p

ro

of

are shown in Fig. 1.

re

Figure 1: Example images from the test databases used in this work. Warsaw-BioBase-Postmortem-Iris-v3

140

na

4. Proposed methodology

lP

(left) and Warsaw-BioBase-Disease-Iris-v1 (right.)

4.1. Baseline iris recognition method OSIRIS, or Open Source for IRIS, is an academic solution developed as a part of

ur

the BioSecure EU project [11] and implements the original iris recognition concept of

Jo

John Daugman, including iris image segmentation and normalization to dimensionless polar coordinate system, followed by the calculation of a binary iris code using phase 145

quantization of the Gabor wavelet filtering outcomes. Fractional Hamming distance calculated using the XOR operation between the two codes is used as a comparison score, with values close to zero expected for two same-eye images, whereas scores oscillating around 0.5 should be produced when two different-eye images are compared. Due to the additional compensation of the eyeball rotation, different-eye score

150

distributions are typically skewed toward 0.4 – 0.45 range. The OSIRIS, although the last version was developed in 2013, is still considered by many researchers in the biometric community as the most accurate open source iris matcher. It also closely follows a dominant pipeline in iris recognition. We are 7

Journal Pre-proof

not aware of any other open-source iris methods that would outperform OSIRIS. The 155

Masek’s implementation4 and the USIT toolkit5 performed worse than OSIRIS on postmortem iris samples in our research, and thus we decided to not include them. The original score calculation method can be complemented by score normalization penalizing comparisons with small number of compared bits, as compared to the typical

of

(here: within a given dataset) number of compared bits, as proposed by Daugman [19]: r n HDnorm = 0.5 − (0.5 − HDraw ) N

ro

This transforms the samples of scores obtained when comparing different eyes into samples drawn from the same binomial distribution, as opposed to drawing sample

160

-p

scores from different binomial distributions with σ dependent on the number of bits n that were available for comparison (commonly unmasked bits). N is the ‘typical

re

number of bits compared (unmasked) between two different irises,’ which Daugman states to be 911 or 960, depending on the data. The N parameter is estimated for a

lP

particular database of iris images. Since this work uses different datasets of iris images and different subsets of these datasets (i.e., post-mortem samples with varying time horizon), the N is calculated separately for each experiment (cf. description in Sec. 5).

na

165

The total number of bits in the OSIRIS code is 1536.

ur

In the following Sections we will evaluate the proposed approach and discuss whether applying this normalization procedure is beneficial for the overall system’s

170

Jo

performance.

4.2. Data-driven image segmentation models The first stage of the proposed scheme consists of employing a data-driven segmentation model based on a deep convolutional neural network (DCNN) for the purpose of localizing the iris within an image. Our models are based on the solution described in one of our previous papers [10], which employs a re-trained off-the-shelf SegNet model

175

[20], and offers a large improvement over the OSIRIS segmentation method measured 4 https://www.peterkovesi.com/studentprojects/libor/index.html 5 http://www.wavelab.at/sources/Rathgeb16a/

8

Journal Pre-proof

with the IoU (Intersection over Union) metric. We fine-tuned it with a database of cadaver iris images and their corresponding ground truth binary masks. The neural networks were trained and tested with entirely subject-disjoint sets of iris images. Overfitting was minimized in a standard way by applying cross-validation with the training 180

subset. The network generalizes well on images collected from subjects unknown during the training. The masks used for training were annotated in a fine-grained manner,

of

which has a purpose of teaching the network to only localize iris regions that are not affected by post-mortem changes, and represent the well-visible iris texture.

185

ro

In addition to the model described above, in this work we have implemented and trained two additional models, ending up with three models in total, which are detailed

-p

below and their example predictions given in Fig. 2

re

1. fine segmentation model: the one published in [10], trained with data from the Warsaw-Biobase-Postmortem-Iris-v1.1 database with fine-grained ground

190

lP

truth masks and yielding predictions of 120 × 160 pixels, 2. fine v2highres segmentation model: similar to the fine model, but trained with data from Warsaw-Biobase-Postmortem-Iris-v1.1 database together with NIR

na

samples from the Warsaw-Biobase-Postmortem-Iris-v2 database for twice as many epochs (120 vs 60), also with fine-grained ground truth masks, but yielding

195

ur

higher-resolution predictions, namely 240 × 320 pixels, 3. coarse segmentation model: trained with both NIR and VIS data from the v1 and

Jo

v2 versions of the Warsaw-Biobase-Postmortem-Iris database for 120 epochs, but this time with coarse ground truth masks, denoting only the inner and the outer iris boundary and eyelids, also producing masks in 240 × 320 size.

4.3. Iris and mask normalization 200

For the proposed method to serve as a drop-in replacement for the Daugman method recognition pipeline, the prediction obtained from the DCNN-based segmentation has to be correctly normalized onto a dimensionless polar-coordinate rectangle. For this stage, a method for localizing pupillary and limbic iris boundaries has been devised, which employs a Hough transform that is applied to the prediction obtained from the

9

Journal Pre-proof

ro

of

OSIRIS segmentation, normalized images, and normalized masks

coarse CNN binary predictions with fitted Hough circles, segmented images,

Jo

ur

na

lP

re

-p

normalized images, and normalized masks

Figure 2: Segmentation results for samples from Fig. 1, obtained from the unmodified OSIRIS method and the coarse segmentation model. Warsaw-BioBase-Postmortem-Iris-v3 (left), Warsaw-BioBase-Disease-Irisv1 (right).

10

Journal Pre-proof

fine and fine v2highres CNN segmented images, normalized images, and normalized

ro

of

masks

-p

Figure 3: Segmentation results for post-mortem sample from Fig. 2, obtained from the fine and fine

205

re

v2highres segmentation models.

coarse segmentation model, similarly to the methodology introduced in [21] and [22],

lP

as this model yields the smoothest prediction. These boundary parameters are then used in all subsequent experiments, including those involving the fine and fine v2highres

na

models. The segmentation methodology employing the coarse segmentation model is visualized in Fig. 2. Similar results are then presented in Fig. 3 for the two remain210

ing fine-grained models. Notably, the original fine model from [10] seems to over-

ur

aggressively mask out some portions of the iris, while at the same time mistakenly

Jo

denoting some portions of the pupil as iris. This is fixed in the fine v2highres model, which was trained with twice the iterations count, cf. right side of Fig. 3. 4.4. IriCore: a commercial benchmark method 215

In addition to evaluating our approach against the unmodified OSIRIS algorithm, we also choose to compare it with one of the leading iris matchers from commercial vendors, namely the IriCore [23], which offered the best performance out of four matchers in our previous evaluations of post-mortem decay impact on various iris recognition methodologies [7]. This commercial matcher was also ranked as one of

220

the two best matchers in NIST IREX I, it has the STQC certification and is part of the world largest biometric project in India (AADHAAR). IriCore comes from the same

11

Journal Pre-proof

manufacturer as the IriShield camera used to collect post-mortem samples in our studies – and therefore it may have an additional advantage in this particular task. Since IriCore follows an undisclosed iris recognition algorithm, and can be only used as a 225

black box taking two images and producing a dissimilarity score between them. The distributions of these dissimilarity scores, which are expected to be ranging from 0 to 1.1 for genuine comparisons and from 1.1 to 2.0 for impostor ones, are then used to

of

construct ROC curves which can be compared against those obtained from original and

5. Iris recognition accuracy evaluation

-p

230

ro

modified OSIRIS. No additional score normalization is introduced for IriCore.

5.1. Comparison score generation

re

In this Section we evaluate the new segmentation approach introduced in Sec. 4 by injecting the segmentation results obtained from the three DCNN-based models into the

235

lP

OSIRIS recognition pipeline, and also comparing their performance with unmodified OSIRIS algorithm, as well as with the accuracy offered by the IriCore. The evaluation

na

is performed on two test databases described in Sec. 3. Generation of genuine and impostor comparison score distributions was done by performing comparisons between all possible iris image pairs. One comparison per a given image pair was performed,

240

image A.

ur

i.e., if image A was matched against image B, then image B is not matched against

Jo

For the post-mortem data, eight separate score distributions are generated for both the genuine and impostor comparisons, taking into account samples collected during different, progressing observation horizons in respect to time that has elapsed since a subject’s death. These include scores obtained from samples collected fewer than 10 245

hours after death, fewer than 24 hours, 48 hours, 60 hours, 110 hours, 160 hours, 210 hours, and finally 370 hours (all available samples). This is to evaluate the recognition accuracy in respect to the increasing post-mortem interval. All possible comparison pairs were evaluated here as well.

12

Journal Pre-proof

Table 1: Summary of the Equal Error Rates (EERs) obtained when performing all possible genuine and impostor comparisons in data subsets as defined in Sec. 5 for the two benchmark methods and the best proposed method (fine v2highres model). Numbers of comparisons for each experiment are included in the last column. original OSIRIS

IriCore

proposed

[%]

[%]

[%]

≤ 10h postmortem

16.89

5.37

0.96

≤ 24h postmortem

18.73

6.31

1.36

61 425

≤ 48h postmortem

20.34

8.00

3.38

88 831

≤ 60h postmortem

23.69

10.46

≤ 110h postmortem

24.72

14.36

≤ 160h postmortem

28.68

17.79

15.02

405 450

≤ 210h postmortem

29.94

20.77

17.60

476 776

≤ 369h postmortem

33.59

25.38

21.95

597 871

disease data

8.90

3.97

2.55

152 076

number of comparisons

ro

of

46 360

7.18

178 503

10.36

296 065

-p

re

lP

Data subset

250

na

5.2. Recognition accuracy: post-mortem dataset The results are presented in the form of receiver operating characteristic (ROC) curves, which plot the correct match rates against false match rates. In addition, equal

ur

error rates (EER) are calculated and denoted in the plot legend. Fig. 7 shows ROC

Jo

curves denoting the performance of the baseline methods (OSIRIS and IriCore) performance compared against performance obtained by employing image segmentation 255

results from the three DCNN-based models for the post-mortem data in four shortest post-mortem time horizons, comprising samples collected earlier than 60 hours after the demise of each subject. Raw Hamming distance scores are plotted in solid lines, whereas the scores after score normalization (cf. Sec. 4.1) are plotted with dotted lines. Fig. 8 contains the analogous graphs, but for the remaining four longer time horizons,

260

namely those with samples collected up to 369 hours post-mortem. Equal Error Rates for all observation horizons for the two benchmark methods, as well as the proposed approach (winning fine v2highres model), are summarized in Tab. 1. Notably, very good performance can be achieved using each one of the proposed 13

Journal Pre-proof

models for samples collected up to 10 hours after death, with EER as low as 0.96%, 265

which only slightly increases to 1.36% when samples collected up to 24 hours postmortem are added to the database. For samples collected up to 48 hours (two days) after death, the lowest EER offered by the fine v2highres model is 3.38%, which means that the method can still be usable. As more difficult samples are being added to the database, the errors increase, reaching 21.95% for the best performing fine v2highres model. Although the fine model given better EER, it also presents a risk of higher

of

270

false match rates than the coarse model (see the region of the ROC curve near the

ro

Y axis). This can be alleviated by introducing score normalization (cf. Sec. 4.1), but this in turn increases the EER by almost 3%. The fine v2highres model, on the

275

-p

other hand, seems to offer similar risk of false matches as the fine model with score normalization, but at the same time preserving most of the overall performance with

re

EER only slightly higher than this of the fine model. Original OSIRIS, on the other hand, is incomparable to the proposed approach from the very beginning, yielding

lP

EER of almost 17% for the ‘easiest’ samples, and the EERs from this method are increasing steadily up to 33.59% for the entire database (samples collected up to 370 hours post-mortem). The IriCore’s performance starts with EER=5.37% for the easiest

na

280

samples, which is still more than 4 percentage points behind our proposed solution, and decreases steadily with the increasing post-mortem time horizon, reaching 25.38% for

ur

the most challenging set of samples (all images). The only advantage the IrisCore may

285

Jo

have over our approach is better performance in selected time post-mortem horizons (samples collected up to 60, 110 and 160 hours) in very low FMR registers. 5.3. Recognition accuracy: elderly/ophthalmic conditions dataset Fig. 4 presents ROC curves obtained when matching samples from the WarsawBioBase-Disease-Iris-v1 databases using a baseline OSIRIS approach and the proposed segmentation approach based on the coarse model. Because this dataset does not con290

tain any post-mortem samples, there is no justification to apply post-mortem-aware fine-grained segmentation model here. Because this test database contains images collected from elderly people, often with various ophthalmic conditions – also challenging ones, the advantage of the proposed 14

Journal Pre-proof

Subset of Disease-Iris-v1, full cross comparisons

0.8

0.6 EER line 0.4 baseline OSIRIS, EER = 8.90% baseline w/ normalization, EER = 9.11% IriCore, EER = 3.97% proposed, EER = 2.55% proposed w/ normalization, EER = 3.11%

0.2

0

0

0.2

0.4

0.6

0.8

Subset of Disease-Iris-v1, full cross comparisons

1

1 - False Non-Match Rate

1 - False Non-Match Rate

1

0.95

0.9

baseline OSIRIS, EER = 8.90% baseline w/ normalization, EER = 9.11% IriCore, EER = 3.97% proposed, EER = 2.55% proposed w/ normalization, EER = 3.11%

0.85

0.8

1

0

0.005

0.01

0.015

0.02

False Match Rate

False Match Rate

of

Figure 4: ROC curves obtained for the Warsaw-BioBase-Disease-Iris-v1 dataset. Original (solid lines) and

ro

normalized (dotted lines) scores are shown. The plot on the right is a close-up of the one on the left.

approach is evident, as it offers EER=2.55% compared to 8.9% yielded by OSIRIS and 3.97% obtained from IriCore.

-p

295

re

5.4. Discussion of score normalization

As argued in Sec. 4.1, score normalization can help rectify significant portions of

lP

false matches that were caused by comparing small numbers of commonly unmasked bits in two iris codes. As seen in Fig. 7, employing this normalization for the very selective segmentation model, namely the old fine model (red plots) can vastly reduce

na

300

the false matches at a little EER-related cost. With less selective segmentation, how-

ur

ever, the normalization cannot offer any advantage over lack of such, e.g., in the best performing fine v2 highres model (black plots). With increasing time horizons, as in

305

Jo

Fig. 8, even the less selective models start to mask out iris portions more aggressively, hence the score normalization can improve the performance in low FMR registers of most of the models.

5.5. Rank list performance In addition to the verification performance evaluations given by the ROC curves and EER values, we perform the experiment involving candidate list accuracy. Figure 310

5 presents Cumulative Match Characteristic (CMC) curves for the post-mortem dataset. The experiments for the post-mortem data are constructed in such a way that the gallery sample to which the subsequent probe samples are being matched is always the image collected during the first acquisition session. The probe samples are those collected at 15

Journal Pre-proof

least after 24 hours post-mortem, 48 hours, 60 hours, 110 hours, 160 hours, and finally 315

after 210 hours, i.e., the test subset is getting more difficult in each experiment, as it encompasses progressively more decayed iris samples. This experiment can reflect the more real-life application of the method, where the iris recognition algorithm is used as an aid to the forensic examiner, by proposing a candidate list, which is then refined by the human expert. Therefore, the Rank10 values are considered in each experiment. We see that even after 210 hours, or almost 9 days,

of

320

the chance of including the correct hit on a candidate list comprising 10 suggestions

ro

can be as high as 70%. It has to be noted, though, that the small size of the dataset can cause these error rates to be either under- or over-estimated. The proposed method

325

-p

outperforms both benchmark approaches in the Rank10 metric, with the advantage of our algorithm being the larger the more difficult samples are in the probe set. The

re

Rank10 identification error rates are summarized in Tab. 2 Table 2: Summary of the Rank10 identification accuracy obtained from the candidate list experiments for the

lP

two benchmark methods and the best proposed method (fine v2highres model). Numbers of comparisons for each experiment are included in the last column. original OSIRIS

IriCore

proposed

[%]

[%]

[%]

67.32

78.35

83.23

240 645

≥ 48h postmortem

65.76

76.30

80.77

226 615

≥ 60h postmortem

59.31

70.42

74.84

151 280

≥ 110h postmortem

50.00

60.05

65.31

138 470

≥ 160h postmortem

53.99

53.61

65.40

58 865

≥ 210h postmortem

51.22

55.15

70.30

35 685

na

Probe subset

Jo

ur

≥ 24h postmortem

number of comparisons

5.6. False Non-Match Rate dynamics In addition to the ROCs, we have also calculated the False Non-Match Rate (FNMR) values at acceptance thresholds which allow the False Match Rate (FMR) values to 330

stay below 1%. This is done for the post-mortem data to reveal the dynamics of the FNMR as a function of post-mortem sample capture horizon, and therefore to know 16

Journal Pre-proof

Post-mortem data, after at least 24h

100

90

Cumulative accuracy [%]

Cumulative accuracy [%]

90 80 70 60 original OSIRIS, Rank10 = 67.32% IriCore, Rank10 = 78.35% proposed, Rank10 = 83.23%

50 40

Post-mortem data, after at least 48h

100

0

10

20

30

40

50

60

70

80

80 70 60 original OSIRIS, Rank10 = 65.76% IriCore, Rank10 = 76.30% proposed, Rank10 = 80.77%

50 40

90

0

10

20

30

Rank Post-mortem data, after at least 60h

90

50 original OSIRIS, Rank10 = 59.31% IriCore, Rank10 = 70.42% proposed, Rank10 = 74.84% 10

20

30

40

50

60

Rank Post-mortem data, after at least 160h

100

60 50 40

original OSIRIS, Rank10 = 50.00% IriCore, Rank10 = 60.05% proposed, Rank10 = 65.31%

30

80

20

0

10

20

30

40

50

60

70

80

Rank Post-mortem data, after at least 210h

100

lP

70 60 50 40 30

original OSIRIS, Rank10 = 53.99% IriCore, Rank10 = 53.61% proposed, Rank10 = 65.40%

20 10

80

90

80

na

Cumulative accuracy [%]

90

70

70

re

0

70

ro

60

80

-p

Cumulative accuracy [%]

70

Cumulative accuracy [%]

Cumulative accuracy [%]

80

30

60

Post-mortem data, after at least 110h

100

90

40

50

of

100

40

Rank

0

10

20

30

40

50

60

70

80 70 60 50 40 30 original OSIRIS, Rank10 = 51.52% IriCore, Rank10 = 55.15% proposed, Rank10 = 70.30%

20

80

10

0

10

20

30

40

50

60

70

80

Rank

ur

Rank

Figure 5: Cumulative match characteristic (CMC) curves plotted collectively for six different scenarios of

Jo

matching the gallery sample (collected shortly after death) with probe samples obtained at least after each of the time periods. Proposed method compared against original OSIRIS method and the commercial matcher. Rank10 scores are given in the plot legends.

the chances for a false non-match as time since death progresses. We plot this dynamics for the two baseline methods: OSIRIS and IriCore, as well as the best performing fine v2highres DCNN-based modification, Fig. 6. Notably, for each moment during 335

the increasing post-mortem sample capture time horizon, our proposed approach consistently offers an advantage over the other two algorithms, allowing to reach nearly perfect recognition accuracy for samples collected up to a day after a subject’s death.

17

Journal Pre-proof

False Non-Match Rate dynamics vs time post-mortem

FNMR @ FMR = 0.01 [%]

60

50

40

30

20

of

proposed (fine v2highres) IriCore

10

original OSIRIS

10 24

48 60

110

160

210

ro

0

369

post-mortem time horizon [hours]

-p

Figure 6: Dynamics of False Non-Match Rates in the function of post-mortem sample capture horizon,

re

plotted for two baseline iris recognition methods, and the approach proposed in this paper.

6. Conclusions

340

lP

This work introduces the first known to us attempt at improving iris recognition reliability for post-mortem samples, by offering a robust image segmentation method

na

that can be used as a drop-in image segmentation replacement for the OSIRIS, or any iris recognition pipeline that involves iris normalization onto a dimensionless polar rectangle.

achieve close-to-perfect recognition accuracy on the subject-disjoint evaluation database

Jo

345

ur

The proposed solution, employing DCNN-based semantic segmentation, allows to

of post-mortem samples, as well as on another challenging database of iris images, collected from elderly individuals with various ophthalmic conditions. This proves the flexibility of the proposed approach in solving problems associated with atypical iris images, which the models introduced in this work can reliably and consistently localize 350

and segment in most of the cases. Despite the expected drop in recognition accuracy with the increasing time horizon of post-mortem sample collection, the proposed approach consistently outperforms both the academic and the commercial, state-of-the-art iris recognition methods. This in confirmed in both the verification and identification scenario, in which the proposed

355

method was able to offer Rank10 accuracy of more than 70% when matching probe 18

Journal Pre-proof

samples collected 9 days after the gallery sample, compared to only 55% offered by the open-source matcher and the commercial solution. The results presented in this work can have great significance for forensics, as they offer substantial improvements in the post-mortem iris matching that can be employed, 360

for instance, for the identification of casualties at crime scenes or mass fatality incidents, as well for ensuring the correctness of cadaver handling. This work directly

of

addresses the current operational trends and needs of forensic practitioners, and we

ro

believe it brings valuable contribution to science.

365

-p

Acknowledgment

The Authors would like to kindly thank Ewelina Bartuzi for her help with generat-

re

ing the CMC curves for the rank list experiments.

lP

References

[1] A. Sansola, Postmortem iris recognition and its application in human identifica-

370

na

tion (Master’s Thesis, Boston University, 2015). [2] S. K. Saripalle, A. McLaughlin, R. Krishna, A. Ross, R. Derakhshani, Post-

ur

mortem Iris Biometric Analysis in Sus scrofa domesticus, IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS)doi:

Jo

10.1109/BTAS.2015.7358789. [3] M. Trokielewicz, A. Czajka, P. Maciejewicz, Post-mortem Human Iris Recogni375

tion (9th IAPR International Conference on Biometrics (ICB 2016), June 13-16, 2016, Halmstad, Sweden, 2016). doi:10.1109/ICB.2016.7550073. [4] M. Trokielewicz, A. Czajka, P. Maciejewicz, Human Iris Recognition in Postmortem Subjects: Study and Database (8th IEEE International Conference on Biometrics: Theory, Applications and Systems, Sep 6-9, 2016, Buffalo, USA,

380

2016). doi:10.1109/BTAS.2016.7791175.

19

Journal Pre-proof

[5] D. S. Bolme, R. A. Tokola, C. B. Boehnen, Impact of environmental factors on biometric matching during human decomposition (8th IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS 2016), Sep 6-9, 2016, Buffalo, USA, 2016). 385

[6] K. Sauerwein, T. B. Saul, D. W. Steadman, C. B. Boehnen, The effect of decomposition on the efficacy of biometrics for positive identification, Journal of Foren-

of

sic Sciences 62 (6) (2017) 1599–1602. doi:10.1111/1556-4029.13484.

ro

[7] M. Trokielewicz, A. Czajka, P. Maciejewicz, Iris recognition after death, IEEE Transactions on Information Forensics and Security 4 (6) (2018) 1501 – 1514. doi:10.1109/TIFS.2018.2881671.

-p

390

re

[8] M. Trokielewicz, A. Czajka, P. Maciejewicz, Presentation Attack Detection for Cadaver Iris, 9th IEEE International Conference on Biometrics: Theory, Appli-

lP

cations, and Systems (BTAS 2018)doi:10.1109/BTAS.2018.8698542. [9] MIT Technology Review, Iris scanner can distinguish dead eyeballs from living ones (https://www.technologyreview.com/s/611675/iris-scanner-can-distinguish-

na

395

dead-eyeballs-from-living-ones/, accessed Dec 7, 2018).

ur

[10] M. Trokielewicz, A. Czajka, Data-driven segmentation of post-mortem iris images, in: 2018 International Workshop on Biometrics and Forensics (IWBF),

400

Jo

2018, pp. 1–7. doi:10.1109/IWBF.2018.8401558. [11] G. Sutra, B. Dorizzi, S. Garcia-Salitcetti, N. Othman, A biometric reference system for iris. OSIRIS v4.1:

http://svnext.it-sudparis.eu/svnview2-

eph/ref syst/iris osiris v4.1/ (accessed: Oct 1, 2014). [12] BBC

News,

The

eyes

have

it

(http://news.bbc.co.uk/2/hi/

sci-

ence/nature/1477655.stm, 9 August, 2001 (accessed: April 10, 2016)). 405

[13] IrisGuard,

EyeBank

Solution

(http://www.irisguard.com/

bank/downloads/EyeBankPer%20Page.pdf (accessed: Jan 2016)).

20

eye-

Journal Pre-proof

[14] IriTech, Biometric Access Control Can Iris Biometric Enhance Better Security? (http://www.iritech.com/blog/iris-biometric-access-control, August 20, 2015 (accessed: January 16, 2016)). 410

[15] Warsaw University of Technology, Warsaw-BioBase-PostMortem-Iris-v1.0: http://zbum.ia.pw.edu.pl/en/node/46 (2016). [16] Warsaw University of Technology, Warsaw-BioBase-PostMortem-Iris-v2.0:

of

Technology,

http://zbum.ia.pw.edu.pl/en/node/46 (2015).

Warsaw-BioBase-Disease-Iris-v1.0:

-p

415

University

ro

[17] Warsaw

of

http://zbum.ia.pw.edu.pl/en/node/46 (2018).

[18] M. Trokielewicz, A. Czajka, P. Maciejewicz, Database of iris images acquired

re

in the presence of ocular pathologies and assessment of iris recognition reliability for disease-affected eyes, 2nd IEEE International Conference on Cybernetics

420

lP

(CYBCONF 2015), Special Session on Reliable Biometrics (BIORELIABILITY 2015), June 24-26, 2015, Gdynia, Polanddoi:10.1109/CYBConf.2015.

na

7175984.

[19] J. Daugman, New methods in iris recognition, IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics 37 (5) (2007) 1167–1175. doi:

[20] V. Badrinarayanan, A. Kendall, R. Cipolla, SegNet: A Deep Convolutional

Jo

425

ur

10.1109/TSMCB.2007.903540.

Encoder-Decoder Architecture for Image Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence 39. doi:10.1109/TPAMI.2016. 2644615. [21] D. Kerrigan, M. Trokielewicz, A. Czajka, K. W. Bowyer, Iris recognition with 430

image segmentation employing retrained off-the-shelf deep neural networks, 12th IAPR International Conference on Biometrics (ICB 2019). URL http://arxiv.org/abs/1901.01028 [22] H. Hofbauer, E. Jalilian, A. Uhl, Exploiting superior cnn-based iris segmentation for better recognition accuracy, Pattern Recognition Letters 120 (2019) 17 – 23. 21

Journal Pre-proof

doi:https://doi.org/10.1016/j.patrec.2018.12.021. URL http://www.sciencedirect.com/science/article/pii/ S0167865518309395

ur

na

lP

re

-p

ro

of

[23] IriTech Inc., IriCore Software Developers Manual, version 3.6 (2013).

Jo

435

22

Journal Pre-proof

Postmortem-Iris-v3, time since death <= 10

1

1

0.8

EER line baseline OSIRIS, EER = 16.89% baseline OSIRIS w/ normalization, EER = 17.08% IriCore, EER = 5.37% coarse, EER = 0.96% coarse w/ normalization, EER = 1.16% fine, EER = 1.00% fine w/ normalization, EER = 1.73% fine v2highres, EER = 0.96% fine v2highres w/ normalization, EER = 2.11%

0.6

0.4

0.2

0

1 - False Non-Match Rate

1 - False Non-Match Rate

0.95 0.9 0.85 0.8 0.75 0.7 0.65 0.6

0

0.2

0.4

0.6

0.8

1

0

0.002

0.004

False Match Rate 1

0.2

0.4

0.6

0.8

Postmortem-Iris-v3, time since death <= 48

1

1

0.002

0.004

0.006

0.008

0.01

0.008

0.01

False Match Rate

EER line baseline OSIRIS, EER = 20.34% baseline OSIRIS w/ normalization, EER = 20.96% IriCore, EER = 8.00% coarse, EER = 3.76% coarse w/ normalization, EER = 3.91% fine, EER = 3.48% fine w/ normalization, EER = 3.78% fine v2highres, EER = 3.38% fine v2highres w/ normalization, EER = 4.29%

na

0.6

0.4

0.2

0

0.2

0.4

0.6

0.95 0.9 0.85 0.8 0.75 0.7 0.65 0.6

0.8

1

0

0.002

False Match Rate

1 - False Non-Match Rate

0.6

EER line baseline OSIRIS, EER = 23.69% baseline OSIRIS w/ normalization, EER = 24.68% IriCore, EER = 10.46% coarse, EER = 7.52% coarse w/ normalization, EER = 7.67% fine, EER = 7.37% fine w/ normalization, EER = 8.52% fine v2highres, EER = 7.18% fine v2highres w/ normalization, EER = 8.95%

0.4

0.2

0.006

0.9

Jo

0.8

0.004

False Match Rate

Postmortem-Iris-v3, time since death <= 60

1

1 - False Non-Match Rate

0.01

ro 0

lP

0.8

0

0.008

0.7

0.65

1

ur

1 - False Non-Match Rate

0.8 0.75

0.6

0

False Match Rate

0

0.9 0.85

re

0.2

0.01

-p

EER line baseline OSIRIS, EER = 18.73% baseline OSIRIS w/ normalization, EER = 18.74% IriCore, EER = 6.31% coarse, EER = 1.36% coarse w/ normalization, EER = 1.71% fine, EER = 1.36% fine w/ normalization, EER = 1.89% fine v2highres, EER = 1.36% fine v2highres w/ normalization, EER = 2.39%

0.4

0

1 - False Non-Match Rate

0.8

1 - False Non-Match Rate

1 - False Non-Match Rate

0.95

0.6

0.008

of

Postmortem-Iris-v3, time since death <= 24

1

0.006

False Match Rate

0.8

0.7

0.6

0.5

0.4 0

0.2

0.4

0.6

0.8

1

False Match Rate

0

0.002

0.004

0.006

False Match Rate

Figure 7: Receiver Operating Characteristic curves obtained when comparing post-mortem samples with different observation time horizons. Results for the baseline OSIRIS performance, IriCore benchmark method, and three modifications introduced by the Authors are presented. Original (solid lines) and normalized (dotted lines) scores are shown. Plots on the right are close-ups of the ones on the left in each pair.

23

Journal Pre-proof

Postmortem-Iris-v3, time since death <= 110

1

0.8

0.8

EER line baseline OSIRIS, EER = 24.72% baseline OSIRIS w/ normalization, EER = 25.46% IriCore, EER = 14.36% coarse, EER = 11.01% coarse w/ normalization, EER = 11.17% fine, EER = 10.35% fine w/ normalization, EER = 11.49% fine v2highres, EER = 10.36% fine v2highres w/ normalization, EER = 12.18%

0.6

0.4

0.2

0.7 0.65 0.6 0.55 0.5 0.45 0.4

0

0.2

0.4

0.6

0.8

1

0

0.002

False Match Rate 0.7

0.2

0.4

0.6

0.8

1

0.01

0.008

0.01

0.008

0.01

ro 0

0.002

0.004

0.006

False Match Rate

EER line baseline OSIRIS, EER = 29.94% baseline OSIRIS w/ normalization, EER = 30.75% IriCore, EER = 20.77% coarse, EER = 19.19% coarse w/ normalization, EER = 19.27% fine, EER = 17.37% fine w/ normalization, EER = 19.52% fine v2highres, EER = 17.60% fine v2highres w/ normalization, EER = 20.37%

na

0.6

0.4

0.2

0

0.2

0.4

0.6

0.6 0.55 0.5 0.45 0.4 0.35 0.3 0.25 0.2

0.8

1

0

0.002

False Match Rate

0.004

0.006

Jo

False Match Rate

Postmortem-Iris-v3, time since death <= 370

0.8

0.6

0.5

0.45

1 - False Non-Match Rate

1

1 - False Non-Match Rate

0.008

0.4

0.35

lP

0.8

ur

1 - False Non-Match Rate

0.5

0.45

Postmortem-Iris-v3, time since death <= 210

1

EER line baseline OSIRIS, EER = 33.59% baseline OSIRIS w/ normalization, EER = 34.84% IriCore, EER = 25.38% coarse, EER = 23.95% coarse w/ normalization, EER = 23.98% fine, EER = 21.45% fine w/ normalization, EER = 24.33% fine v2highres, EER = 21.95% fine v2highres w/ normalization, EER = 25.41%

0.4

0.2

0

0.01

0.3

0

False Match Rate

0

0.6 0.55

re

0.2

0.008

-p

EER line baseline OSIRIS, EER = 28.68% baseline OSIRIS w/ normalization, EER = 29.34% IriCore, EER = 17.79% coarse, EER = 16.23% coarse w/ normalization, EER = 16.36% fine, EER = 14.92% fine w/ normalization, EER = 16.72% fine v2highres, EER = 15.02% fine v2highres w/ normalization, EER = 17.58%

0.4

0

1 - False Non-Match Rate

0.8

1 - False Non-Match Rate

1 - False Non-Match Rate

0.65

0.6

0.006

False Match Rate

Postmortem-Iris-v3, time since death <= 160

1

0.004

of

0

1 - False Non-Match Rate

1 - False Non-Match Rate

0.75

0.4

0.35

0.3

0.25

0.2

0

0.2

0.4

0.6

0.8

1

False Match Rate

0

0.002

0.004

0.006

False Match Rate

Figure 8: Same as in Fig. 7, but plotted for comparison scores obtained from long-term samples collected from 210 up to 369 hours post-mortem.

24

Journal Pre-proof

Highlights

ur

na

lP

re

-p

ro

of

post-mortem-specific iris image segmentation tool drop-in segmentation stage replacement for typical iris recognition pipelines new dataset of cadaver iris images (42 subjects) experiments showing substantial matching accuracy improvements

Jo

   

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8