Fingerprint pore matching using deep features

Pattern Recognition 102 (2020) 107208

Contents lists available at ScienceDirect

Pattern Recognition journal homepage: www.elsevier.com/locate/patcog

Fingerprint pore matching using deep features Feng Liu a,b,c, Yuanhao Zhao a, Guojie Liu a, Linlin Shen a,∗ a

College of Computer Science and Software Engineering, Shenzhen University, Shenzhen 518060, China SZU Branch, Shenzhen Institute of Artificial Intelligence and Robotics for Society, China c Guangdong Key Laboratory of IntelligentInformation Processing, Shenzhen University, Shenzhen 518060, China b

a r t i c l e

i n f o

Article history: Received 13 April 2019 Revised 18 December 2019 Accepted 16 January 2020 Available online 16 January 2020 Keywords: Fingerprint recognition Pore representation Direct pore matching Convolutional neural networks

a b s t r a c t As a popular living fingerprint feature, sweat pore has been adopted to build robust high resolution automated fingerprint recognition systems (AFRSs). Pore matching is an important step in high resolution fingerprint recognition. This paper proposes a novel pore matching method with high recognition accuracy. The method mainly solves the pore representation problem in the state-of-the-art direct pore matching method. By making full use of the diversity and large quantities of sweat pores on fingerprints, deep convolutional networks are carefully designed to learn a deep feature (denoted as DeepPoreID) for each pore. The inter-class difference and intra-class similarity of pore patch pairs can be well solved using deep learning. The DeepPoreID is then used to describe the local feature for each pore and finally integrated into the classical direct pore matching method. More specifically, pore patches, which are cropped from both Query and Template fingerprint images, are imported into the well-trained networks to generate DeepPoreID for pore representation. The similarity between those DeepPoreIDs are then obtained by calculating the Euclidian Distance between them. Subsequently, one-to-many coarse pore correspondences are established via comparing their similarity. Finally, classical Weighted RANdom SAmple Consensus (WRANSAC) is employed to pick true pore correspondences from coarse ones. The experiments carried on the two public high resolution fingerprint database have shown the effectiveness of the proposed DeepPoreID, especially for fingerprint matching with small image size. Meanwhile, better recognition accuracy is achieved by the proposed method when compared with the existing state-of-theart methods. About 35% rise in equal error rate (EER) and about 30% rise in FMR10 0 0 when compared with the best result evaluated on the database with image size of 320 × 240 pixels. © 2020 Elsevier Ltd. All rights reserved.

1. Introduction Nowadays, automated fingerprint recognition systems (AFRSs) have been widely used in the field of security [1,2]. Recognition is usually done by matching features (e.g. minutiae, sweat pore, ridge contour) extracted from fingerprints rather than matching the fingerprint images directly. In general, there are three levels of fingerprint features, namely Level-1, Level-2 and Level-3 features. . Singular points and global ridge patterns describe the macro details, i.e. Level-1 features, of fingerprints. They are usually used for fingerprint indexing/classification since they are not very distinctive. The Level-2 features mainly refer to minutiae (i.e. ridge endings and bifurcations, as labeled in Fig. 1). Most AFRSs take minutiae feature for recognition since they are the most distinctive and stable features [3–5,26–28]. Level-3 features describe the dimensional attributes of the ridges. Sweat pores, ridge contours, and

∗

Corresponding author. E-mail address: [email protected] (L. Shen).

https://doi.org/10.1016/j.patcog.2020.107208 0031-3203/© 2020 Elsevier Ltd. All rights reserved.

ridge edge features are all Level-3 features. There are usually taken as additional features to support more accurate and robust fingerprint recognition. Specifically, the performance of minutiae based fingerprint recognition is limited by the small number of matched minutiae pairs if the overlapping region between fingerprint pairs is small. As the example shown in Fig. 1, there are only several minutiae pairs in the overlapping region framed by black rectangle, which has an effective fingerprint area of about 4.0 mm × 2.2 mm. It is difficult to judge that whether the fingerprint pair given in Fig. 1 is from the same finger or not by matching minutiae. Meanwhile, with the increasing demand for higher security levels, there is a pressing need to explore Level-3 features for recognition. Level-3 features are used for more accurate fingerprint matching and can achieve better performance due to the large quantities and accurate locations. Sweat pores, among various Level-3 features, have attracted the most attention [6–13,22–24,29–37]. As labeled in Fig. 1, sweat pore is a living feature and has proved to be discriminative to identify human identity [7]. While effective pore extraction methods have been proposed continuously

2

F. Liu, Y. Zhao and G. Liu et al. / Pattern Recognition 102 (2020) 107208

Fig. 1. An example of genuine fingerprint pairs with small overlapping area. The black rectangle indicates the overlapping region. Table 1 EERs of different pore matching methods in the literature on two public database with different image size. Method

Database EER (%) DBI with image size of 320 × 240 pixels

MINU in [14] DP in [9] SRDP in [10] TDSWR in [11]

17.67 15.42 6.59 3.25

DBII with image size of 640 × 480 pixels 0.72 7.05 0.97 0.53

[22–25,32–37], the research on pore matching are relatively limited [8–12,24]. Current pore matching approaches fall into two camps. One is pre-aligned approach [8], which aligns two fingerprint images before matching the pores in them. The classical align method in this category is a minutiae-based method proposed by Jain et al. [8]. They proposed to align two fingerprint images using minutiae. Then, those sweat pores near the aligned minutiae pairs are matched by a modified iterative closest point (ICP) algorithm. However, the performance relies heavily on the minutiae matching result. It requires a sufficient number of minutiae for effective alignment and considers only the pores in a small neighborhood of aligned minutiae. Another kind of pore matching method is the so called direct pore matching method [9–11]. Those methods match sweat pores directly in fingerprints. They usually follow a hierarchical coarseto-fine matching strategy. Those methods describe each pore by its local descriptor and establish one-to-one or one-to-many pore correspondences in the coarse step, then refine the correspondences by a global transformation algorithm in the fine step. In [9], Zhao et al. proposed to establish one-to-one pore correspondences based on the correlation between the local patches around the pores, and then refined pore correspondences by a RANdom Sample Consensus (RANSAC) algorithm. In [10,11], Liu et al. proposed a sparse representation-based matching method (SR) and the improved version (denoted by tangent TD-sparse) to determine the pore correspondences in the coarse step, then established one-tomany pore correspondences, and finally refined the coarse result by a weighted RANSAC (WRANSAC) algorithm. As summarized in Table 1, the equal error rates (EER) are relatively high when those direct pore matching methods were evaluated on the two public databases, especially for fingerprints with small image size of 320 × 240 pixels. The main reason for improvement of the method [12] over those methods in [9–11] is that the proposed TD-Sparse is robust to noises and distortions of pore patches. Thus, how to

better represent pore patches to establish more accurate coarse pore correspondence is critical for direct pore matching methods. Accurate coarse pore correspondences establishment in direct pore matching is a very challenging problem. Due to the elastic distortion of skin, the same pore patches from two images of the same finger captured at different time may look quite different, as the pore patches (green circled) shown in Fig. 1. In this situation, it is very easy to miss true pore correspondence. Meanwhile, different pore patches from different region of two images of the same finger can look very similar when the pore patches mainly consists of parallel ridges, as the example (red circled) shown in Fig. 1. In this case, false pore correspondence will be established. These mistakes will heavily affect the final matching accuracy. As summarized in Table 1, the equal error rates (EER) are relatively high when those direct pore matching methods were evaluated on the two public databases, especially for fingerprints with small image size, i.e. 320 × 240 pixels. Thus, it is very important to propose an effective pore representation approach to solve the existing problems in direct pore matching methods. In this paper, we propose a new deep feature descriptor, namely DeepPoreID, for pore representation in direct pore matching. A coarse-to-fine pore matching strategy similar to DP [9] is adopted to match fingerprints. However, we used the deep feature learned from a well-trained convolutional neural networks (CNNs) to describe the local feature of each pore. This idea is inspired by the good performance of DeepID [15] in face recognition domain. To fully explore the diversity and large quantities of sweat pores on fingerprints, we converted the pore representation into a classification problem. Then, four DeepPoreID networks were designed and trained to learn a highly compact 128-dimensional high-level pore features. Experiments carried on the database of DBI with image size of 320 × 240 pixels showed that the proposed method outperforms other state-of-the-art methods, in terms of recognition accuracy. The experiment showed a 32% rise in accuracy compared with the best result achieved in [12]. The rest of this paper is organized as follows. Section 2 presents the studies on pore representation and the overviews of CNNs briefly. Section 3 introduces the proposed pore representation and matching method. Section 4 reports and analyzes the experimental results. Section 5 concludes the paper. 2. Related works 2.1. Pore representation There are lots of research works about the representation of sweat pores [9–12,16]. Most of researches used gradients in the

F. Liu, Y. Zhao and G. Liu et al. / Pattern Recognition 102 (2020) 107208

local neighborhood of each pore to represent pore feature. Jain et al. [16] proposed to use a set of Gabor filters to generate a vector to describe the ridges pattern around the sweat pore. DP method [9] firstly used the Gaussian filter to smooth the fingerprint image to remove noise, and then cropped a circular neighborhood area for each pore and rotated the cropped region along ridge orientation to finally form the descriptor. The feature vector is normalized to have zero mean and unit length. However, these representation are sensitive to noise and distortion. In order to alleviate the impact of noise and distortion, Liu et al. proposed to calculate distance of pore descriptor instead of simple intensity-correlationbased method. SRDP method [10] proposed to use sparse representation technique to calculate the difference between pores instead of correlation-based technique. TDSWR method [11] used tangent distance (TD) instead of Euclidean distance (ED) to calculate the similarity of pores. Combined with sparse representation, TD performed better than ED because TD considered the rotation and distortion of pore descriptor. Furthermore, to prevent missing the correct pore correspondences, one-to-many pore correspondences were established in the coarse matching step. By comparing those methods, it can be seen that a better pore descriptor robust to deformation of same pores (green circled in Fig. 1) and tolerant to similar of different pores (red circled in Fig. 1) will largely improve the performance of pore matching.

3

trained in an offline training stage. Then, the well-trained DeepPoreID networks are used to generate pore representation, namely DeepPoreID (i.e. P in Fig. 2), for each pore in both Query and Template fingerprint images. The similarity (i.e. S in Fig. 2) between those DeepPoreIDs are achieved via calculating the distance between them. One-to-many coarse pore correspondences (i.e. C in Fig. 2) are established by comparing their similarity. Finally, WRANSAC [26] is employed to pick true pore correspondences from coarse ones. The proposed method is then presented in two parts, i.e. offline DeepPoreID network training and pore matching using DeepPoreID, as given in Fig. 2. 3.1. Offline DeepPoreID network training How to get effective DeepPoreID is the core issue in the proposed method. To make DeepPoreID robust to those intra-class differences caused by skin elasticity, image grayscale, image contrast and pattern noise, as well as the inter-class similarity due to similar local ridge structure, all of those variations are considered in our learning model. Similar to DeepID [15], our learning model is treated as a multi-class pore classification problem, namely to classify the identity of a pore image. We then introduce in details the training stage of DeepPoreID, including data preparation and augmentation, design of DeepPoreID networks, and training details in the following subsections.

2.2. CNNs and DeepID In recent years, CNNs have gained popularity in many computer vision applications such as classification [17,18], object detection [19,20], etc. CNNs have large learning capacity and can be controlled by varying their depth and breadth. The CNNs can use several filters to extract the most distinctive features from images without any prior knowledge. The weights of filters can be learned automatically from training data through back propagation. With a large quantities of training data, CNNs can extract discriminative features, fit the object function and achieve high accuracy in different tasks. Meanwhile, CNNs are robust to gray variation, noises and a certain degree of deformation. For these reasons, CNNs are widely used to extract features for biometrics, such as face [15], iris [21], and fingerprint [22–24]. Among these works, DeepID based methods [15] achieved great performance in face recognition domain. This kind of method firstly trained CNNs to classify different faces, then verified test face using Joint Bayesian technique or trained neural network by comparing the extracted deep feature (denoted as DeepID) with the trained features belongs to different faces. The experimental results show that DeepID can describe features better and more accurate than traditional descriptors (e.g. Gabor based descriptor, gradient based descriptor), so as to achieve good recognition accuracy in human level. Experiments also indicate that concatenating the representation from different networks trained independently will help to achieve better performance. Based on above analysis, we found deep feature learned from well-trained CNNs may be a good choice to represent local pore feature to solve the problems in current direct pore matching methods. Since sweat pores on fingerprints have the characteristics of large quantity, they fit the framework of DeepID very well. This paper thus tried to design DeepID-like CNNs to learn a deep pore feature. 3. The proposed direct pore matching method The proposed pore matching method follows a hierarchical coarse-to-fine strategy, which is a typical direct pore matching method. Fig. 2 gives the scheme of the proposed method. Firstly, The DeepPoreID networks designed to extract pore feature are

3.1.1. Data preparation and augmentation Since our learning model is a classification task, the first step is to generate a large number of training data including pore patches from the same class and pore patches from different classes for learning. Benefit from the large quantity of sweat pores, the categories of pore patches can be very large. These pore patches both from different objects and different region of the same object can be taken as independent classes. Different K classes of pore patches were randomly selected from all patches cropped from different fingerprint images included in the public database. Those pores on fingerprint images were extracted by the method proposed by Zhao et al. [25]. It should be noted that classes of pore patches (i.e. K) set in this paper is large, since experiments given in DeepID showed that more robust deep feature can be obtained if the number of face classes is large. Due to the limited number of fingerprint images of the same object and the existence of deformation and noise in fingerprint images, data augmentation is necessary and an effective way to generate training data of pore patches for the same class. By data augmentation, enough data for the same class can be obtained to ensure the network converged. Meanwhile, the deformations including skin elasticity, image grayscale, image contrast, and pattern noise can be simulated in data augmentation so as to make the proposed DeepPoreID robust enough to intra-class differences. The data augmentation procedure includes the following five steps, where st-1 (x,y) is the input gray value and st (x,y) is the output gray value of the coordinates x, y, and t is the step number: 1. Grayscale transformation and contrast variation: this step changes pore patch grayscale and contrast randomly, where s1 min = [0.15, 0.35] and s1 max = [0.60, 1.00] are the upper and lower limits after grayscale conversion, γ = [0.65, 1.35] specifies the shape of the curve describing the relationship between the values of input and output, the purpose of this step is to simulate the different grayscale of different fingerprint images. As shown in Eq. (1),



s1 (x, y ) =

s0 (x, y ) − min(s0 (x, y )) max(s0 (x, y )) − min(s0 (x, y ))

×(s1 max − s1 min ) + s1 min

γ

(1)

4

F. Liu, Y. Zhao and G. Liu et al. / Pattern Recognition 102 (2020) 107208

Fig. 2. The flowchart of the proposed pore matching method.

2. Image rotation: this step rotates a pore patch randomly within an angle θ = [−20, 20]. The purpose of this step is to simulate the angle transformation introduced by fingerprint collection. As shown in Eq. (2) and Eq. (3), u, v is the input coordinates and the x, y is the output coordinates.

 



x cos θ = y sin θ

− sin θ cos θ

  u

(2)

v

s2 (x, y ) = s1 (u, v ) = s1 (x ∗ cosθ + y ∗ sinθ , y ∗ cosθ − x ∗ sinθ ) (3) 3. Affine transform: this step does affine transformation to a pore patch. Patches of the same pore can be different in two captured fingerprint images due to skin distortion. The purpose of this step is to simulate the situation of skin shrink. Of course, the transform parameters a11 = [0.8, 1.2], a12 = [−0.2, 0.2], a21 = [−0.2, 0.2], a22 = [0.8, 1.2] are selected randomly in the range. As shown in Eq. (4) and Eq. (5), u, v is the input coordinates and the x, y is the output coordinates.

  x y 1



=

a11 a21 0

a12 a22 0

b1 b1 1

  u

v

 s3 (x, y ) = s2 (u, v ) = s2

(4)

1 x ∗ a22 − y ∗ a12 + a12 b2 − a22 b1 , a11 a22 − a12 a21

y ∗ a11 − x ∗ a21 − a11 b2 − a21 b1 a11 a22 − a12 a21



(5)

4. Noise addition: this step adds noise into pore patches. As shown in Eq. (6), η(x,y) is the Gaussian noise function, both μ1 , μ2 = [-0.001, 0.001] and σ 1 2 , σ 2 2 = [0,0.001] are chosen

randomly in range.

s4 (x, y ) = s3 (x, y ) + η (x, y ) = s3 (x, y ) +

1 2 π σ1 σ1

e

−

(x−μ1 )2 2 σ1 2

−

(y−μ2 )2 2 σ2 2

(6) 5. Random cropping: this step crops images into smaller patches. Generally, pores cannot be located particularly accurate but a small bias (about a few pixels). The purpose of this step is to compensate the bias of pore location caused by automatic pore extraction method. As shown in Eq. (7), tx , ty = [0, 2] are the bias of shift value along two directions and chosen randomly.

s5 (x, y ) = s4 (x + tx , y + ty )

(7)

Fig. 3 shows the flowchart of the proposed data augmentation. Due to the fifth step in data augmentation, the original size of the pore patch should be bigger than the target training patch size. For example, if the size of training data is set to 15 × 15 pixels, the size of the original pore patch should be 17 × 17 pixels. An example of data augmentation is shown in Fig. 4. The top left corner is the original pore patches cropped from fingerprint image while others are generated by data augmentation. The generated pore patches is considered to have the same label as the original pore patch. In this paper, 100 pore patches are generated for each class of pore patch, 80 of them are used for training while the rest are used for validation. 3.1.2. DeepPoreID networks Network design is a central step in deep learning. Inspired by ResNet [18], an effective and popular model in deep learning domain in recent years, we designed our DeepPoreID network architecture. The network consists of a series of residual blocks, pooling layers, fully-connected layers, and finally a softmax output layer. The residual structure avoids gradient disappearance and gradient

F. Liu, Y. Zhao and G. Liu et al. / Pattern Recognition 102 (2020) 107208

5

Fig. 3. The flowchart of data augmentation. Data augmentation consists of five steps: (1) Grayscale transformation and contrast variation, (2) image rotation, (3) affine transform, (4) noise addition and (5) random cropping.

Fig. 4. Example of pore patches after data augmentation. The top left corner is the original pore patch while others are generated by data augmentation.

explosion in some way. The convolution filters in residual blocks help to extract and fuse the features from lower layers. Two fullyconnected layers are followed with final max pooling layer. The output of the second fully-connected layer in the network will be used as DeepPoreID. The goal of training is to classify pore patch classes correctly. Therefore, the output of the second fullyconnected layer is fed to the K-way softmax. Since different input scale of pore patches will lead to different network architecture, four kinds of input scale of pore patch, that are 15 × 15 pixels, 19 × 19 pixels, 25 × 25 pixels and 31 × 31 pixels, are considered and formed four kinds of DeepPoreID network architectures in the paper. Details of the DeepPoreID network architectures are illustrated in Table 2. Limited by the size of feature map, there are fewer layers of the network architectures with input scale of 15 × 15 pixels and 19 × 19 pixels than these with input scale of 25 × 25 pixels and 31 × 31 pixels. Furthermore, some of the pooling operations in different networks are different in order to better use the information in border. All layers, apart from the output softmax layer, are followed by a rectified linear unit (ReLU), and all of them share a standard set of hyper-parameters. Batch normalization is also applied in all convolutional layers. In order to prevent overfitting, both fully-connected layers use dropout with a ratio of 0.5. DeepPoreID is then learned with two supervisory signals. The Identification signal, shown in Eq. (8), is trained to minimize the cross-entropy loss in order to classify each pore patch into one of K different identities, where fi is the extracted DeepPoreID feature vector, t denotes the target class of fi , pˆ i is the predicted probability distribution and pi is the target probability distribution.

Ident ( fi , t ) = −

K  i=1

pi log pˆ i = − log pˆt

(8)

The Verification signal, shown in Eq. (9), is trained to reduce the distance of features extracted from pore patches with the same identity and increase the distance of features between different classes, where fi and fj are feature vectors of two compared pore patches, yij = 1 means that fi and fj are from the same identity. On the contrary, yij = -1 means that fi and fj are from different identity.

V eri f ( fi , f j , yi j ) =



2 1

fi − f j

2 1 2

2





max (0, m − fi − f j )

2

2

i f yi j = 1 i f yi j = −1

(9)

The gradient of fi and fj are shown in Eq. (10) and Eq. (11), where λ is set to 0.05 by experience.

∇ fi =

∂ V eri f ( fi , f j , yi j ) ∂ Ident ( fi , t ) +λ ∂ fi ∂ fi

(10)

∇ fj =

∂ V eri f ( fi , f j , yi j ) ∂ Ident ( f j , t ) +λ ∂ fj ∂ fj

(11)

3.1.3. Training details The training implementation details are stated as follows. The weights in each layer was initialized to obey a zero-mean Gaussian distribution with standard deviation of 0.01. Stochastic gradient descent (SGD) was adopted for optimization with 0.001 weight decay and 0.9 momentum. Mini-batch was set to 256. Initial learning-rate was set to 0.1 and was divided by 10 every 25,0 0 0 iterations. The max-iteration was set to 10 0,0 0 0, which means 10 0 0 0 0iters ×256training_data/iters Kclasses ×80training_data/classes

=

320 0 0 0 K

epochs for K classes. When the

loss function converged for a sufficient period, training can be stopped manually. Caffe framework was used to implement the network training and testing. Workstation’s CPU is 2.8 GHz RAM 32 GB and GPU is NVIDIA Quadro M50 0 0M 8 GB.

6

F. Liu, Y. Zhao and G. Liu et al. / Pattern Recognition 102 (2020) 107208 Table 2 Architectures of DeepPoreID networks. Four different networks corresponding to four kinds of input scales are designed. Building blocks are the residual building block. The shape and operations in blocks and the number of stacked blocks are listed in the table. Input scale of pore path 15 × 15 CNN

19 × 19 CNN

25 × 25 CNN

15 × 15 × 32

19 × 19 × 32

25 × 25 × 32

31 × 31 × 32

8 × 8 × 32

9 × 9 × 32 (3 × 3 pool, s = 2)

13 × 13 × 32

16 × 16 × 32 (3 × 3 pool, s = 2)

Conv2

⎣ 3 × 3, 16⎦ × 2

8 × 8 × 64

9 × 9 × 32

13 × 13 × 32

16 × 16 × 32

Pool 2

1 × 1, 64 2 × 2 max pool, stride 2

4 × 4 × 64

4 × 4 × 64 (3 × 3 pool, s = 2)

7 × 7 × 64

8 × 8 × 64

4 × 4 × 128

4 × 4 × 128

7 × 7 × 128

8 × 8 × 128

2 × 2 × 128

2 × 2 × 128

4 × 4 × 128

4 × 4 × 128

2 × 2 × 256

2 × 2 × 256

4 × 4 × 256

4 × 4 × 256

1 × 1 × 256

1 × 1 × 256

2 × 2 × 256

2 × 2 × 256

——

——

2 × 2 × 512

2 × 2 × 512

—— 1 × 1 × 512 1 × 1 × 128 1×1×K

——

1 × 1 × 512

1 × 1 × 512

layer name Conv1 Pool 1

building Res-blocks



3 × 3, 32 3 × 3, 32 2 × 2 max pool, stride 2

⎡

⎡ Conv3 Pool 3 Conv4 Pool 4 Conv5 Pool 5 FC 1 FC 2 Output



1 × 1, 16

⎤

1 × 1, 32

⎤

⎣ 3 × 3, 32 ⎦ × 2 1 × 1, 128 2 × 2 max ⎤ pool, stride 2 ⎡ 1 × 1, 64 ⎣ 3 × 3, 64 ⎦ × 2 1 × 1, 256 2 × 2 max ⎤ pool, stride 2 ⎡ 1 × 1, 128 ⎣3 × 3, 128⎦ × 2 1 × 1, 512 2 × 2 max pool, stride 2 256-d 128-d K – softmax

3.2. Pore matching using DeepPoreID Once the DeepPoreID networks are trained and verified at offline training stage, they will be used to describe the local feature of each pore on both Template and Query fingerprint images in the proposed matching method. As illustrated in Fig. 2, the matching procedure follows the hierarchical coarse-to-fine strategy in direct pore matching. More specifically, in the coarse pore matching step, a pore patch will be cropped around each pore and will input into the trained networks to get its DeepPoreID feature vector with a length of 128 (i.e P in Fig. 2). Euclidean Distance was used to measure the distance (i.e. dij ) between DeepPoreID feature pairs {(P1i , P2j ), i = 1,2,…,N; j = 1,2,…,M}, where M, N is the number of pores in the Template and Query fingerprint images. The inverse of the distance corresponding to all DeepPoreID feature pairs formed a similarity matrix S, as formulated in Eq. (12). One-to-many coarse pore correspondences were then established based on S using the following strategy:



 

1

S = Si j =

1 di j

di j = 0 otherwise

i = 1, 2, · · · , M; j = 1, 2, · · · , N (12)

1. for each pore in the Template fingerprint image, the most similar pore in the Query fingerprint image was picked out based on Sij , as formulated in Eq. (13);





Smax i =

max

{ S i j } , i = 1 , 2 , · · · ,M

j=1,2,···,N

(13)

2. the average of these similarity values was computed as a threshold;

M S=

Smax i M

i=1

(14)

31 × 31 CNN

3. these pore pairs whose similarity value is larger than Sin matrix S is selected to form the final coarse pore correspondence list C = {C1 , C2 , · · · , CL }, where L is the number of selected pore pairs. In the fine pore matching step, the WRANSAC algorithm [26] was adopted to refine the coarse pore correspondences. All of the pore pairs in C have a weight, which is the similarity value calculated by Eq. (12). The larger the weight is, the more likely the corresponding pore pair is chosen to estimate the parameters of affine transformation model in WRANSAC. Finally, those pore pairs chosen by WRANSAC are taken as the true pore correspondences. 4. Experimental results and analysis In order to evaluate the effectiveness of the proposed DeepPoreID, as well as the performance of the matching method, a series of experiments were organized. In this section, the database and protocols was given at first, then the effectiveness of DeepPoreID was evaluated and analyzed. Finally, the matching results and performance was presented and discussed. 4.1. Database and protocols In the experiment, two databases (denotes as DBI and DBII) of high-resolution (~ 1200 dpi) fingerprint images were used for evaluation, which contain 1,480 fingerprints from 148 different fingers. Each finger is collected five times in each of two sessions separated by about two weeks. The only difference of DBI and DBII is the spatial size of fingerprint image. DBI has fingerprint images with a spatial size of 320 pixels by 240 pixels, while DBII has the same fingerprint images but with a larger image size of 640 pixels by 480 pixels. It is noted that these two databases are the same as those used in [9–12]. In DeepPoreID feature learning stage, the class number K was determined by the quantity of pore patches we selected. Due to

Vali.

0.2106 0.1478 0.0333 0.0283 0.2044 0.1442 0.0322 0.0275

Train Vali.

100 100 100 100 100 100 100 100

Train Vali.

99.93 99.98 100 100 99.94 99.98 100 100

Train Vali.

0.1139 0.0769 0.0222 0.0159 0.0769 0.0745 0.0213 0.0153

Train Vali.

100 100 100 100 100 100 100 100

Train Vali.

99.95 100 100 100 99.99 100 100 100

Train Vali.

0.1106 0.1077 0.0123 0.0090 0.1053 0.1043 0.0117 0.0087

Train Vali.

100 100 100 100 100 100 100 100

Train

99.94 99.98 100 100

Vali. Train

99.97 99.99 100 100 15 19 25 31 × × × × 15 19 25 31

loss top-5 20,0 0 0 classes

top-1 loss top-5

15,0 0 0 classes

top-1 loss top-5

10,0 0 0 classes

Effective DeepPoreID could be achieved only if the proposed networks converge well after training. This paper thus tested the proposed DeepPoreID networks by training data and validation data to show their performance. Because the pore class numbers were set to optional from 10,0 0 0, 15,0 0 0 and 20,0 0 0 in this paper, there are twelve models when combined with the proposed four DeepPoreID networks (i.e. networks with input scale of 15 × 15 pixels, 19 × 19 pixels, 25 × 25 pixels and 31 × 31 pixels). From the results given in Table 3, we can see that each network achieves top-5 accuracy of 100% on both training set (i.e. Train in Table 3) and validation set (i.e. Vali. in Table 3). The top-1 accuracy of all twelve models are all less than one in a thousand. Results fully demonstrated that the models fit their object functions well and were discriminative to classify pore correctly. The small numerical difference of value from training set and validation set also proved that our models were not overfitting on the training data. Thus, it convinced that the DeepPoreID generated by the proposed models will perform well for pore representation. The results also showed that the input scale of pore patch and the class number have influence on the performance of DeepPoreID at the training stage. With the same class number, when the input scale of pore patch increases, the identification accuracy became higher and the loss value declined. However, when the input scale of pore patch remains unchanged, the identification accuracy had little changes and the loss value seems irregular but kept a small change when the class number changed. This paper thus analyzed the matching performance based on the proposed twelve models and their combinations in the subsection.

top-1

4.2. Effectiveness validation of the proposed DeepPoreID

7

Input scale (pixels)

the lack of the open source and authoritative high resolution fingerprint database, different pore patches are cropped from 148 different fingerprints in the first session of DBI. In the experiments, 10,0 0 0, 15,0 0 0 and 20,0 0 0 of class number K were tested to evaluate the influence of class number K on model training, and accounted for ~5%, ~7.5%, and ~10% of the total amount of the pore patch in the DBI database. Since 100 pore patches of each class were generated after data augmentation, the total amount of data are 1,0 0 0,0 0 0, 1,50 0,0 0 0 and 2,0 0 0,0 0 0 for 10,0 0 0, 15,0 0 0 and 20,0 0 0 of class number, respectively. We randomly chose 80% of the pore patches in each class (i.e. 80 0,0 0 0, 1,20 0,0 0 0 and 1,60 0,0 0 0 for different K settings) for training and the rest 20% for validation. Top-1, Top-5 identification accuracy and the loss value were adopted to show the performance. Same to other direct pore matching methods, minutiae on the fingerprint images were extracted by using the algorithm introduced in [14], pores were extracted using the method proposed in [25]. The equal error rates (EER) and FMR10 0 0 were used to evaluate the recognition accuracy based on the following matches. 3700 genuine matches are generated by matching each fingerprint image in the first session with all five images of the same finger in the second session. 21,756 imposter matches are obtained by comparing the first fingerprint image of each finger in the second session with the first fingerprint image of all the other fingers in the first session. This setting is also the same as that adopted by other direct pore matching methods, including DP [9], SRDP [10], TDSWR [11], and muti-feature method [12]. The minutiae matching results are also included in our experiments. The minutiae were extracted and matched by using the method in [14] (denoted by MINU). It should be noted that all of the original training data for DeepPoreID learning are from the first session of DBI, the pores in the above matches will not appear in training data simultaneously.

Table 3 Top-1, Top-5 classify accuracy (%) and loss value of designed twelve different models for four input scales and three kinds of class number. Both training (Train) and validation (Vali.) data are tested.

F. Liu, Y. Zhao and G. Liu et al. / Pattern Recognition 102 (2020) 107208

8

F. Liu, Y. Zhao and G. Liu et al. / Pattern Recognition 102 (2020) 107208 Table 4 The EERs of the proposed method with twelve DeepPoreID models and two kinds of combination evaluated on DBI. Class number (K) Input scale (pixels) 15 19 25 31 ∗

× × × ×

15 19 25 31

10, 0 0 0

15, 0 0 0

20, 0 0 0

∗

10.37 7.35 3.80 1.46 1.36

9.45 6.64 3.34 1.42 1.25

8.76 6.07 2.86 1.50 1.39

8.60 5.88 2.52 1.18 N/A

Table 5 The EERs of different fingerprint matching methods on two databases. Method

DBI

DBII

MINU[14] MICPP[8] DP[9] Multi-feature + DP [12] Sparse Representation + DP[10] TDSWR [11] Multi-feature + TDSWR[12] AFMM-G (with pores) [24] AFMM-B (with pores) [24] Our method with best results (single model) Our method with best results + TDSWR Our method with best results (single model) + Multi-feature

17.67 30.45 15.42 6.99 6.59 3.25 2.17 4.67 3.66 1.42 1.13 —

0.72 7.83 7.05 5.93 0.97 0.53 0.17 — — 0.51 0.20 0.16

Fig. 5. Fingerprint recognition accuracy in DBI of the proposed method with twelve DeepPoreID models.

4.3. Performance analysis of the proposed method Since different input scale and class number contribute to different DeepPoreID models, this paper evaluated and analyzed the influence of these factors to the performance of the proposed pore matching method. Table 4 and Fig. 5 give the matching results based on these twelve different DeepPoreID models and their combinations. As the EERs shown in Table 4, better results were achieved by DeepPoreID network with large input scale of pore patch, while DeepPoreID model with appropriate class number like 15,0 0 0 can lead to the best result. It also can be seen from the result that an effective DeepPoreID network should be designed with large input scale of pore patch. However, as the curves shown in Fig. 5, the recognition accuracy increases slowly from input scale of 25 × 25 pixels to 31 × 31 pixels, but the enlargement of input scale will dramatically increase the amount of calculation. The class number changed from 10,0 0 0 to 20,0 0 0 has little impact on the recognition accuracy and tends to be stable when the input scale of pore patch is big enough. Thus, DeepPoreID network with input scale of 31 × 31 pixels and class number of 15,0 0 0, which lead to an EER of 1.42%, achieves the best result for fingerprint pore matching using single model. Since works have shown that better results will be obtained if different networks are combined for classification [17,18], two combinations of deep features obtained from different models were tried in the experiments. One is concatenating the deep features with different input scale but same class number to form the final DeepPoreID feature. The other one is concatenating the deep features with different class number but same input scale to form the final DeepPoreID feature. The results are shown in Table 4. Fig. 6 also shows the ROC curves of the results in the second combination (i.e. 15_∗ , 19_∗ , 25_∗ and 31_∗ ) in DBI, where “∗ ” denotes combination. For example, “15_∗ ” means that the final DeepPoreID feature is formed by concatenating three deep features trained

Fig. 6. ROC curves of the proposed method with different concatenated DeepPoreID features in DBI.

by network with input scale of 15 × 15 pixels and class number of 10,0 0 0, 15,0 0 0 and 20,0 0 0 separately. From the result, we can see that the results are all improved after model combination. The best EER is 1.18%, which is achieved by concatenating deep features generated by three models with input scale of 31 × 31 pixels and class number of 10,0 0 0, 15,0 0 0 and 20,0 0 0 separately. 4.4. Comparison with other methods The proposed method was compared with other existing pore matching methods, including MICPP in [8], DP in [9], AFMM-G (with pores) and AFMM-B (with pores) in [24], Sparse Representation (denotes as SR) in [10], TDSWR in [11], and Multi-feature + DP or Multi-feature + TDSWR in [12] on two databases. Here, “Multifeature” means that not only sweat pores, but also singular points and minutiae features are considered. Similar to sweat pores, the features of both singular points and minutia are taken as point features for matching. MICPP is a pre-aligned pore-based matching method we’ve mentioned in the introduction part. AFMM-G (with pores) and AFMM-B (with pores) are both Fourier transform based pore matching methods. A fusion of pore match scores obtained by TDSWR and the proposed method was also given using the weighted sum rule as in [9]. Different weights from 0.01 to 1 with an interval as 0.01 were tested and the best EER was reported. The EERs are given in Table 5 and the FMR10 0 0s are also shown in Table 6. It can be seen that the EER of the best results in single model is improved about ~35% on DBI, when compared with the Muti-feature + TDSWR method. Meanwhile, the FMR10 0 0 is improved about ~30% compared with TDSWR when evaluated on DBI. EERs are further improved if the proposed method is fused with TDSWR both in DBI and DBII. The ROC curves of TDSWR and

F. Liu, Y. Zhao and G. Liu et al. / Pattern Recognition 102 (2020) 107208

9

Fig. 7. Matching results of four genuine fingerprint pairs using the proposed method and TDSWR [11]: left–TDSWR, right–the proposed method with the best model.

Table 6 The FMR10 0 0 of different pore matching methods on two databases. Method

DBI

DBII

TDSWR [11] Our method with best results

11.07 7.69

4.07 4.05

the proposed method are given in Fig. 9. The results show the huge improvement of the proposed method to match partial fingerprint images with small size. Fig. 7 gives matching results of four genuine fingerprint pairs using the proposed method with the best training model, as well as matching result using TDSWR. It can be seen that dozens of true pore correspondences are established using the proposed DeepPoreID while a few pairs exist using TDSWR.

The Fig. 7 (a) is the example pairs which mentioned in Fig. 1. Besides, Fig. 8 shows the matching results of four imposter fingerprint pairs, which have similar fingerprint ridge flow in global and ridge parallel in partial area. It can be seen that only a few false correspondences are established using the proposed method while a dozens of false pore pairs are established by TDSWR. All of the examples shown in Fig. 7 and Fig. 8 further demonstrate the good performance of the proposed method to handle the inter-class difference and intra-class similarity of pore pair establishment in direct pore matching method. On DBII, our best result with single model of EER is 0.51% and FMR10 0 0 is 4.05% when matching pores only. The ROC curves of TDSWR and the proposed method are given in Fig. 10. An EER of 0.16% can be achieved if other fingerprint features are included for matching. The results showed the improvement of the proposed

10

F. Liu, Y. Zhao and G. Liu et al. / Pattern Recognition 102 (2020) 107208

Fig. 8. Matching results of four imposter fingerprint pairs using the proposed method and TDSWR [11]: left–TDSWR, right–the proposed method with the best model.

Fig. 9. ROC curves of proposed method and TDSWR in DBI.

Fig. 10. ROC curves of proposed method and TDSWR in DBII.

F. Liu, Y. Zhao and G. Liu et al. / Pattern Recognition 102 (2020) 107208

11

Table 7 Time cost comparison of coarse pore establishment evaluated on two databases. Method

DBI

DBII

TDSWR [11] The proposed method with model of input scale 15 × 15 pixels The proposed method with model of input scale 31 × 31 pixels

18.63s 2.80s 3.40s

71.82s 12.86s 15.91s

method compared with the best one currently, but the improvement of the proposed method is not obvious. This may be because large amount of coarse pore correspondences will be established if a large overlapping region is offered by two compared fingerprints. The large population of coarse pore correspondences compensates the limit of poor pore representation. The proposed method has lower time cost than TDSWR method since the DeepPoreID network is trained offline. Based on the computer configuration mentioned in Subsection 3.1.3, when using single model it takes about 2.80 s and 3.40 s using the proposed method with input scales of 15 × 15 pixels and 31 × 31 pixels for coarse pore correspondences establishment on DBI, while it takes about 12.86 s and 15.91 s respectively on DBII. For TDSWR, they cropped pore patch with size of 31 × 31 pixels for coarse pore correspondences establishment, it takes about 18.63 s and 71.82 s evaluated on DBI and DBII, respectively. The time cost comparison of the proposed method with TDSWR for coarse pore correspondences establishment evaluated on DBI and DBII are also listed in Table 7. Since the step of refinement is similar to TDSWR, the time cost of the proposed method and TDSWR for correspondences refinement is almost equal and is about 6.78 s and 7.12 s on DBI and DBII, respectively. It can be seen from the table that it is time consuming to match fingerprints with large image size using the direct pore matching methods. Since such methods describe pore features one by one for pore representation in the coarse matching stage, the time cost will be very high if the pore population is large. To make the method applicable, one solution is to reduce the pore population or image size. However, the recognition accuracy will be affected when the image size is small, as the EERs shown in Table 5. Fortunately, our proposed method made a tradeoff between recognition accuracy and time cost, when evaluated on DBI with image size of 320 × 240 pixels. On the whole, the proposed method outperforms other existing direct pore matching methods in EER, FMR10 0 0 and time consumption, especially matching partial fingerprint images with small image size. 5. Conclusions This paper proposed a new direct pore matching method using deep feature. Different from previous direct pore matching methods, we proposed to learn a deep feature, namely DeepPoreID, for each pore to represent pore patch more effectively for matching. Experiments show that the proposed DeepPoreID is a very effective descriptor since both of inter-class difference and intraclass similarity of pore patches are taken into account in deep network learning. The experimental results also show the effectiveness of the proposed matching method. Many pore correspondences could be obtained when matching fingerprint pairs with very small overlapping region, which proved the powerful capability of the proposed method to solve partial fingerprint recognition problem. When compared with other direct pore matching methods, the proposed method achieved higher accuracy than others on both EER and FMR10 0 0. The code of the proposed method is available at https://github.com/CVI-lzl/PoreCorrespond. However, the computational complexity of the proposed method is still high since it follows the classical coarse-to-fine strategy of direct pore matching method.

In the light of the outstanding performance of our deep learning based method, in the future work, we consider to learn more robust deep features and classify fingerprint images directly using a group-classification network [38–40], so as to balance the effectiveness and efficiency of pore matching method. We hope the method can not only keep the recognition accuracy but also improve the time efficiency by generating more pore representations and identifying group of pores in a fingerprint image. Declaration of Competing Interests The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgements The work is partially supported by Natural Science Foundation of China (61672357, 61703283 and U1713214), the Shenzhen Fundamental Research fund (JCYJ20180305125822769, JCYJ20160331185006518, JCYJ20160530141902978), the Tencent “Rhinoceros Birds”-Scientific Research Foundation for Young Teachers of Shenzhen University, and the Science and Technology Funding of Guangdong Province (2018A050501014). References [1] D. Maltoni, D. Maio, A.K. Jain, S. Prabhakar, Handbook of Fingerprint Recognition, Springer, New York, 2003. [2] N. Ratha, R. Bolle, Automatic Fingerprint Recognition Systems, Springer, New York, 2004. [3] D. Maio, D. Maltoni, Direct gray-scale minutiae detection in fingerprints, IEEE Trans. Pattern Anal. Mach. Intell. 19 (1) (1997) 27–40. [4] J. Khodadoust, A. Khodadoust, Fingerprint indexing based on minutiae pairs and convex core point, Pattern Recognit. 67 (2017) 110–126. [5] K. Cao, A.K. Jain, Automated latent fingerprint recognition, IEEE Trans. Pattern Anal. Mach. Intell. 99 (2018) 1-1. [6] M. Lu, Z. Chen, W. Sheng, A pore-based method for fingerprint liveness detection, Proceeding of the IEEE International Conference on Computer Science and Applications (ICCSA), 2017. [7] CDEFFS, Data format for the interchange of extended fingerprint and palmprint features,Working Draft Version0.4, http://fingerprint.nist.gov/tandard/ cdeffs/index.html. [8] A.K. Jain, Y. Chen, M. Demirkus, Pores and ridges: High-resolution fingerprint matching using level 3 features, IEEE Trans. Pattern Anal. Mach. Intell. 29 (1) (2007) 15–27. [9] Q. Zhao, L. Zhang, D. Zhang, N. Luo, Direct pore matching for fingerprint recognition, in: Proceedings of International Conference on Biometrics (ICB), 2009, pp. 597–606. [10] F. Liu, Q. Zhao, L. Zhang, D. Zhang, Fingerprint pore matching based on sparse representation, in: Proceedings of the 20th IEEE International Conference on Pattern Recognition (ICPR), 2010. [11] F. Liu, Q. Zhao, D. Zhang, A novel hierarchical fingerprint matching approach, Pattern Recognit. 44 (8) (2011) 1604–1613. [12] F. Liu, Y. Zhao, L. Shen, Feature Guided Fingerprint Pore Matching, in: Proceedings of the 12nd Chinese Conference on Biometric Recognition (CCBR), 2017, pp. 334–343. [13] K. Nguyen, C. Fookes, S. Sridharan, M. Tistarelli, M. Nixon, Super-resolution for biometrics: A comprehensive survey, Pattern Recognit. 78 (2018) 23–42. [14] J. Feng, Combining minutiae descriptors for fingerprint matching, Pattern Recognit. 41 (1) (2008) 342–352. [15] Y. Sun, D. Liang, X. Wang, X. Tang, DeepID3: Face recognition with very deep neural networks, in: Proceedings of the 21th International Conference on Multimedia Modeling (MM), 2015. [16] A.K. Jain, L. Hong, R. Bolle, On-line Fingerprint Verification, IEEE Trans. Pattern Anal. Mach. Intell. 19 (4) (1997) 596–600.

12

F. Liu, Y. Zhao and G. Liu et al. / Pattern Recognition 102 (2020) 107208

[17] A. Krizhevsky, I. Sutskever, G. Hinton, ImageNet classification with deep convolutional neural networks, in: Proceedings of the 26th Annual Conference on Neural Information Processing Systems (NIPS), 2012. [18] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. [19] Y. Li, S. Wang, Q. Tian, X. Ding, Feature representation for statistical-learning-based object detection: A review, Pattern Recognit. 48 (11) (2015) 3542–3559. [20] K. He, G. Gkioxari, P. Dollár P, R. Girshick, R-CNN Mask, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. [21] N. Liu, H. Li, M. Zhang, J. Liu, Z. Sun, T. Tan, Accurate iris segmentation in non– cooperative environments using fully convolutional networks, in: Proceedings of International Conference on Biometrics (ICB), 2016, pp. 1–8. [22] R. Labati, A. Genovese, E. Munoz, V. Piuri, F. Scotti, A novel pore extraction method for heterogeneous fingerprint images using convolutional neural networks, Pattern Recognit. Lett. (2017). [23] H. Jang, D. Kim, S. Mun, H. Lee, DeepPore: fingerprint pore extraction using deep convolutional neural networks, IEEE Signal Process Lett. 24 (12) (2017) 1808–1812. [24] H. Su, K. Chen, W. Wong, S. Lai, A deep learning approach towards pore extraction for high-resolution fingerprint recognition, in: Proceedings of 42nd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017. [25] Q. Zhao, D. Zhang, L. Zhang, N. Luo, Adaptive fingerprint pore modeling and extraction, Pattern Recognit. 43 (8) (2010) 2833–2844. [26] D. Zhang, W. Wang, Q. Huang, S. Jiang, W. Gao, Matching images more efficiently with local descriptors, in: Proceedings of the 19th International Conference on Pattern Recognition (ICPR), 2008. [27] X. Jiang, W.Y. Yau, Fingerprint minutiae matching based on the local and global structures, in: Proceedings of the 15th International Conference on Pattern Recognition (ICPR), 20 0 0. [28] M. Tico, P. Kuosmanen, Fingerprint matching using an orientation-based minutia descriptor, IEEE Trans. Pattern Anal. Mach. Intell. 25 (8) (2003) 0–1014. [29] C. Yi, A.K. Jain, Dots and incipients: extended features for partial fingerprint matching, in: Proceedings of the Biometrics Symposium, 2008. [30] V. Lathajothi, S. Arumugam, High-resolution fingerprint matching using level 3 incipient ridges and scars, Int. J. Comp. Appl. 48 (8) (2013). [31] J.D. Stosz, L.A. Alyea, Automated system for fingerprint authentication using pores and ridge structure, in: Proceedings of the International Society for Optics and Photonics, 1994, pp. 210–224. [32] R.F. da Silva Teixeira, N.J. Leite, in: M. Kamel, A. Campilho (Eds.), On Adaptive Fingerprint Pore Extraction, 7950, Springer, Heidelberg, 2013, pp. 72–79, doi:10.1007/978- 3- 642- 39094- 4_9. ICIAR 2013. LNCS.

[33] Y. Xu, G. Lu, F. Liu, Y. Li, et al., in: J. Zhou, et al. (Eds.), Fingerprint Pore Extraction Based on Multi-Scale Morphology, 10568, Springer, Cham, 2017, pp. 288– 295, doi:10.1007/978- 3- 319- 69923- 3_31. CCBR 2017. LNCS. [34] R. Labati, A. Genovese, E. Muñoz, V. Piuri, F. Scotti, A novel pore extraction method for heterogeneous fingerprint images using convolutional neural networks, Pattern Recognit. Lett. (2017). [35] H. Wang, X. Yang, L. Ma, R. Liang, et al., in: J. Zhou, et al. (Eds.), Fingerprint Pore Extraction Using U-Net Based Fully Convolutional Network, 10568, Springer, Cham, 2017, pp. 279–287, doi:10.1007/978- 3- 319- 69923- 3_30. CCBR 2017. LNCS. [36] H. Jang, D. Kim, S. Mun, S. Choi, H. Lee, Deeppore: fingerprint pore extraction using deep convolutional neural networks, IEEE Signal Process. Lett 24 (12) (2017) 1808–1812. [37] Y. Zhao, F. Liu, L. Shen, Fingerprint pore extraction using convolutional neural networks and logical operation, Chinese Conference on Biometric Recognition, Springer, Cham, 2018. [38] W. Xie, X. Jia, L. Shen, M. Yang, Sparse deep feature learning for facial expression recognition, Pattern Recognit. 96 (2019) 106966. [39] A.T. Lopes, E. de Aguiar, A.F. De Souza, T. Oliveira-Santos, Facial expression recognition with convolutional neural networks: coping with few data and the training sample order, Pattern Recognit. 61 (2017) 610–628. [40] J. Gu, Z. Wang, J. Kuen, L. Ma, et al., Recent advances in convolutional neural networks, Pattern Recognit. 77 (2018) 354–377. Feng Liu is currently an assistant professor at School of Computer Science & Software Engineering, Shenzhen University. She received her Ph.D. degree from the Department of Computing at the Hong Kong Polytechnic University in 2014. Her research interests include pattern recognition and image processing, especially focus on fingerprints. Yuanhao Zhao is currently a M.Sc. student majoring pattern recognition at School of Computer Science & Software Engineering, Shenzhen University. He received the Bachelor degree in computer science from Shenzhen University in 2016. His research interest includes pattern recognition and image processing. Guojie Liu received his B.S. degree from School of Computer Science & Software Engineering, Shenzhen University. He now is a graduate student at Shenzhen University. His research interests include pattern recognition, biometrics and image processing. Linlin Shen is currently a professor at School of Computer Science & Software Engineering, Shenzhen University. He received his Ph.D. degree from University of Nottingham, UK in 2005. His research interest covers pattern recognition, medical image processing and deep learning.