Multi-directional local gradient descriptor: A new feature descriptor for face recognition

Multi-directional local gradient descriptor: A new feature descriptor for face recognition

Image and Vision Computing 83–84 (2019) 39–50 Contents lists available at ScienceDirect Image and Vision Computing journal homepage: www.elsevier.co...

3MB Sizes 0 Downloads 233 Views

Image and Vision Computing 83–84 (2019) 39–50

Contents lists available at ScienceDirect

Image and Vision Computing journal homepage: www.elsevier.com/locate/imavis

Multi-directional local gradient descriptor: A new feature descriptor for face recognition☆ Vishwanath C. Kagawade a,⁎, Shanmukhappa A. Angadi b a b

Department of Computer Applications, Basaveshwar Engineering College, Bagalkot, India Department of Computer Science and Engineering, Centre for Post Graduate Studies, VTU, Belagavi, India

a r t i c l e

i n f o

Article history: Received 2 May 2018 Received in revised form 19 November 2018 Accepted 6 February 2019 Available online 14 February 2019 Keywords: Gradient features Face representation Symbolic data objects Viola-Jones Binary pattern

a b s t r a c t The performance of the face recognition systems is vulnerable to occlusion, light and expression changes and such constraints need to be handled effectively in a robust face recognition system. This paper presents a new multi-directional local gradient descriptor (MLGD) method for face recognition based on local directional gradient features that exploit the edges/line information in multiple directions. The proposed technique exploits advantage of similarity of a face image in small blocks. The weighted gradient features of face images in different directions and zones are computed based on co-relation between pixel elements. These features referred to as multi-directional local gradient descriptor (MLGD), which capture adequate edge information by integrating different directional gradients. Further, the directional gradient features extracted through MLGD operator are represented as a symbolic data object. The face identification is carried out by using the symbolic object representation of test image and employing a symbolic similarity measure. The experimental results on AR (97.33%) and LFW (97.25%) benchmark face databases demonstrate that the symbolic data representation of the new directional gradient magnitude of face image significantly improves the recognition performance as compared to local gradient descriptors and other state-of-the-art methods. © 2019 Elsevier B.V. All rights reserved.

1. Introduction Face is one of the universal biometric trait available in all the human beings. Due to the non-intrusive nature and ease of acquisition, it has been widely used biometric trait for person identification/recognition. Its wide range of applications includes information security, smart card, law enforcement, access control, video surveillance, entertainment, cooperative user applications etc. [1]. In the last few years, various techniques have been proposed and developed for person identification/ recognition based on face biometric. There are many factors including variations in pose, illumination, facial expression and occlusions affecting the performance of such biometric systems [2]. Based on processing of face data they can be classified into global approaches and local approaches. The global approaches use entire face image data as input to the recognition system. A few such methods include Eigenfaces, Fisherfaces, Laplacianfaces, the local directional pattern (LDP) etc. [3–5]. The local approaches do not consider the whole face, but only some features or areas of the face for face identification/recognition task. Image Gradient Orientation (IGO) descriptors are used to describe local edge information of face images [6,7]. The IGO-based techniques ☆ This paper has been recommended for acceptance by Prof. S. Todorovic. ⁎ Corresponding author. E-mail address: [email protected] (V.C. Kagawade).

https://doi.org/10.1016/j.imavis.2019.02.001 0262-8856/© 2019 Elsevier B.V. All rights reserved.

are invariant to illumination changes. However, these techniques are sensitive to local variations such as expression and pose on face images. In order to provide importance to higher order statistical relations, local feature descriptors such as Gabor wavelets, Local Binary Patterns (LBP), Local Phase Quantization (LPQ) and Scale Invariant Feature Transformation (SIFT) have recently gained considerable attention [2,3,8–10]. The local feature descriptors are resilient to multiple variations on face images by enforcing spatial locality in both pixel and patch levels. Gabor wavelets are the most widely used local appearance-based facial descriptors to extract local information in various directions in face recognition. However, Gabor-based methods are computationally expensive and require too much memory space [11,12]. To improve the face recognition performance, the LBP and its variants and SIFT techniques operate on small blocks of face image. The LBP initially used for texture analysis, is fast to compute facial features and has also a low memory cost [9]. However though it is robust to monotonic illumination changes, it shows poor performance in the presence of non-monotonic illumination changes and noise [12]. There are many extensions of the original LBP operator, such as the complete LBP, the dominant LBP, local gradient hexa pattern (LGHP) [2], Multi-scale LBP, Multi-block LBP, Local directional gradient pattern (LDGP), local derivative pattern (LDP) [8], semantic pixel sets-based LBP (spsLBP) [2,9,11] and the like. Most of them have emphasized on obtaining a more robust coding strategy. The LPQ is another texture descriptor similar to LBP technique and is

40

V.C. Kagawade, S.A. Angadi / Image and Vision Computing 83–84 (2019) 39–50

Fig. 1. The block diagram of Symbolic Data Modeling Approach to Face Recognition.

insensitive to image blurring and efficient for both blurred and sharp images [10]. In order to discover local structures of face image in the gradient domain, gradients are computed from multiple directions. The extracted gradients are represented as a set of binary strings [3]. Recently, the gradient of adequate strength gained importance in a given task of classification [13]. To address issues of LBP and variation of the LBP, many multiple directional gradient descriptors are proposed [2,4,12,14]. LDP feature is extracted by estimating edge information in multiple directions at each reference pixel of the face image plane. Based on the directions, the Local tetra pattern (LTrP) extracts the features by estimating the relationship between the referenced pixel and its neighbors in horizontal and vertical directions [15]. Local vector pattern (LVP) is an extension over LTrP [2,16]. However, the technique ignores the relationships between pixels (which is essential for classification task). The LGHP measures the relationship between the reference pixel and its neighboring pixels at different directions and is able to estimate discriminative information that exists in the local neighborhood [2]. Traditional local gradient descriptor LVP captures the discriminating information along 0° and 45°, 45° and 90°, 90° and 135°, 135° and 180° [14], LAG operator computes gradient information along 0°, 45°, 90°, 135°, 225°, 270° and 315° [4] and LGHP descriptor estimates discriminating information across different angular widths 0°, 45°, 90° and 135° [2]. Multi-Level Dual-Cross Patterns (MDML-DCPs) encodes multi-level invariant textural structure characteristics of eyes, eyebrows, and lips and nose in the horizontal and vertical directions respectively. The technique also encodes textural structure information present in the end parts of facial components in the diagonal directions (π/4 and 3π/4) and the technique has achieved better performance compared to LBP, even when sub DCP of exactly the same time and memory cost [17]. However, there exist many other different directions to compute gradients in a face image block. Local derivate patterns extract the features based on the distribution of edges across different directions in an image. These observations have motivated us to introduce a novel multi-directional local gradient descriptor (MLGD) method to estimate gradient weight in multiple directions. The effective representation of extracted features plays a key role in the task of classification. In the past few years various active learning, machine learning and deep learning techniques have been proposed and developed to do such task. Many active learning techniques

consider both informativeness and representativeness sampling criteria for designing general frame work to improve the classification accuracy [18]. Such frameworks introduced three similarities such as similarity between the unlabeled data set and query sample, similarity between the labeled data set and query sample, and similarity between any two candidate query samples to handle sample discrepancy problem over large data set [18]. On other hand, machine learning (supervised/ unsupervised learning) techniques are able to handle large amount of labeled data during classification. The supervised learning techniques require large amount of labeled data for classification task (especially convolution neural network) compared to unsupervised learning techniques. Recently, an unsupervised learning technique called stacked convolution denoising auto-encoder has been introduced for effective feature representation [18,19]. In which high dimensional raw image data are transformed into high level feature representations, which will boost the performance of the classifier. Recent representation learning with convolution neural network provides an effective tool for face recognition [20]. The FaceNet approach based on deep convolutional network has achieved greater representational efficiency and compared to other deep networks-based face recognition techniques [21] and also reduces error reported for DeepFace by more than a factor of 7 [22]. The Trunk-Branch Ensemble CNN model (TBECNN) for video-based face recognition extracts features efficiently from face images and employed triplet loss function to enhance the discriminative power of the face feature representations learnt by TBECNN [20]. In traditional feature representation techniques, features are represented as single valued variable. Such representation may not be able to represent effectively variation of feature values of the samples of the same subject. To address this issue, in recent years a new feature representation technique called symbolic data modeling has been introduced [23–27]. The symbolic data modeling technique offers a formal methodology to represent variable feature values. Due to its effectiveness and flexibility to represent feature values as symbolic objects, it gains importance in a variety of applications including face recognition, postal address component labeling, document analysis, character recognition etc. Initially, the symbolic modeling approach was introduced for classification of various types of entities such as fat oil and microcomputers and the technique was able to automatically determine/identify the number of classes [23]. The approach is further extended for labeling

Fig. 2. The computation of facial feature Directional Gradient Magnitude (DGM).

V.C. Kagawade, S.A. Angadi / Image and Vision Computing 83–84 (2019) 39–50

of postal address components. The approach represents extracted postal address component as symbolic hoard object [24–28]. The symbolic similarity measure has been employed to determine the distance between trained symbolic objects and query symbolic object. The variant of Kohonen's self-organizing map (SOM), Symbolic SOM (S-SOM) is introduced [28]. The S-SOM competitive learning neural network is able to classify/determine the class of microcomputer data (microcomputer with different properties such RAM, ROM, Display and key are represented as symbolic object data type). A face recognition technique based on symbolic PCA is introduced [29]. In the work, facial features are represented as symbolic object. The reported work uses minimum number of features to achieve the same accuracy as compared to some of the existing methods mentioned in [29]. In order to overcome issues of symbolic PCA, symbolic LDA-based face recognition technique has been introduced [30]. The technique is capable of representing facial features in lower dimension compared to conventional PCA. The symbolic LDA plus PCA technique has been presented in order to represent class specific discriminative information in lower dimension space [31]. Even though experimental results are encouraging, methods are evaluated on Yale and/or ORL face databases (Yale and ORL are considered as less constrained databases compared to other databases like LFW, AR etc.). Recently, a symbolic data modeling approach to face recognition is introduced to show effectiveness of representing facial features as symbolic objects [32]. The method is able to represent Polar FFT features of face images effectively. The work also adapted symbolic similarity measure to estimate similarity between trained symbolic objects to test symbolic object. The effectiveness and flexibility to represent face feature made us to use symbolic data modeling approach to represent multidirectional local gradient magnitude obtained from face image. The symbolic similarity measure adapted in this work can be found in our recent works on face recognition [32,33]. The MLGD operator is based on the properties gained from the image gradient orientation representation [3,4]. Unlike LVP, LDP and LGHP local gradient descriptors, proposed MLGD descriptor is able to effectively capture discriminative information at different radial widths such as 15°, 25°, 45°, 90°, 180° and 225° along different angular widths 0°, 45°, 90°, 135°, 225°, 270° and 315°. The proposed approach computes face features based on relationships between pixels in the face image blocks and effectively represent these features through symbolic data modeling approach [23–42]. The approach is capable of detecting texture/edges information of face images at various directions. Extensive experiments on several benchmark face databases demonstrate the significant advantages of the MLGD-based descriptors over existing face descriptors. The MLGD feature with symbolic data representation obtains accuracy up to 97.33% and 97.25% on the widely-used AR and LFW databases respectively. The organization of rest of the paper is as follows; In Section 2, the proposed symbolic data modeling approach to face recognition is presented and the methodology employed for feature extraction and symbolic object representation is described. A modified form of the symbolic similarity measure employed for classification and is discussed in Section 3, further experimental results conducted on AR and LFW databases and performance analysis are brought out in Section 4. Section 5 concludes the work.

(x, y)

θ xy

Fig. 3. Directional change in the gradients of x and y.

detection and Pre-processing; 2.Symbolic Modeling and construction of symbolic knowledge base; 3. Symbolic Similarity Measure. In face detection and pre-processing stage, the face part is detected and cropped from the original image using the Viola–Jones algorithm [43]. The cropped face is then down sampled to 64p × 64p size of gray scale image. The symbolic modeling and construction of symbolic knowledge base stage is performed in two phases; feature extraction and symbolic data modeling of facial features. In feature extraction phase, the multidirectional local gradient facial feature is computed from cropped face images of N classes. Further, the extracted features are represented as symbolic objects through symbolic data modeling technique to construct symbolic knowledge base. The final step is symbolic similarity measure, in which probe symbolic object is compared against the reference symbolic objects stored in the symbolic knowledge base. The detailed implementation of the proposed technique has been discussed in subsequent sections.

2.1. Facial features The proposed methodology uses local correlation between neighboring pixels (elements) w.r.t. reference pixel in an image block for the computation of directional gradient magnitude. In this work,

2. Symbolic data modeling approach to face recognition In this work, a new appearance-based face recognition using symbolic modeling of local gradient features is proposed. The technique effectively computes texture information such as lines/edges. Initially, micro texture information is extracted from face images. The obtained facial features are then represented as a symbolic faces (symbolic objects). During testing, the similarity between reference symbolic objects to probe symbolic object is estimated using newly devised modified form of symbolic similarity technique. In general, the proposed methodology consists of three major steps as illustrated in Fig. 1: 1. Face

41

Fig. 4. The computation of binary pattern in k directions.

42

V.C. Kagawade, S.A. Angadi / Image and Vision Computing 83–84 (2019) 39–50

Fig. 5. Sub-blocks. a. Case1: α = {−7.5°,82.5°,172.5°,−97.5°} and β = {7.5°,97.5°,−172.5°,−82.5°} b. Case 2: α = {−22.5°, 67.5°,157.5°, −112.5°} and β = {22.5°, 112.5°, −157.5°, −67.5°} c. Case 3: α = {−12.5°, 77.5°, 167.5°, −102.5°} and β = {12.5°,102.5°, −167.5°, −77.5°} d. Case 4: α = {−45°, 135°} and β = {45°, −135°}, e. Case 5: α = {185°} and β = {−185°} f. Case 6: α = {−90.5°, 135°} and β = {90.5°, −135°}

initially the edge response values or gradient change on a gray scale face image is determined. In this process, from the input gray scale image horizontal and vertical gradient matrices H and V are computed respectively. Further, cardinal gradient matrix MAG and the gradient

Neutral

Smile

Sunglass

orientation θ are estimated using H and V matrices. Based on gradient orientation θ and specified range of angular orientation, the binary pattern matrix FBP is generated. Next, an enhanced facial descriptor called directional gradient DG is determined by convoluting FBP patterns and

Scarves

Fig. 6. Cropped sample images of AR database.

Discus

Illumination

V.C. Kagawade, S.A. Angadi / Image and Vision Computing 83–84 (2019) 39–50

43

Table 1 DGM values of images in Fig. 6. Feature

Neutral

Smile

Sunglass

Scarves

Expression

Illumination

DGM

1.490e+04

1.495e+04

1.696e+04

1.796e+04

1.532e+04

1.650e+04

gradient magnitude MAG. Finally, the total directional gradient magnitude DGM is estimated i.e. total edge response is computed at different directions along the empirically chosen values α and β. The entire process involved in the computation of directional gradient magnitude i.e. DGM is shown in Fig. 2. The detailed estimation/extraction of face feature DGM from face images is described in the following: Let Ѱ (x, y) be an n × n size gray scale image. Then, the gradient (difference in x values) of an image Ѱ (x, y) in horizontal directions can be defined as in Eq. (1): hxy ¼

∂Ψ ∂x

ð1Þ

The gradient change of an image Ѱ (x, y) at xy th pixel indicates change in intensity in horizontal direction. In general, gradient change of Ѱ (x, y) image for all pixels in horizontal direction can be expressed as:   H ¼ hxy ∀x ¼ 1 to n−1 and∀y ¼ 1 to n−1

ð2Þ

In general, cardinal gradient matrix MAG of a face image Ѱ (x, y) for all pixels can be computed using Eq. (5) and is expressed as in Eq. (6). h i MAG ¼ magxy ∀ x ¼ 1 to n−1 and ∀ y ¼ 1 to n−1

Further, the gradient orientation θxy at xyth pixel can be estimated as:   θxy ¼ tan−1 hxy ; vxy ∀x ¼ 1 to n−1 and∀ y ¼ 1 to n−1

vxy ¼

∂Ψ ∂y

ð3Þ

The gradient change of an image Ѱ (x, y) at xyth pixel indicates change in intensity in vertical direction. In general, gradient change of face image Ѱ (x, y) for all pixels in vertical direction can be expressed as:   V ¼ vxy ∀x ¼ 1 to n‐1 and∀y ¼ 1 to n‐1

ð4Þ

Now, cardinal gradient magnitude mag of Ѱ (x, y) at xyth pixel can be estimated as: magxy ¼

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 v2 xy þ h xy ∀x ¼ 1 to n−1 and∀y ¼ 1 to n−1

ð5Þ

ð7Þ

In general, gradient orientation matrix Θ of a face image Ѱ (x, y) for all pixels can be computed using Eq. (7) and is expressed in Eq. (8).   Θ ¼ θxy ∀x ¼ 1 to n−1 and ∀y ¼ 1 to n−1

ð8Þ

The estimation of gradient orientation θxy at each pixel location (x, y) is shown in Fig. 3, where ∀ θxyϵΘ. Further using Eq. (8), the binary pattern bpxy(k) in kth direction at xyth pixel can be estimated as defined in Eq. (9). ðkÞ

Similarly, the gradient (difference in y values) of a face image Ѱ(x, y) in vertical directions can be defined as in Eq. (3):

ð6Þ

bpxy ¼



1 if θxy ≥ α and θxy b β ∀x ¼ 1 to n−1 and∀y ¼ 1 to n−1 0 otherwise ð9Þ

where, α and β are empirically chosen specified range of angular orientation widths of an image block and k represents the four face image blocks in left, top, right and bottom directions i.e. k = 1 to 4. The computation of binary pattern bpxy(k) in k directions with a specified range of α and β of a face image plane is depicted in Fig. 4. Fig. 4(a) represents the gradient orientation of a face image plane in four directions and Fig. 4(b) represents computation of binary pattern in k directions. The gradient information present in an image block may vary at left, right, top, and bottom directions due to non uniform illumination, pose and expression changes and partial occlusion. The presence of edges/ lines/corners shows higher gradient response in some particular directions [12]. In order to take advantage of this, several algorithms have been proposed to calculate local gradient at different directions in the range of 0° to 90° or 0° to 180° or 0° to 315° [2,4,12,14]. Since variation on face image, edge/lines texture information show high or low

Fig. 7. Illustration of Symbolic representation of directional gradient magnitudes features a. Original sample images, b. corresponding cropped face images of AR and Assertion symbolic objects representation and (c) Hoard object representation.

44

V.C. Kagawade, S.A. Angadi / Image and Vision Computing 83–84 (2019) 39–50

Fig. 8. Phases of symbolic data modeling approach to face recognition.

response values at particular direction. The edge response values present in different directions on face image may not be equally important [12]. Hence in order to account gradient change at diffract directions, in the proposed approach face image block is divided into equal blocks in the range of θ(0, 2 π). The proposed multi-directional gradient calculation method can take advantage of likelihood of a face image in small blocks. The method begins segmentation of face part from original image. The segmented face image is subdivided into sub-blocks for further evaluation. The multi-

directional gradient calculation approach resolves problems associated with variation on face image in different directions. To know the most prominent directions and to evaluate the proposed algorithm the image block is divided into sub-blocks, some of the typical cases are shown in Fig. 5. The edge response values in different direction are calculated and integrated to compute total gradient weight of a face image. Unlike LVP, LDP and LGHP local gradient descriptors, the proposed multi-directional local gradient descriptor is able to capture discriminative information along different gradient directions. In the proposed

Fig. 9. Sample face images: (a) original sample images (AR), and (b) corresponding cropped images of AR database, (c) original Angelina Jolie images (LFW), and (d) corresponding Angelina Jolie cropped images of LFW database.

V.C. Kagawade, S.A. Angadi / Image and Vision Computing 83–84 (2019) 39–50

MLGD technique for face recognition, after thorough experiments the image plan is divided into equal blocks using empirically chosen α and β values in k directions in the range of θ(0, 2 π). In Fig. 5(a), image block is divided into four sub-blocks having 15°wide bins (Case 1). Case 2: Image block is divided into four sub-blocks having 45°-wide bins shown in Fig. 5(b). Case 3: Image block is divided into four sub-blocks having 25°-wide bins shown in Fig. 5(c). Case 4: Image block is divided into two sub-blocks having 90°-wide bins shown Fig. 5(d). Case 5: Image block is divided into two sub-blocks having 185°-wide bins shown in Fig. 5(e). Case 6: Image block is divided into two sub-blocks having 225°-wide bins shown in Fig. 5(f). As in Fig. 5, different cardinal gradient vectors are computed along different angular displacement at different directions in order to account distribution of gradient information in an image block. Gradient vectors whose direction is in one of the k cardinal direction can be labeled as a cardinal gradient vector. Now, using Eq. (9) prominent binary pattern matrix BP in kth direction can be determined as in Eq. (10). h i ðkÞ BPðkÞ ¼ bpxy

ð10Þ

The full binary pattern matrix FBP of all image blocks is computed by logically ORing the matrices computed from Eq. (10) i.e. BP1, BP2, BP3 and BP4, which can be formulated as: FBP ¼ BP 1 ∨BP 2 ∨BP 3 ∨BP 4

ð11Þ

where ∨ indicates ‘logical OR’ operation. The methodology utilizes spatial and orientational correlation. Hence directional gradient matrix DG is estimated by spatial convolution of matrices FBP and MAG as in Eq. (12). DG ¼ ðFBP:  MAGÞ

ð12Þ

45

Table 2 Performance of the proposed technique on AR and LFW databases. Different cases considered for computation of DGM feature

AR

LFW

Case 1 Case 2 Case 3 Case 4 Case 5 Case 6

95.08 95.25 88.92 95.92 96.17 97.33

96.91 95.12 88.43 97.07 97.25 96.17

Synthetic types [23,28,32]. An Assertion symbolic object is a collection of events (an event indicates a value-variable pair). A Hoard symbolic object is a collection of two or more Assertion objects and a Synthetic symbolic object is a collection of two or more Hoard objects [23]. Literature reveals that symbolic modeling of face features as symbolic objects can effectively represent various class of variations on face images [32,33]. In this work, initially face image is cropped from the original image using the Viola–Jones algorithm. The directional gradient magnitudes features are extracted from cropped image and are represented as Assertion object. Further, a Hoard object is constructed using Assertion objects belonging to the same class using maximum–minimum rule. The collection of such Hoard objects constitutes Synthetic object for entire face database. To illustrate the symbolic representation of facial features as Hoard object, few samples of AR database has been considered and shown in Fig. 7. Fig. 7(a) shows four sample images of AR database, which have variations in expression(neutral), occlusion(scarf and sunglass) and change in illumination(right side light on) [44], Fig. 7 (b) shows cropped face images and their corresponding Assertion object representation and Fig. 7(c) representation of Hoard object by considering maximum and minimum values of Assertion objects.

The directional gradient matrix DG represents high or low response values of face edges at particular direction depending upon gradient information present in that direction. Hence, in this work to estimate total gradient magnitude weights, all the directional gradient matrices obtained at different directions are integrated. Finally, the required facial descriptor; directional gradient magnitude DGM of a face image Ѱ at different directions is estimated as defined in Eq. (13). DGMðΨÞ ¼

n −1 n −1 X X

DG

ð13Þ

x¼1 y¼1

More edge/line/corner information represents higher variation on face image [13]. The proposed MLGD local descriptor considered all the neighboring pixels at particular direction to determine directional gradient magnitude using the Eq. (13). The method accumulates sufficient gradient information, which is essential for the development of robust and efficient face recognition system. So, the accumulation of directional gradient magnitudes (DGM) information is supposed to play vital role in face recognition in the presence of listed class of variations on face images. Such variations on face images can found in AR database. The computed DGM feature values and corresponding sample images which vary in expression, occlusion and non uniform illumination changes are shown in Fig. 6 and computed DGM values are listed in Table 1.

(a) ROC curve on ARdatabase

2.2. Symbolic data modeling of face features Rooted in the proposed approach, is the symbolic modeling of face features as symbolic objects. Symbolic objects are “unified” by means of relationship types, whereas in conventional data sets objects are “individualized” [23]. Symbolic objects are more complex than numeric data. Based on the complexity they can be of Assertion, Hoard and

(b) ROC curve LFW database Fig. 10. ROC curves on publically available AR and LFW databases.

46

V.C. Kagawade, S.A. Angadi / Image and Vision Computing 83–84 (2019) 39–50

3. Symbolic similarity analysis for face recognition

Table 3 Distribution of AR database in different sets. Image variations

Training set (session 1)

Test set (session 2)

1, 5–7 1–4 8 and 11 8 and 9 12 and 13

14, 18–20 14, 15–17 21 and 24 23–25 22 and 23

Illumination (S1) Expressions (S2) Occlusion (sunglass + scarf) (S3) Scarf with illumination (S4) Sunglass with illumination (S5)

The overall process of modeling of facial features as symbolic objects can be described as follows: Let face database consists of N number of face classes and each class consists of M number of samples. In general all face classes can be expressed as: U ¼ fΩi ; ∀i ¼ 1 to Ng

ð14Þ

where Ωi = {ψj; j = 1 to M}. Further, using Eq. (13) the directional gradient magnitudes of face images of ith class are computed and represented as: n    o SVðiÞ ¼ DGM Ψi1 ; DGM Ψi2 ; …; DGM ΨiM ∀i ¼ 1 to N

ð15Þ

Further, directional gradient magnitude features of M number of samples of ith class can be represented as weighted directional gradient WGD(i) (a symbolic hoard object), using maximum and minimum value of SV(i) as expressed in the Eq. (16). WGDðiÞ ¼

n

  o max SV ðiÞ ; min SV ðiÞ ∀i ¼ 1 to N

ð16Þ

In general, multi directional gradient magnitude features of M number samples of ith class can be expressed as: n o SOðiÞ ¼ SVðiÞ ; WGDðiÞ ∀i ¼ 1 to N

ð17Þ

where SV(i) is 1 × M vector of DGM(Ψij) values, for ∀j = 1 toM. Similarly, N number of face classes are represented as symbolic hoard objects to build symbolic knowledge base as described in Eq. (18) for classification of the test symbolic object in relation to trained symbolic objects. n o SKB ¼ WGDðiÞ ∀i ¼ 1 to N

ð18Þ

The symbolic knowledge-based SKB matrix represents facial features of N number of face classes and SKB ϵ RN×2. N represents number face image classes and the digit 2 represents maximum and minimum values of WGD i.e. interval values maximum and minimum. The SKB contain only essential information present in different directions, which carries higher intra class discriminating capacity to classify face images. The symbolic data modeling approach to face recognition not only increases the discriminative power among the face classes but also simplifies computational complexity significantly.

In literature it is found that the symbolic features type had been widely studied and based on different scales of measurements are classified into quantitative, qualitative and structured variables [23,28,32,33]. The similarity value between symbolic feature components can be computed by employing position, span and content measures [23,32,36]. The choice of such similarity measures depends upon the type of symbolic features that are considered for representing the object. Assume that, two symbolic objects A and B of feature components Ak, Bk of kth feature consist of feature vector of size n, then Sp(Ak, Bk) for “position” exists only when the symbolic feature component type is quantitative, Ss(Ak, Bk) for the “span” indicates the relative sizes of the feature values without referring to common parts between them, and Sc(Ak, Bk) for the “content” is a measure of the common parts between Ak and Bk feature values. According to Chidananda Gowda, the quantitative features type may represent continuous ratio, discrete, absolute and interval values and such features can be measured based on content [23]. Hence, the proposed modified form of content-based similarity measure is defined on the basis of content of symbolic objects. The procedure employed in this work for symbolic similarity measurement is described as follows: Let Pobj = {DGM} be a directional gradient magnitude feature extracted from the probe face image Ѱprob SO(i) be the symbolic object that belongs to ith class in the symbolic knowledge base SKB/SV(i). In this work, SO(i) ϵ SKB/SV(i),∀i = 1 to n, represents qualitative interval feature vector of ith class. Then, the definition of symbolic similarity measure S between two quantitative intervals valued types SO(i) and Pobj can be defined as: Let al = lower bound of interval WGD(i)(minimum value), belonging to (i) SO . au = upper bound of interval WGD(i) (maximum value), belonging to (i) SO . bl = lower bound of probe object Pobj (minimum value). bu = upper bound of probe object Pobj (maximum value). inters = number of common elements between SV(i) and Pobj. ls = span length of WGD(i)and Pobj. =|max (au, bu) – min(al,bl)|. In our proposed work, upper bound and lower bound are equal in probe object Pobj i.e. bl==bu. Hence ls value is estimated in this work as; ls = | max (au, bu) − al|. Then, the symbolic similarity components due to content for ith class is:  SðiÞ c SOðiÞ ; P obj ¼ intersðiÞ =ls

ð19Þ

where inters(i) is determined by employing overlap measure method as in [32]. In which, if the feature values of SO(i) and Pobj are approximately equal then a similarity of 1 is assigned otherwise similarity as 0.

Table 4 Composition of AR database and recognition results. Individuals/Labels Illumination variation (S1) Expression (S2) Occlusion (Scarves + sunglasses) (S3) Scarves — Illumination (S4) Sunglasses — Illumination (S5) Total number of images (S1 + S2 + S4 + S5)

Total number samples used for testing

Number of samples recognized correctly

Number of samples misclassified

Accuracy in %

480 420 (73 + 25) = 98 244 56 1200

440 400 (70 + 25) = 95 227 55 1122

40 20 (03 + 00) = 03 17 01 78

91.67 95.24 96.94 93.03 98.21 93.50

V.C. Kagawade, S.A. Angadi / Image and Vision Computing 83–84 (2019) 39–50

47

for test symbolic object against face symbolic objects present in the symbolic knowledge. 4. Results and performance analysis 4.1. Experimental results

Fig. 11. Average recognition rate of each class of variation.

In general, overlap measure between SO(i) and Pobj for ith class can be estimated, such that intersðiÞ ¼

m X

 CMP SOij ; Pobj ∀i ¼ 1 to N

ð20Þ

j¼1

where  CMPðSOij ; PobjÞ ¼

1 if DGMðψij Þ≅DGMðPobjÞ ; ∀i ¼ 1 to N and∀ j ¼ 1 to M, 0 otherwise

Further the net similarity vector is computed against probe object to all symbolic Hoard objects present in the knowledge base and represented as in Eq. (21). n o SIM ¼ SðiÞ ; ∀i ¼ 1 to N

ð21Þ

where SIM is a net similarity vector of size 1 × N. Finally, by using Eq. (21) the maximum similarity score is found as in Eq. (22). Class ID ¼ arg maxðSIMÞ

ð22Þ

where max(.) functions finds index of the recognized person class. Since the proposed symbolic modeling approach to face recognition requires minimum number of features to represent a face image i.e. each face image is represented by single valued object (ref Eq. (15)). Hence, the proposed work is more robust and efficient compared to some of the existing works as it requires minimum number of comparisons between probe object and all trained objects. In summary, the overall process of symbolic data modeling of facial features and symbolic similarity measure employed in this work for face recognition is illustrated in Fig. 8. Assume that the face database consists of M number of samples for an individual. Then, the proposed method involves building of gray scale image gallery by detecting and cropping face part from the original face images. In training stage, the multidirectional local gradient facial feature (DGM) is extracted from M samples of N classes and is represented as symbolic objects. Further during testing stage, symbolic similarity measure score is computed

The performance of the proposed method is evaluated by considering AR and LFW face databases and comparing with some of the stateof-the-art classification methods. The AR database has been considered to investigate the robustness of the proposed method for class of variations such as illumination changes, facial expression/pose changes and different occlusion. It consists of 120 individual frontal face images. In two sessions, 26 images of each individual are captured and are numbered 1 to 13 and 14 to 26 respectively. The subjects are in an upright, frontal position against a dark homogenous background. The LFW database consists of 13,233 color face images of 5749 individuals. All the face images of LFW database are collected from various web sources and vary in scale and quality, expression, pose, lighting conditions and occlusion [45,46]. The proposed MLGD descriptor has been implemented using R2014a Matlab tool. In the proposed work, the Viola–Jones algorithm is used to detect and crop the face part from the original image. The cropped face images are then converted into 64 × 64 pixel gray scale images. Some of the sample images of AR and LFW databases used for training and testing are depicted in Fig. 9. Tenfold cross validation technique is adapted to validate the proposed methodology. The image restricted settings protocol is employed on LFW database. During the performance analysis, the selection of number of samples for training and testing depends upon the works considered for comparison. To verify the effectiveness of the proposed method, a series of experiments were performed on AR and LFW face databases by considering different radial width and angular orientation from case 1 to case 6 (ref. Section 3). The following Table 2 shows experimental results obtained for all the cases. The integrated directional gradient magnitudes features computed at different direction with equal division of θ(0, 2 π) along the given empirically chosen values of α and β shows outstanding performance in most of the cases (ref. Section 2.1). In case 3, the recognition rate of the proposed work is comparably low than other cases (the method unable to compute the potential facial texture information required for face recognition). The MLGD descriptor with DGM features shows outstanding performance with respect to class of variations such as illumination, expressions, pose variations and occlusion. The performance of the proposed MLGD descriptor with DGM features is robust against illumination, expressions, pose variations and occlusion, which made it more suitable for many real applications. Fig. 10 show ROC curves for results of Table 2 on AR and LFW databases. Investigating Fig. 10 and Table 2 reveals interesting outcomes. As the occlusion consistently is increased as in the case of AR database, recognition rate deteriorates considerably. We observed that the performance of the proposed method was consistently better on LFW databases compared to the AR database. To build image gallery of 120 persons, AR databases has been divided into five subsets (Numbered from subsets 1 to 5). The subset S1 represents illumination changes. Subset 2 indicates expression change. Subset 3 represents occlusion (scarf and sunglass). S4 represents images occluded by scarf with/without varying light conditions. Subset S5

Table 5 Classification accuracy (%) of proposed work with LAG–LDA operator. Method

Illumination variation (S1)

Expression change (S2)

Scarves—Illumination (S4)

Sunglasses—Illumination (S5)

LAG-LDA Proposed MLGD method

90.67% (Exp 6) 95.24%

93.25% (Exp 1) 91.67%

73.33% (Exp 8) 93.03%

92% (Exp 7) 98%

Bold values indicates best recognition rate.

48

V.C. Kagawade, S.A. Angadi / Image and Vision Computing 83–84 (2019) 39–50

Table 6 Classification accuracy (%) of proposed work with ECNR method. Method

Illumination variation (S1)

Expression changes (S2)

Occlusion (S3)

Occlusion and illumination (S4 + S5)

98.75% 95.24%

88.33% 91.67%

92.50% 96.94%

82.50% 94.00%

ECNR Proposed method Bold values indicates best recognition rate.

represents images occluded by sunglass with/without varying light conditions. For training1200 samples of session 1 were selected and similarly session 2 images were considered for testing. The distributions of AR database images into five subsets are shown in Table 3. Several experiments were conducted on subsets S1 to S5 to evaluate the performance of the proposed symbolic modeling of face features for face recognition and results obtained from experimentation on S1–S5 set are reported in Table 4. The results of experiments in Table 4 show that the performance of the proposed algorithm is encouraging against illumination, expression changes and occlusion. The ROC curve in Fig. 11 shows the effectiveness of proposed approach on expression, illumination and occlusion variations. Fig. 11 also shows that the proposed face recognition technique is fairly robust to expression, illumination and occlusion variations of face images. It is evident that the proposed method works well on the five subsets of AR database. 4.2. Performance analysis The proposed approach was evaluated in quest of the performance of face biometric on AR and LFW databases and the results obtained are among the best and are comparable to existing state-of-the-artworks. This section discusses the comparative analysis of the proposed work with local approximation gradient operator [4] and different feature representation techniques mentioned in [43,44] on AR database. The proposed MLGD approach is also compared with some of the local pattern descriptors [14] and other feature representation techniques namely low rank and symbolic data modeling on LFW database. In the performance analysis the best case results of Table 2 are considered. 4.2.1. Performance analysis on AR database In order to determine the effectiveness of the proposed approach, series of experiments (abbreviated as Exp) are conducted on AR database and have been compared with LAG-LDA operator [4]. The operator works based on local approximation gradient of an image block. The LAG-LDA operator obtains 90.67% (Exp 6), 93.25% (Exp 1), 73.33% (Exp 8), and 92% (Exp 7) accuracy against illumination variation, expression, scarves—illumination, Sunglasses—illumination respectively. For comparison best results obtained from LAG-LDA operator and proposed method are considered and tabulated in Table 5. The results indicate that the proposed method achieves higher recognition rate of 95.24%, 93.03% and 98% for illumination, expressions changes and occlusion with illumination changes respectively on face images of AR database. However the performance reduced to 91.67% when change in expression was considered. It is also observed that LAG-LDA operator malfunctions in case of occlusion with illumination environment (S4). Proposed MLGD method is able to estimate more edge information of an image, which achieves a more robust features dissimilarity between face images. The proposed work was also compared with another similar work called extended collaborative neighbor representation (ECNR) proposed by Waqas Jadoon et al.. The ECNR exploits a new feature representation technique for face recognition [47]. Hence, in this section performance of the proposed feature representation as symbolic objects through symbolic modeling technique is also compared with ECNR method. The experimental results of the proposed work and ECNR method are tabulated in Table 6. The results in Table 6 indicate that

proposed symbolic similarity method for face recognition yields best results in the presence of occlusion and expression changes in face images. The results presented in Tables 5 and 6 support the proposed method being more robust to class variations (illumination/expression/occlusion on face images). To compare the performance of the proposed method to some of supervised learning methods introduced in [48], 25 men and 25 women images are uniformly selected to build a gallery for both training and testing. All the gallery images are resized 32p × 32p. 10 fold cross validation on whole face data set is chosen as evaluation protocol. The experimental results obtained from the proposed MLGD method and methods defined in [48] are shown in Table 7. Table 7 shows that the proposed MLGD approach has achieved average recognition rate from 88.92% to 97.33% and is better than GMD, SRC, RF, SVM(P), SVM(R), KNN(E) and KLDA(P) methods specified in [48]. The experimental results of MLGD method clearly suggests the effectiveness of proposed multidirectional gradient feature extraction and symbolic modeling technique and solves the problems associated with pose, illumination and occlusion in an unified manner. It is evident from Tables 5 to 7 that the proposed symbolic representation of face features yields better performance compared to ECNR, LAG-LDA and minimax representation techniques and is also more robust to variations that include illumination, expression and occlusion on face images. 4.2.2. Performance analysis on LFW database In this work the LFW database has also been considered to evaluate the recognition performance of proposed MLGD method in unconstrained environment, which is considerably harder than AR dataset. Experimental analysis is done on a subset of LFW database. For the performance analysis, a low-rank representation technique and symbolic objects representation techniques are considered. Many supervised metric learning techniques have been proposed and designed to learn a discriminative metric that minimizes the distance between data points within-class and maximizes among the classes on LFW database. Recently, a low-rank representation technique called Discriminative Low-rank Metric Learning (DLML) technique has been presented for face recognition [49]. In order to show the effectiveness of facial feature representation as symbolic objects, the proposed technique has been compared with low rank metric representation method mentioned in [49]. The proposed work is also compared with similar work, a symbolic modeling of Polar FFT features proposed in [32] and CNN approach [21]. For comparison, similar experimental setup has been made i.e. cropped face images are rescaled to the dimension of 120p × 120p and the image restricted settings protocol is adapted. From 80 individuals 800 face images were considered to build a gallery. For training and testing, arbitrarily 9 images and Table 7 Classification accuracy (%) of proposed work with state-of-the-art techniques mentioned in [48] on AR database. The results are drawn from [43]. Our method 96.17%

GMD

SRC

RF

SVM(P)

SVM(R)

KNN(E)

KLDA(P)

92.88%

93.29%

89.80%

38.30%

94.55%

95.64%

94.72%

Legend: GMD-Generalized Multiplicative Distortion, SRC-Sparse Representation Classifier, RF-Random Forest, SVM(P)-Support Vector Machine, SVM(R)-SVM with kernels, KNN(E)kernel K-Nearest Neighbor classifier with Euclidian distance, KLDA(P)- Kernel Linear Discriminant Analysis with Polynomial.

V.C. Kagawade, S.A. Angadi / Image and Vision Computing 83–84 (2019) 39–50 Table 8 Comparison of experimental results of proposed work with state-ofthe-art-works on LFW database. Methods Xing DML-eig SILD ITML LDML KISSME DLML FaceNet PFFT features Our method

Recognition rate (%) 74.64–80.82 82.28–87.94 80.07–86.04 79.98–85.94 80.65–86.64 83.37–88.92 85.35–91.15 99.63 ± 0.09 97.07 97.25

remaining one is selected respectively. The experimental results of proposed technique and some of the state-of-the-art methods mentioned in [21,32,49] are depicted in Table 8. The experimentation results of Table 8 reveal that unconstrained environment deteriorates performance of the previous ones reported in [49] compared to proposed local directional gradients feature-based face recognition technique. It is also observed that FaceNet technique achieves 99.63 ± 0.09% accuracy by employing standard protocol for unrestricted, labeled outside data [21] (the protocol adapted in this technique is different from proposed technique). The experimental results also provide sufficient proof that proposed MLGD method provides a robust mechanism in handling within class variation in an unconstrained environment (as LFW represents). Recognition rate of proposed MLGD method is also compared to some of the local pattern descriptor methods such as LVP, LBP, LDP, LTrP [14], LGHP [2], MDML-DCPs + PLDA + Score averaging, MDMLDCPs + PLDA + Linear SVM, MDML-DCPs + JB + Linear SVM and MDML-DCPs + PLDA + JB + Linear SVM [17]. To the demonstrate the effectiveness of Local Vector Pattern (LVP) descriptor, K. C. Fan divided the LFW database into four subsets and performance has been determined on each subset [14]. In this work, similar settings are used for experiments as used in [2,14] i.e. 10 fold cross validation technique on LFW database (referred subset 1 mentioned in [14]). The best results obtained using LVP, LBP, LDP, LTrP, LGHP techniques with R = 3 and proposed method are considered for comparison and indicted in Table 9. The average recognition rate of the proposed method has improved from 82.96%, 76.88%, 80.84%, 83.16%, 94.57 ± 0.30%, 95.13 ± 0.33%, 95.40 ± 0.33%, 95.58 ± 0.34% and 97.25% as compared with LVP, LBP, LDP, LTrP, LGHP and some of the techniques proposed in [17] respectively. Experimental results tabulated in Table 9 demonstrate that the proposed method outperforms several state-of-the-art local pattern descriptors in face recognition. The results in Tables 8 and 9 show that the proposed approach achieves better recognition rate than existing techniques under uncontrolled environment. Due to the fact that the Table 9 Comparison of experimental results of proposed work with state-of-the-art-works based on local pattern descriptors on LFW database. Methods LVP LDP LTrP LBP LGHP MDML-DCPs + PLDA + Score averaging MDML-DCPs + PLDA + Linear SVM MDML-DCPs + JB + Linear SVM MDML-DCPs + PLDA + JB + Linear SVM Proposed MLGD method

Recognition rate (%) N82.96 N76.88 N80.84 N83.16 N87.71 94.57 ± 0.30 95.13 ± 0.33 95.40 ± 0.33 95.58 ± 0.34 97.25

Legend: LVP- Local Vector Pattern, LDP-Local Derivative Patterns, LTrP- Local Tetra Patterns, LBP-Local Binary Pattern, LGHP- Local Gradient Hexa Pattern.

49

proposed MLGD method is able extract more discriminative information of face images by considering gradient change along at different directions. Comparison of results using AR and LFW databases shows that proposed MLGD method based on directional gradient magnitude of face image is superior than other contemporary face recognition systems. Since edge responses are more stable than intensity values, the proposed technique provides batter values even if there is some presence of illumination, expression change and occlusion. The proposed face recognition approach is robust to class of variations (illumination, expression change and occlusion); since facial features are represented as symbolic objects (require minimum number of features). The proposed descriptor works even better in unconstrained environment as shown in the performance analysis on LFW database. The experimental results from Tables 5–9 show that the proposed symbolic modeling of directional gradient features of face images and newly devised modified form of content-based symbolic similarity measure can be effectively used in many real-time face recognition applications. 5. Conclusion In this paper, a new appearance-based face recognition technique is presented. The approach considers gradient change in neighboring pixels in multiple directions for classification. Total gradient weight of each face image sample is calculated by integrating gradient change in multiple directions. The gradient change in multiple directions accumulates more edge/line information, leading to effective discrimination within the face image class and among different classes of face images. The approach also explores the symbolic data modeling technique to represent the face features, which effectively represents face image features. The experimental results obtained show that modified form of content-based symbolic similarity measure yields higher recognition rate compared to existing methods. The experiment results show that the proposed approach is efficient and robust to illuminations, expression changes and the presence of occlusions and are comparable to some of the state-of-the-art works. Future work will look into how well proposed local directional gradient descriptor and symbolic modeling technique suitable for the development of multi model biometrics system, which works in unconstrained environment. Conflicts of interest The work is original and not published anywhere. The work not funded by any organization or institute. The work is completely owned by the authors. References [1] Jun-Yong Zhu, Wei-Shi Zheng, Feng Lu, Jian-Huang Lai, Illumination invariant single face image recognition under heterogeneous lighting condition, Pattern Recogn. 66 (2017) 313–327, https://doi.org/10.1016/j.patcog.2016.12.029. [2] Soumendu Chakraborty, Satish Kumar Singh, Chakraborty, Local gradient hexa pattern: a descriptor for face recognition and retrieval, IEEE Trans. Circuits Syst. Video Technol., 1051–8215 (c) 2016 IEEE, DOI https://doi.org/10.1109/TCSVT.2016.2603535. [3] Weilin Huang, Hujun Yin, Robust face recognition with structural binary gradient patterns, Pattern Recogn. 68 (2017) 126–140, https://doi.org/10.1016/j.patcog. 2017.03.010. [4] Zhaokui Li, Yan Wang, Chunlong Fan, Jinrong He, Image preprocessing method based on local approximation gradient with application to face recognition, Pattern. Anal. Applic., Springer 2015, DOI https://doi.org/10.1007/s10044-015-0470-6. [5] Amrit Kumar Agrawal, Yogendra Narain Singh, Evaluation of face recognition methods in unconstrained environments, Procedia Computer Science 48 (2015) 644–651www.sciencedirect.com. [6] Baochang Zhang, Yongsheng Gao, Sanqiang Zhao, Jianzhuang Liu, Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor, IEEE Trans. Image Process. 19 (2) (February 2010). [7] Liang Yunjuan, Feng Hongyu, Zhang Lijun, Miao Qinglin, Gradient direction based human face positioning algorithm applied in complex background, Advances in Technology and Management, 2012, AISC, 165, Springer-Verlag Berlin Heidelberg 2012, pp. 385–391, https://doi.org/10.1007/978-3-642-29637-6_49.

50

V.C. Kagawade, S.A. Angadi / Image and Vision Computing 83–84 (2019) 39–50

[8] S. Chakraborty, S.K. Singh, P. Chakraborty, Local directional gradient pattern: a local descriptor for face recognition, Multimed. Tools Appl. (2015) 1–16. [9] Zi Liu, Xiaoning Song, Zhenmin Tang, Fusing hierarchical multi-scale local binary patterns and virtual mirror samples to perform face recognition, Neural Comput. & Applic. 26 (8) (November 2015) 2013–2026, https://doi.org/10.1007/s00521015-1863-6. [10] Mohamed Dahmane, Langis Gagnon, Local phase-context for face recognition under varying conditions, Procedia Computer Science 39 (2014) 12–19. [11] Zhenhua Chai, Heydi Mendez-Vazquez, Ran He, Zhenan Sun, Tieniu Tan, Semantic pixel sets based local binary patterns for face recognition, ACCV 2012, Part II, LNCS, 7725, 2013, pp. 639–651. [12] Taskeed Jabid, Hasanul Kabir Md, Oksam Chae, Robust facial expression recognition based on local directional pattern, ETRI J. 32 (2010) Number 5, October. [13] Shys-Fan Yang-Mao, Yung-Fu Chen, Yung-Kuan Chan, Meng-Hsin Tsai, Yen-Ping Chu, Gradient direction edge enhancement based nucleus and cytoplasm contour detector of cervical smear images, ICMB 2008, LNCS 4901, pp. 290–297, SpringerVerlag Berlin Heidelberg 2007. [14] Olarik Surinta, Mahir F. Karaaba, Lambert R.B. Schomaker, Marco A. Wiering, Recognition of handwritten characters using local gradient feature descriptors, Eng. Appl. Artif. Intell. 45 (2015) 405–414, https://doi.org/10.1016/j.engappai.2015.07.017. [15] Subrahmanyam Murala, R.P. Maheshwari, R. Balasubramanian, Local tetra patterns: a new feature descriptor for content-based image retrieval, IEEE Trans. Image Process. 21 (5) (May 2012) 2874–2886. [16] Kuo-Chin Fan, Tsung-Yung Hung, A novel local pattern descriptor-local vector pattern in high-order derivative space for face recognition, IEEE Trans. Image Process. 23 (7) (May. 2014) 2877–2891. [17] Changxing Ding, Jonghyun Choi, Dacheng Tao, Larry S. Davis, Multi-directional multilevel dual-cross patterns for robust face recognition, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 38 (3) (2016) 518–531. [18] Bo Du, Wei Xiong, Jia Wu, Lefei Zhang, Liangpei Zhang and Dacheng Tao, Stacked convolutional denoising auto-encoders for feature representation, IEEE Transactions on Cybernetics, vol. 47, NO. 4, April 2017, doi: https://doi.org/10.1109/TCYB.2016. 2536638,2016. [19] Bo Du, Zengmao Wang, Lefei Zhang, Liangpei Zhang, Wei Liu, Jialie Shen, Dacheng Tao, Exploring representativeness and informativeness for active learning, IEEE Trans. Cybern. 47 (1) (January 2017) 14–26. [20] Changxing Ding and Dacheng Tao, Trunk-Branch Ensemble Convolutional Neural Networks for Video-based Face Recognition, IEEE Trans. Pattern Anal.Mach. Intell. (T-PAMI), vol. 40, no. 4, pp. 1002–1014, 2018. [21] Florian Schroff, Dmitry Kalenichenko, James Philbin, FaceNet: a unified embedding for face recognition and clustering, Computer Vision and Pattern Recognition (CVPR), 2015. [22] Y. Taigman, M. Yang, M.A. Ranzato, L. Wolf, Deepface: Closing the gap to humanlevel performance in face verification, Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (2014) 1701. [23] K. Chidananda Gowda, Edwin Diday, Symbolic clustering using a new similarity measure, IEEE Trans. Syst. Man Cybern. 22 (2) (1992) 368–378. [24] Nagabhushan P., Angadi S.A., Anami B.S., Symbolic data structure for postal address representation and address validation through symbolic knowledge base, SpringerVerlag Berlin Heidelberg, 2005, LNCS 3776, pp. 388–394. [25] P. Nagabhushan, S.A. Angadi, B.S. Anami, Symbolic data structure for postal address representation and address validation through symbolic knowledge base, Pattern Recognition and Machine Intelligence Proceedings, 2005, https://doi.org/10.1007/ 11590316_59. [26] T.V. Ravi, K. Chidananda Gowda, A new nonhierarchical clustering procedure for symbolic objects, IDEAL (2000) 35–41. [27] T.V. Ravi, K. Chidananda Gowda, An ISODATA clustering procedure for symbolic objects using a distributed genetic algorithm, Pattern Recogn. Lett. 20 (7) (1999) 659–666.

[28] Miin-Shen Yanga, Wen-LiangHung, De-HuaChen, Self-organizing map for symbolic data, Fuzzy Sets Syst. (2012), https://doi.org/10.1016/j.fss.2012.04.006 [29] Hiremath P. S., Prabhakar C. J., Face recognition technique using symbolic PCA method, Springer-Verlag Berlin Heidelberg, 2005, LNCS 3776, pp. 266–271. [30] Hiremath P.S., Prabhakar C.J., Face recognition technique using symbolic linear discriminant analysis method, Springer-Verlag Berlin Heidelberg, P. Kalra and S. Peleg (Eds.): ICVGIP, 2006, LNCS 4338, pp. 641–649. [31] Hiremath P.S., Prabhakar C.J., Face recognition using symbolic KPCA plus symbolic LDA in the framework of symbolic data analysis: symbolic kernel fisher discriminant method, ACIVS, 2008, LNCS 5259, pp. 982–993, 2008, Springer-Verlag Berlin Heidelberg. [32] Shanmukhappa A. Angadi, Vishwanath C. Kagawade, A robust face recognition approach through symbolic modeling of polar FFT features, Pattern Recogn. 71C (2017) 235–248, https://doi.org/10.1016/j.patcog.2017.06.014. [33] Shanmukhappa Angadi, Vishwanath Kagawade, Face recognition through symbolic modeling and analysis of segmented face images using savitzky golay filter features, International Journal of Technology and Science, ISSN (Online) 2350-1111, (Print) 2350-1103 Volume IX, Issue 1 (2016) 50–55. [34] Bapu B. Kiranagi, Guru D. S., A new symbolic dissimilarity measure for multivalued data type and novel dissimilarity approximation techniques, Int. J. Comput. Appl., 2010, 0975–8887, Volume 1– No. 26. [35] G. Raghavendra Rao, K. Chidananda Gowda, New dissimilarity measure to improve the GA performance, IEA/AIE 1 (1998) 487–492. [36] K. Chidananda Gowda, T.V. Ravi, Agglomerative clustering of symbolic objects using the concepts of both similarity and dissimilarity, Pattern Recogn. Lett. 16 (6) (1995) 647–652. [37] K. Chidananda Gowda, T.V. Ravi, Divisive clustering of symbolic objects using the concepts of both similarity and dissimilarity, Pattern Recogn. 28 (8) (1995) 1277–1282. [38] K. Chidananda Gowda, Edwin Diday, Symbolic clusters using a new dissimilarity measure, Pattern Recogn. 24 (6) (1991) 567–578. [39] K. Chidananda Gowda, T.N. Vikram, Shalini R. Urs, 2 directional 2 dimensional pairwise FLD for handwritten kannada numeral recognition, ICADL (2007) 499–501. [40] M.S. Dinesh, Gowda K. Chidananda, P. Nagabhushan, Fuzzy-symbolic analysis for classification of symbolic data, PReMI 2005 (2000) 338–343. [41] Nagabhushan P., Angadi S.A., Anami B.S., A fuzzy symbolic inference system for postal address component extraction and labelling, 2006, Springer-Verlag Berlin Heidelberg LNAI 4223, pp. 937–946. [42] P. Nagabhushan, K. Chidananda Gowda, Edwin Diday, Dimensionality reduction of symbolic data, Pattern Recogn. Lett. 16 (2) (1995) 219–223. [43] P. Viola, M. Jones, Robust real-time face detection, Int. J. Comput. Vis. 57 (2) (2004) 137–154. [44] Martinez A. M, Benavente R, The AR Database, CVC technical report, 1998. [45] Arash Rikhtegar, Mohammad Pooyan, Mohammad Taghi Manzuri-Shalmani, Genetic algorithm-optimised structure of convolutional neural network for face recognition applications, IET Comput. Vis. 10 (6) (2016) 559–566. [46] G.B. Huang, M. Ramesh, T. Berg, E. Learned-Miller, Labeled faces in the wild: a database for studying face recognition in unconstrained environments, University of Massachusetts, Amherst, Tech. Rep. 07–49 (October, 2007). [47] Waqas Jadoon, Lei Zhang, Yi Zhang, Extended collaborative neighbor representation for robust single-sample face recognition, Nat. Comput. App. (2015), https://doi.org/ 10.1007/s00521-015-1843-x. [48] Qiang Cheng, Hongbo Zhou, Jie Cheng, Huiqing Li, A minimax framework for classification with applications to images and high dimensional data, IEEE Trans. Pattern Anal. Mach. Intell. 36 (11) (November 2014). [49] Zhengming Ding, Sungjoo Suh, Jae-Joon Han, Changkyu Choi, Yun Fu, Discriminative Low-Rank Metric Learning for Face Recognition, 978-1-5799-6026/15, IEEE, 2015.