Local energy oriented pattern for image indexing and retrieval

J. Vis. Commun. Image R. 64 (2019) 102615

Contents lists available at ScienceDirect

J. Vis. Commun. Image R. journal homepage: www.elsevier.com/locate/jvci

Local energy oriented pattern for image indexing and retrieval q G.M. Galshetwar a,⇑, L.M. Waghmare b, A.B. Gonde a, S. Murala c a

Centre of Excellence in Signal and Image Processing (COESIP), Department of ECE, SGGSIET, Nanded, Maharashtra, India Department of Instrumentation Engineering, SGGSIET, Nanded, Maharashtra, India c Computer Vision and Pattern Recognition Laboratory, Department of EE, IIT Ropar, India b

a r t i c l e

i n f o

Article history: Received 6 December 2017 Revised 16 May 2019 Accepted 22 August 2019 Available online 26 August 2019 Keywords: Local Binary Patterns (LBP) Local Mesh Patterns (LMeP) Local Directional Mask Maximum Edge Patterns (LDMaMEP)

a b s t r a c t A novel image indexing algorithm for Content Based Image Retrieval (CBIR) using Local Energy Oriented Patterns (LEOP) is proposed in this paper. LEOP encodes pixel level energy orientations to find minute spatial features of an image whereas existing methods use neighborhood relationship. LEOP maps four pixel progression orientations to find top two maximum energy changes for each reference pixel in the image i.e. for each reference 3
3 grid, two more 3
3 grids out of four pixel progression are extracted. Finally, LEOP encodes the relationship among pixels of three 3
3 local grids extracted. LEOP is applied on four different image databases named MESSIDOR, VIA/I-ELCAP, COREL and ImageNet Database using traditional CBIR framework. To test the robustness of proposed feature descriptor the experiment is extended to a learning based CBIR approach on COREL database. The LEOP outperformed state-of-the-art methods in both traditional as well as learning environments and hence it is a strong descriptor. Ó 2019 Elsevier Inc. All rights reserved.

1. Introduction In recent years there has been technological developments in the field of image acquiring devices as well as evolution of social media, that lead to creation of enormous amount of image data all over the internet in digital form let it be due to natural photography, wildlife photography etc. Also, there has been technological revolution in the field of medical science in near past, which has led to noticeable changes in diagnosis taking help of medical imaging techniques. These revolutions have made a great impact on our day-to-day life. Automatically huge image databases are being created in raw form. Images captured by medical imaging techniques are mostly used once and later they are stored as trash in the systems. 1.1. Motivation In last few years, there has been a rapid growth in the severe and critical diseases in India and all over the world. Because of this growth, there is increasing need of expert medical services in urban as well as in remote places especially in developing countries. In areas where general clinical practices are present, we q

This paper has been recommended for acceptance by ‘Zicheng Liu’.

⇑ Corresponding author.

E-mail address: [email protected] (G.M. Galshetwar). https://doi.org/10.1016/j.jvcir.2019.102615 1047-3203/Ó 2019 Elsevier Inc. All rights reserved.

can provide them a technological solution which can assist those clinics in bringing expert medical services to their help. Bio-medical imaging has emerged as a very useful technological development in medical diagnostic field. It is used to create visual representation of different interior parts of body that are used for medical analysis and diagnosis. There are different types of Biomedical imaging techniques such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI) and Fundus Imaging etc. With these techniques in hand, engineers can provide further technological solutions to medical field. Content Based Image Retrieval (CBIR) is one of such technological solutions in which engineering and medical streams can work hand-in-hand. CBIR works in two stages first is the feature extraction stage in which features are extracted from the images in database and feature vector database is created. In the second stage, similarity matching is done in which distance of query image feature is measured from each of the images in feature database. The detailed literature survey about CBIR is given in [1–8]. 1.2. Related work Color is the significant feature of an image, if it is maintained semantically intact and in a perceptually oriented way. CBIR system considering color as most significant feature is proposed in [9]. The concept of color histogram is introduced in [10] to find the relationship between the color distributions of pixels and

2

G.M. Galshetwar et al. / J. Vis. Commun. Image R. 64 (2019) 102615

spatial correlation between pairs of color. In [11] authors have introduced the concept of the color correlogram. Another algorithm for image retrieval is proposed in [12] in which authors used first three central moments of each color space called mean, standard deviation and skewness. In the above proposed algorithms the retrieval accuracy of feature descriptors is less, especially on large database because of insufficiency in discrimination power. Along with color feature, the shape feature of an image is also widely used. After doing segmentation of the image into region, the shape feature of each region is used for feature extraction. In [13] authors used Fourier descriptor to collect shape description. However, this method is very sensitive to the noise and invariant to the geometric transformation. To overcome these limitations, the modified Fourier descriptor is proposed in [14], which overcomes the limitations of the existing Fourier descriptor. Along with color and shape, another important characteristic of an image is texture, so texture based feature descriptor also plays an important role in the field of CBIR. For texture investigation (texture arrangement and texture based image retrieval), Discrete Wavelet Transform (DWT) [15], Gabor transform [16], Rotated wavelet filters [17], Dual-Tree Rotated Complex Wavelet Filter (DT-CWT) [18] based feature are used. Rotation invariant complex wavelet filters are proposed in [19] for texture based image retrieval to address the problems present in DWT (limited directional information). Local Binary Pattern (LBP) is introduced in [20] for texture classification. Rotational invariant LBP is proposed in [21] texture classification. In [22] authors proposed the Completed LBP (CLBP) for texture classification. To address the problem present in LBP feature descriptor (LBP gives all directional first order derivatives), in [23] authors presented the local derivative pattern for pattern recognition. Local maximum edge binary patterns are presented in [24] for image retrieval and object tracking. In [25] authors proposed a new CBIR system in which the feature vector of CH (color histogram) and SOT (spatial orientation tree) is constructed and stored for similarity matching. Using binary wavelet transform and local binary pattern Directional Binary Wavelet Pattern (DBWP) is proposed in [26]. Local Tetra Pattern (LTrP) presented in [27] calculates the first-order derivatives in vertical and horizontal direction and based on these directions the relationship between the referenced pixel and its neighbors is encoded. Motif Co-Occurrence Matrix (MCM) proposed in [28] divides the whole image in to non-overlapping 2x2 grid patterns to obtain the motif-transformed image which later constructs MCM. In [29] authors combined the color, texture and MCM feature to construct feature vector, used for CBIR purpose. The Modified Color MCM is presented in [30] that obtains the inter-correlation among the RG-B color planes for CBIR; this is absent in color MCM. Another CBIR method Directional Local Ternary Pattern (LTP) is proposed in [31] that gives directional information with respect to reference pixels. Local Ternary Co-occurrence Pattern (LTCoP) proposed in [32] integrates the concept of Local ternary pattern and cooccurrence matrix for CT and MRI image retrieval, which encodes the co-occurrence between ternary images. Directional Local Motif XoR Patterns (DLMXoRP) [33] calculated motif using flexible 1
3 grids to construct motif image, later XoR operations are applied on motif images. In [51] authors have proposed sparse embedded hashing framework for image retrieval. They first generate sparse representations in a data-driven way and later, the projection matrix is learned by employing matrix factorization technique to approximate the Euclidean structures of the original data. In [52] authors have designed multi-scale trajectory-pooled 3D convolutional descriptors. In this approach they calculate multi-scale dense trajectories from the input video and perform trajectory pooling on feature maps of 3D Convolutional Neural Network (3D CNN). In [55] authors proposed a multi-graph learning based medical image retrieval technique in which, the query category

prediction is done by formulating query into a multi-graph structure with a set of selected subjects in the database to learn the relevance between the query subject and the existing subject categories based on which a set of subjects in the database are selected as candidate retrieval results. Later, the relationship between these candidates and the query is further learned with a new multi-graph which is used to rank the candidates. The affective image retrieval via multi-graph learning is proposed in [53] where, authors used combination of extracted generic features and features derived from elements-of-art are extracted as lowlevel features. Later, they constructed single graph for each kind of feature to test the retrieval performance. Local Gaussian Difference Extrema Pattern (LGDEP) is proposed in [54] for face recognition. In this method authors calculated prominent edges from three Gaussian scaled images. Volumetric Local Directional Triplet Patterns (VLDTP) for biomedical image retrieval is proposed in [55], where in authors considered directional information of 3D image to construct directional local triplet pattern. In this proposed method, we are calculating amount of pixel progression in four gradient directions to indicate maximum changes in those specific directions. Our proposed pattern encodes the relationship among pixels of three 3
3 local patches, out of which two 3
3 local patches are extracted by moving the reference 3
3 local patch in two gradient directions which give first two maximum pixel progressions out of four pixel progression directions. 1.3. Main contribution We have proposed a novel pattern to encode the gradient directional information among 3
3 local blocks of image extracted from image. First 3
3 reference block is extracted by considering each pixel as center pixel. Later by moving this local block in two gradient directions giving maximum pixel progression, we extract two more 3
3 local blocks. We encode the comparison of the pixel intensity of each pixel in first local block with each corresponding pixel in two extracted directional local blocks. Arrangement of paper is as follows: Section 2 gives introduction to some existing local patterns which inspired us for our approach. Section 3 gives detail description of proposed methodology. The experimental results analysis of proposed feature descriptor is depicted in Section 4. Section 5 gives concluding remarks. 2. Local patterns 2.1. Local Binary Pattern (LBP) A pattern very much useful for texture classification is presented in [20] popularly known as LBP, which carries out pixel intensity comparison of center pixel with the neighbor pixels. The comparison is encoded using (1).

LBPP;R ¼

P X

2ðn1Þ
f 1 ðIn  Ic Þ

ð1Þ

n¼1

where,



1 xP0 where In and Ic indicate pixel intensity of 0 else neighbor and center pixel respectively. P gives total number of surrounding neighbors and R is the neighborhood radius. f I ðxÞ ¼

2.2. Local Mesh Pattern (LMeP) Local mesh pattern popularly known as LMeP is proposed in [36]. For each pixel in an image the relationship among its

3

G.M. Galshetwar et al. / J. Vis. Commun. Image R. 64 (2019) 102615

surrounding neighbors is encoded to compute LMeP and is carried out using (2).

LMePkN;R

¼

N X

2

ðn1Þ


f 1 ðIa j

R

 Ii j R Þ

ð2Þ

n¼1

3.1. Gradient calculation The horizontal (Gx ) and vertical gradient (Gy ) calculation at each pixel is done using (3) and (4)

Gx ¼

@I @x

ð3Þ

Gy ¼

@I @y

ð4Þ

a ¼ 1 þ mod ððn þ N þ k  1Þ; NÞ   N 8 k ¼ 1; 2; :::; 2 where LMeP index is given by k and mod ðA; BÞ returns the remainder for fA=Bg operation. P gives number of surrounding neighbors and R is the neighborhood radius.

Here I is the input image of size ðX; YÞ. Using horizontal and   vertical directions from (3) and (4) gradient magnitude (Gxy ) and gradient direction (aðx; yÞ) are calculated at each pixel using (5) and (6).

ffi   qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Gxy  ¼ G2 þ G2 x y

2.3. Local Directional Mask Maximum Edge Pattern (LDMaMEP) In [37] maximum directional edge information in an image is encoded to construct LDMaMEP. Here they first applied eight directional masks to obtain directional edges in the images, which are later used for computations of MEP (maximum edge patterns) and MEPP (maximum edge position patterns). The histograms of MEP and MEPPs are combined to form the final feature vectors and that are later used to do similarity matching between query image and database images.

3. Local energy oriented pattern The descriptors proposed in [20,36,37] inspired us to propose a novel pattern Local Energy Oriented Pattern (LEOP). Fig. 1 gives detail block diagram representation of the proposed system framework. In the first stage, LEOP calculates gradient magnitude and gradient direction at each pixel and quantize all gradient directions n o in four gradient directions þ900 ; þ450 ; 00 ; 450 to indicate maximum changes in those specific directions. Further, in the second stage, we select two directions out of four gradient directions, showing first maximum and second maximum changes at each pixel. In the third stage, for a given reference pixel, we extract reference local 3
3 patch and two directional 3
3 local patches considering first and second maximum directions selected previously in second stage. In fourth stage, LEOP encodes the relationship among pixels of three 3
3 local patches extracted in third stage.

aðx; yÞ ¼ tan1

ð5Þ

  Gy Gx

ð6Þ

Here ðx; yÞ gives location of current pixel in an input image of size ðX; YÞ. 3.2. 1.1 Direction quantization All the gradient directions calculated from (6) are quantized and stored into four directional maps by computing maximum directional change at each pixel using (7). For computing maximum directional change, we are considering map of size ð2  N 1 þ 1Þ
ð2  N 2 þ 1Þ.

IaL ðx; yÞ ¼

N1 X

N2 X   Gxy ðx; yÞ
f ðaL ; aU ; aðx; yÞÞ 2

where

 f 2 ða; b; cÞ ¼

1 if a 6 c < b 0

else

8 aL ¼ þ90; þ45; 0; 45;

aU ¼ aL  45;

In (7) IaL is the directional map showing amount of pixel progression in gradient direction aL . For experimental purpose, we

Maximized (Quantized ( ) Magnitude) MAP

3-D Grid Extracon

Image Database Gradient Magnitude

Direcon 1 Direcon 2 Direcon 3

Gradient Direction

ð7Þ

x¼N 1 y¼N2

1st Maxima Grid 1st Maxima Reference Grid 2nd Maxima

Extrema Encoding

Paern Calculaon

2nd Maxima Grid

Direcon 4 Histogram

Query Image Retrieval Result Fig. 1. System block diagram of proposed framework.

Similarity Measurement

Feature Vector

4

G.M. Galshetwar et al. / J. Vis. Commun. Image R. 64 (2019) 102615

Input Image (I)

59

45

31

44

43

37

900

-450

-450

900

-450

-450

00

57

43

22

5

15

20

31

00

450

450

450

-450

00

00

22

55

23

10

1

15

11

32

-450

900

-450

-450

900

450

450

16

34

3

3

2

4

13

27

00

450

-450

900

450

450

450

Gy

31

2

1

2

3

14

28

00

00

-450

-450

00

450

450

Gx

30

6

2

2

4

15

26

00

00

900

-450

450

450

450

36

30

28

29

29

30

29

00

-450

900

900

900

900

450

54

39

32

47

28

17

76

51

39

30

44

31

24

43

32

33

33

32

31

31

30

29

31

29

27

28

30

30

31

31

28

17

22

29

28

29

29

26

14

29

28

29

28

25

Gx2

Gxy

( x, y )

(33 31)

Vertical Difference Around Center Pixel

90

Gxy Map

1

tan

900

I (29 29) x Horizontal Difference Around Center Pixel

I y

G y2

13

Gx

Gy

Map

{ 450 , 900 ,00 , 450 }

93

68

20

Quantized

Magnitude Gxy Map

450

450

00

1

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

0

1

1

0

1

1

0

0

0

0

0

0

0

0

0

1

1

1

0

0

0

1

0

0

0

0

1

1

0

0

0

0

1

0

0

0

1

0

0

1

0

0

0

0

0

0

0

1

1

0

0

0

0

0

0

0

1

0

1

1

0

0

0

0

0

0

1

0

0

0

0

1

0

0

1

1

1

1

0

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

1

1

1

1

0

0

1

0

0

0

0

1

1

0

0

0

0

0

1

0

0

0

0

0

0

0

0

1

1

1

1

1

0

0

0

0

0

0

0

0

1

0

0

0

0

0

1

1

1

1

0

0

0

0

0

0

0

1

1

0

0

0

0

0

0

0

1

0

0

0

0

0

0

45

0

0

0

Maps

450

93

0

0

31

0

0

0

0

0

0

0

0

0

0

0

0

44

43

0

0

0

0

0

0

0

0

43

22

5

0

0

0

57

0

0

0

0

20

31

0

0

0

0

15

0

0

0

23

0

0

15

0

0

0

0

0

0

0

11

32

0

0

0

0

0

0

0

55

0

10

1

0

0

0

0

0

0

2

0

0

0

0

3

0

0

4

13

27

34

0

0

0

0

0

0

0

0

3

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

14

28

31

2

0

0

3

0

0

0

0

1

2

0

0

0

0

0

2

0

0

0

0

0

0

0

0

4

15

26

30

6

0

0

0

0

0

0

0

0

2

0

0

0

0

0

28

29

29

30

0

0

0

0

0

0

0

29

36

0

0

0

0

0

0

0

30

0

0

0

0

0

Maximized (

90 1 0

45

0

0

0

0

0

0

37

0

59

45

0

Gxy ) MAP

450 4

00 3

2

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

282

x

x

x

x

x

x

276

x

x

x

x

x

x

287

x

x

x

x

x

x

310

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

Maxima( )

arg max( 900 , 450 ,00 , 450 ) i.e.max(1,2,3, 4)

90

1

1st Maxima ( 1 )

0

2

450

3

4

00

1

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

4

x

x

x

x

x

x

3

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

Directional Block ( I1 )

450

2nd Maxima ( 2 )

Reference Block ( I 2 )

2

Directional Block ( I 3 )

31

28

17

33

33

32

32

31

22

29

26

14

29

31

29

29

27

16

28

25

13

30

31

31

31

28

17

Gradient Directions (γ)

Binary Pattern Binary Pattern

1 0

if ( I 2 ( g i )

I1 ( g i )& I 3 ( g i )

or I 2 ( g i )

I1 ( g i )& I 3 ( g i ))

Elsewhere

Binary Weights

1

1

1

1

2

4

0

1

1

128

256

8

0

1

1

64

32

16

{1+2+4+8+16+32+256=319} 319

Fig. 2. Pattern calculation of proposed method.

selected N 1 ¼ N 2 ¼ 3. We particularly selected 7
7 neighboring size local patch for our operations. As the distance between reference patch center pixel and directional patch center pixel is 2 as shown in Fig. 2. So, we need to consider local patches with center pixel at distance 2 in all the four directions; now the neighboring size becomes 7
7.

3.3. Maxima computation We now have four directional maps giving maximum pixel progression at each pixel in the image in four directions. Using (8) we record first and second maximum gradient direction for every pixel location ðx; yÞ.

5

G.M. Galshetwar et al. / J. Vis. Commun. Image R. 64 (2019) 102615 k

ck ðx; yÞ ¼ arg max ðIaL ¼900 ðx; yÞ; IaL ¼450 ðx; yÞ; IaL ¼00 ðx; yÞ; IaL ¼450 ðx; yÞÞ a

3.6. Similarity matching

L

ð8Þ where k ¼ 1; 2 th

In (8) maxk ðA; B; C; DÞ gives k maximum value among A; B; C and D. This stage gives us top two maximum pixel progression directions which allows us encoding the maximum energy changes of the image in the specified direction. This step makes our pattern a strong encoder which is rotation invariant as well as illuminance invariant. This will improve the matching of critical changes in the image resulting in improved retrieval accuracy. This complex approach of encoding pixel progressions made our pattern superior compared to other simpler encoding approaches in literature.

d1 distance measure: The calculated histograms are stored as feature vectors of each image, we get feature vector database created out of image database jDBj denoted as F DBj ¼ ðF DBj1 ; F DBj2 ; :::; F DBjn Þ where, j ¼ 1; 2; :::jDBj and query image feature vector is denoted as F q ¼ ðF q1 ; F q2 ; :::; F qn Þ. Our primary aim is for a given query image; n best similar images are to be selected. For this purpose, we select n top matches to query image by calculating the distance of query image from images in given database jDBj. For similarity matching, d1 similarity distance metric is used which is computed using (13):

Dðq; DBÞ ¼

3.4. Pattern calculation

    n F DBji  F qi  X i¼1

The directional maxima computations help us to extract first and second maxima sub-grids for the given reference grid. The reference 3
3 grid is extracted from original image using (9)

Iref ¼ Iðx þ t; y þ tÞ

ð9Þ

where t ¼ 1 : 1 Two gradient directional blocks are extracted in (10) from original image considering the reference grid Iref .

Ick ¼ Iðx  m þ t; y þ n þ tÞ

ð10Þ

th

th

where F DBji is i feature of j image in the database jDBj and n is feature vector length. To illustrate why we preferred d1 distance measure, we compared retrieval results using d1 distance measure and other standard distance measures. The other distance measures compared are given in (14)–(16) as below: Manhattan (L1 or city-block) distance measure:

DðQ ; DBÞ ¼

N   X   f DBp;q  f Q q 

ð14Þ

q¼1

where for each ck ; m and n take values as given below

8 ck ¼ 1 ! m ¼ 2 > > > ck ¼ 3 ! m ¼ 0 > > : ck ¼ 4 ! m ¼ 2

ð13Þ

1 þ F DBji þ F qi

9 n ¼ 0> > > n¼2=

Euclidean (L2) distance measure:

n ¼ 2> > > ; n¼2

DðQ ; DBÞ ¼

N  2 X   f DBp;q  f Q q 

!1=2 ð15Þ

q¼1

The final LEOP is calculated using (11)

LEOPðx; yÞ ¼

P X

Canberra distance measure:

i

2
f 3 ðIref ðs; tÞ; Ic1 ðs; tÞ; Ic2 ðs; tÞÞ

ð11Þ

i¼0

where

 f 3 ða; b; cÞ ¼

1 if a < c&b OR a > c&b 0 else

s; t ¼ 1 : 3 where Iref ; Ic1 ; Ic2 indicate reference, first and second maximum gradient directional grids respectively. P denotes number of surrounding neighbors and R is neighborhood radius. The detail procedure for pattern calculation is described in pictorial form in Fig. 2.

    N f DBp;q  f Q q  X     DðQ ; DBÞ ¼     q¼1 f DBp;q  þ f Q q 

ð16Þ

In traditional CBIR systems choosing the distance measure is as crucial as the robustness of the feature extractor. The evaluation measures are highly affected by the choice of different distance measures [44]. So we tested performance of our feature descriptor using different standard distance measures and the retrieval result analysis is given in experiment #6. 3.7. Proposed approach

3.5. Histogram calculation Input: Grayscale Image, Output: Feature Vector After pattern calculation from (11), histogram of the LEOP map of size ðX; YÞ is computed using (12)

HistðlÞ ¼

XX i2X

f 4 ðLEOPði; jÞ; lÞ

l 2 ½0; 511

ð12Þ

j2Y

where

 f 4 ða; bÞ ¼

1 if a ¼ b 0

else

This histogram of the LEOP map is stored as feature vector for the image which is later used for similarity matching and distance calculation purpose.

1. Load input image 2. Calculate gradient magnitude and gradient directions at each pixel in image using Eqs. (3)–(6). 3. Quantize gradient directions in four directions n o þ900 ; þ450 ; 00 ; 450 using Eq. (7). 4. Calculate maximum pixel progression at each pixel using Eq. (8). 5. Find first and second maximum pixel progression at each pixel. 6. Code LEOP maps using Eq. (9)–(11) 7. Calculate LEOP histogram using Eq. (12).

6

G.M. Galshetwar et al. / J. Vis. Commun. Image R. 64 (2019) 102615

4. Results and discussions We performed experiments on Four different databases namely MESSIDOR [47] database of retinal images, VIA/I-ELCAP [48] database of lung CT Images, COREL 5 k as well as COREL 10 k [49] natural image database and ImageNet [50] database of miscellaneous objects. In the first part, we carried out experimentations using traditional CBIR systems and in the second part, we carried our experimentations using learning based CBIR system. Abbreviations used for analysis of results are given below: Abbreviations DT-CWT + GGD-Dual tree complex wavelet transform [40] LBP: Local binary patterns [34] GLBP: LBP with Gabor transform [47] LDP: Local derivative patterns [23] GLDP: LDP with Gabor transform [23] LTP: Local ternary patterns [34] GLTP: LTP with Gabor transform [34] DLEP: Directional local extrema patterns [41] LEPSEG: Local edge patterns for segmentation [38] LEPINV: Rotational invariant local edge patterns [38] BLK_LBP: Block based LBP [39] CS_LBP: Circular symmetric local binary patterns [27] LBPu2: LBP with uniform patterns [20] LTCoP: Local ternary co-occurrence pattern [32] GLTCoP: LTCoP with Gabor transform [32] GLCM1: Gray level co-occurrence matrix1 [42] INTH: Intensity histogram [42] LTrP: Local tetra patterns [27] SS3D: Spherical symmetric 3-D local ternary patterns [35] ES-LBP: Edgy salient local binary pattern [43] VLDTP: Volumetric local directional triplet patterns for biomedical image retrieval [55]

patients of diabetic retinopathy. These images are divided into four groups based on the severity of disease. These images are available in three sizes 1440
960, 2240
1488 and 2304
1536. All the images are annotated with retinopathy grades as given in the Table 1. The specifications of retinopathy grading are based on number of microaneurysms, haemorrhages and the sign of neovascularisation. Images in which above abnormalities are absent, are considered as normal images. The results of the experiment are given in Table 2. The proposed method is compared with other methods in terms of group wise retrieval precision (ARP) as well as average retrieval precision of each method applied. Fig. 3. gives performance comparison of proposed method with other existing methods in terms of Group wise and Average precision. ARP has improved significantly compared to the previous methods. The plot of group wise ARP and Average precision is given in Fig. 3. Among the earlier published methods ES-LBP could get highest ARP with the feature vector length of 6
256 and our proposed descriptor has feature vector length of 1
512 i.e. with less computational complexity for retrieval we could get remarkable improvement in ARP.

Table 1 Number of images in each retinopathy grading group with specifications. Retinopathy Grading Groups

No. of Images

Specification

Group0 Group1 Group2

546 153 247

Group3 Total

254 1200

(Normal) (0 < mA  5) and (H = 0) (5
Table 2 Comparison of different methods in terms of group wise and average arp (%).

The performance evaluation in each experiment is done by obtaining precision, average retrieval precision (ARP), recall and average retrieval rate (ARR) which are calculated using (17)-(20).

ARP ¼

d 1X PðIi Þjn610 d i¼1

Recall; R ¼

ARR ¼

NR \ NRT nRT

NR \ NRT nR

d 1X RðIi ÞjnP10 d i¼1

ð17Þ

ð18Þ

Group0

Group1

Group2

Group3

Average

Complex Wavelet LBP DT-CWT + GGD VLDTP Wavelet ES-LBP LEOP

66.70 65.79 64.00 70.03 70.70 72.05 86.70

34.32 36.90 35.00 34.06 33.68 33.67 73.59

42.75 44.86 40.00 41.94 46.72 47.61 75.37

45.24 43.89 56.00 51.58 48.33 49.92 72.00

53.08 53.15 53.70 55.73 56.28 57.82 79.58

ð19Þ Complex Wavelet Wavelet

ð20Þ

where NR is set of all relevant images in the database, N RT is set of all retrieved images from database, (N R \ N RT ) gives total number of relevant images retrieved. Also nR is total number of relevant images in database, nRT is total number of retrieved images from the database and Ii is ith query image and d is total number of images in database.

LBP ES-LBP

DT-CWT+GGD LEOP

VLDTP

90

80

70

ARP (%)

Precision; P ¼

Method

60

50

40

4.1. Experiments by traditional CBIR system 30

4.1.1. Experiment 1: Performance analysis on MESSIDOR retinal image database In experiment #1, we applied our proposed method to MESSIDOR database which consists of 1200 retinal images captured from

Group0

Group1

Group2

Group3

Average

GROUPS

Fig. 3. Performance comparison of proposed method with other existing methods in terms of group wise and average precision on MESSIDOR database.

7

G.M. Galshetwar et al. / J. Vis. Commun. Image R. 64 (2019) 102615

LTP

LBP

LTCoP

GLDP

GLTP

GLBP

GLTCoP

LEOP

95

95

90

90

85

85

80

80

75

75

ARP (%)

ARP (%)

LDP

70 65

LBPu2

LTPu2

LDPu2

GLBPu2

GLDPu2

LTCoPu2

GLTCoPu2

LEOP

GLTPu2

70 65

60

60

55

55

50

50

45

45

40

40 35

35 10

20

30

40

50

60

70

80

90

10

100

20

30

40

No. of top matches considered

50

60

70

80

90

100

No. of top matches considered

(a)

(b)

Fig. 4. (a)–(b) performance comparison of proposed method with other existing methods in terms of ARP on VIA/I-ELCAP-CT database.

4.1.2. Experiment 2: Performance analysis on VIA/I-ELCAP CT image database Experiment #2 is performed on VIA/I-ELCAP CT database which is a publicly available database jointly created by vision and image analysis group (VIA) and international early lung cancer program (I-ELCAP). These CT images are of 512
512 resolutions, are recorded in digital imaging and communications in medicine (DIACOM) format. We used 1000 such images of CT scans which are in total 10 scans of 100 images in each scan. Fig. 4(a)–(b) compares performance of proposed approach with existing approaches in terms of ARP on VIA/I-ELCAP CT database. Fig. 5(a)–(b) gives compares performance of proposed approach with existing approaches in terms of ARR on VIA/I-ELCAP CT database. In case of VIA/I-ELCAP CT database GLTCoP could get ARP of 89.41% and LEOP achieved 95.44% ARP, there is noticeable improvement in ARP. Pixel level energy mapping and local level feature encoding approach of our feature descriptor could fetch such a noticeable improvement in the results. 4.1.3. Experiment 3: Performance analysis on COREL-5K image database In experiment 3, the COREL-5K database is used for measurement of the retrieval accuracy of proposed algorithm in terms of ARP and ARR. COREL-5K contains fifty different types of images like animals, sports, bikes, etc., and each category contains 100 differ-

LBP

LTCoP

GLDP

GLTP

LDP

LTP

GLBP

GLTCoP

ent images (50x100). Fig. 6(a)–(b) compares performance of proposed approach with existing approaches in terms of ARP and ARR on COREL5K database. 4.1.4. Experiment 4: Performance analysis on COREL-10K image database In experiment 4, the Corel-10K database is used for measurement of the retrieval accuracy of proposed algorithm in terms of ARP, ARR. Corel-10K contains hundred different types of images like animals, sports, bikes, etc., and each category contains 100 different images (100
100). Fig. 7(a)–(b) shows comparative analysis of performance of proposed approach with existing approaches in terms of ARP and ARR on COREL10K database. 4.1.5. Experiment 5: Performance analysis on ImageNet database In experiment #5 we tested our pattern on ImageNet database [50] which is publicly available large database containing about 15 million images of different categories. For experimentation purpose we have used 80 classes with 1001 images in each class i.e. total 80,080 images. ImageNet database consists of set of images of different objects such as bags, balls, cow etc. Performance analysis of our proposed method on ImageNet database is done in terms of ARP and ARR. Fig. 8(a)–(b) gives comparison between performance analysis of proposed and existing approaches in terms of ARP and ARR on ImageNet database. The proposed descriptor is

LEOP

LDPu2

GLBPu2

GLTPu2

GLDPu2

LTPu2

LTCoPu2

GLTCoPu2

LEOP

LBPu2

61

62 55

57

49

52 47

ARR (%)

ARR (%)

43

37

31

42

37 32 27

25

22 19

17 13

12 7

7 10

20

30

40

50

60

70

No. of top matches Considered

(a)

80

90

100

10

20

30

40

50

60

70

80

No. of top matches Considered

(b)

Fig. 5. (a)–(b) performance comparison of proposed method with other existing methods in terms of ARR on VIA/I-ELCAP-CT database.

90

100

8

G.M. Galshetwar et al. / J. Vis. Commun. Image R. 64 (2019) 102615

CS_LBP

LEPSEG

LEPINV

BLK_LBP

LBP

DLEP

LTP

CS_LBP

LEOP

56

51

LEPINV

BLK_LBP

LBP

DLEP

LTP

LOEP

21

46

19

41

17

ARR (%)

ARP (%)

LEPSEG

23

36

15 13

31

11 26

9 21

7 16

5

11

3 10

20

30

40

50 60 No. of top matches considered

70

80

90

100

10

20

30

40

50 60 No. of top matches considered

(a)

70

80

90

100

(b)

Fig. 6. (a)–(b) performance of proposed method with other existing methods in terms of ARP and ARR on COREL5K database.

CS_LBP

LEPSEG

LEPINV

BLK_LBP

LBP

DLEP

LTP

LEOP

CS_LBP LBP

LEPSEG DLEP

LEPINV LTP

BLK_LBP LEOP

20

50

18

45

16 14

35

ARR (%)

ARP (%)

40

30

12 10

25 8

20

6

15

4

10

2

10

20

30

40

50

60

70

80

90

100

10

20

30

40

50

60

70

80

90

100

No. of top matches considered

No. of top matches considered

(a)

(b)

Fig. 7. (a)–(b) gives performance comparison of proposed method with other existing methods in terms of ARP and ARR on COREL10K database.

GLBP

LBP

LDP

LTP

LTrP

GLBP

LEOP

Average Retrieval Rate (ARR) (%)

Average Retrieval Precision (ARP) (%)

LBP

LDP

LTP

LTrP

LEOP

30

40 35 30 25 20 15 10

25

20

15

10

5

5

0 100

200

300

400

500

600

700

800

900

1000

No. of top matches considered

(a)

0 100

200

300

400

500

600

700

800

900

1000

No. of top matches considered

(b)

Fig. 8. (a)–(b) gives performance analysis of proposed method with other existing methods in terms of ARP and ARR on ImageNet database.

able to find prominent edges in different pre-assumed directions. Pixel level tracking of intensity changes has effectively collected all the minute underlying texture information of image and hence the strong encoding of spatial information. The proposed descrip-

tor which is rotational invarient as well as illuminance independent, has achieved ARP of 31.02% for top 100 matches which is remarkable improvement in retrieval results compared to other existing methods, having ARP below 10%.

9

G.M. Galshetwar et al. / J. Vis. Commun. Image R. 64 (2019) 102615

4.1.6. Experiment 6: Performance analysis of proposed method using different distance matrices In this experiment we tested performance of our proposed method using different distance measures named as L1-distance measure, L2-distance measure, Canberra distance measure and d1 distance measure. The d1 distance measure is improved form of Canberra distance measure i.e. in denominator 1 is added to avoid divide by zero condition. Performance of the proposed method with different distance measures is shown in Table 3. This comparison shows clearly that selection of d1 distance measure will leave positive impact on retrieval results.

Input: Image Database, Output: Retrieval Results

4.2. Experiments by learning based CBIR system In Traditional CBIR systems choosing the distance measure is as crucial as the robustness of the feature extractor. The evaluation measures are highly affected by the choice of different distance measures [44]. In above four experiments we applied traditional CBIR system on handcrafted features and the analysis of the results is done. We extended our work to designing of learning based CBIR system by using ANN to check the robustness of the proposed feature descriptor. To do so, we applied the extracted features from proposed feature descriptor to Artificial Neural Network (ANN). The parametric classifier like ANN can classify many complex patterns effectively and is popularly used in many machine learning applications for classification purpose [45,46,56,57]. 4.2.1. Experiment 1: Performance analysis on COREL-10K image database using learning based CBIR framework Here, we used bi-layer feed-forward-neural-network, tansigmoid activation function at hidden layers and softmax activation function at output layer. The hidden layer neurons are equal to 2/3rd of input layer neurons. For experimentation purpose we used COREL10K dataset. We trained ANN using stochastic gradient descend back propagation algorithm for that we divided the dataset images into training (80%), testing (10%) and validation (10%) sets. On the trained network we applied traditional CBIR framework. For a given query image we carried out classification using

Table 3 Performance of proposed method using different distance measures in terms of groupwise and average arp (%). Distance metrics

Group 0

Group 1

Group 2

Group 3

Average

L1 L2 Canberra d1 Distance

86.48 82.56 86.48 86.70

73.33 69.41 73.33 73.59

74.23 64.47 75.61 75.37

71.92 66.35 71.45 72.00

79.20 73.73 79.38 79.58

LBP

LTrP

LTP

trained network and decided top N-significant probable classes and their probabilities. Based on the top N-significant probabilities we selected number of images from respective probable class randomly i.e. probability of class decides the number of images to be selected from that class. On those selected set of images, we applied CBIR framework to retrieve top matches for given query image. The results for this learning-based approach are shown in Fig. 9. From the results it can be clearly seen that the proposed energy-based learning approach outperforms the existing stateof-art methods. Algorithm for learning based CBIR is given below:

SS3D

1. Divide the input Images into training (80%), testing (10%) and validation (10%) randomly sets for all classes and extract features. 2. Train the ANN using stochastic gradient descend back propagation algorithm. 3. Input a query image. 4. Extract features for query image. 5. Classify the query image using trained ANN and record ANN scores. 6. Sort the ANN scores in descending order to decide top N Probable classes (for this experiment we selected N = 3). 7. Collect images from top N probable classes randomly (i.e. probability of class decides the number of images to be selected from that class). 8. Apply traditional CBIR framework on collected images to get retrieval results.

We have conducted experiments in two different ways, on four different publicly available databases. In the first approach, we directly used the handcrafted features for retrieval purpose. In the second approach we used learning based retrieval technique. The rigorous experimentations are carried out to prove the ability of the proposed descriptor to encode the underlying texture information of different images such as rotated images, illuminance affected images etc. This became possible, as the proposed descriptor is able to find prominent edges in different pre-assumed directions. Pixel level tracking of intensity changes has effectively collected all the minute underlying texture information of the image and later local level encoding of all the collected information to create a compact feature vector has resulted into a strong and robust feature descriptor. The performance analysis of all the 6 experiments is given in Figs. 3–9 which clearly shows that there is a significant improvement in the results, compared to state-of-the-art methods. Table 4 Table 4 Feature vector length of query image for different methods.

LEOP

100

95

ARP (%)

90

85

80

75

70

65 1

2

3

4

5

6

7

8

9

10

No. of top matches considered

Fig. 9. Gives performance comparison of proposed method with other existing methods in terms of ARP on COREL10K database.

Method

Feature Vector Length

LBP LTP LDP DLEP GLBP_u2 GLDP VLDTP ES-LBP BLK_LBP DT-CWT GLMEBP GLBP LTrP SS3D LEOP

256 2
256 4
256 4
512 12
59 12
256 1
1024 6
256 16
256 104 3
4
512 12
256 13
59 10
256 512

10

G.M. Galshetwar et al. / J. Vis. Commun. Image R. 64 (2019) 102615

Table 5 Running time of different methods for a given query image.

Acknowledgment

Method

GLDP

GLBP

LTP

LBP

LDP

LEOP

Running Time (Seconds)

3.54

0.77

0.63

0.35

0.28

0.09

shows the comparison of feature vector length of image for all the compared methods. LEOP has feature vector length of 512 only, while other methods have larger feature vector length, due to this LEOP has less computational complexity in retrieval and we could get remarkable improvement in results with reduced computational complexity. We have chosen the Feature Vector Length (FVL) for the analysis of computational complexity because FVL is independent of hardware (PC) configuration. We tried to reduce the complexity by reducing the number of comparisons to be done for given query image in our novel learning based CBIR approach. In traditional CBIR approach we obtain top best matches for query image from the entire dataset (i.e.10,000 in this case of COREL 10 K database) whereas by applying ANN learning framework we first obtained a reduced set of classified images (i.e. few hundred images) which is later used for obtaining top best matches for a given query image. Because of less computational complexity, which is further reduced in learning based approach, the proposed method is well suited for online applications with smaller number of comparisons. By keenly observing both the ways of experimentations carried out in this paper i.e. traditional as well as learning based CBIR approach, the proposed feature descriptor has proved itself very much strong and robust in different environments. We tried to apply the proposed algorithm on a bigger database (i.e. ImageNet database) and the results were 31% only for top 100 retrieved images. Even other compared methods in literature could not do well on larger database. The actual size of ImageNet database is around 15 million images, but we tested our approach on 80,080 images only (total 80 class having 1001 images in each class). The performance may degrade if we go for retrieval from millions of stored images using traditional CBIR method. That’s why we employed learning-based retrieval technique to reduce number of comparisons based on the classification probabilities. The next step may be adopting deep learningbased approach for retrieval on huge databases having millions of images. The running time for a given query image from ImageNet database is shown in Table 5. The running time depends on many factors such as computer hardware specification, number of images in database, programming skills of individual etc. We computed running time on a system with intel(R) core (TM) i7-4770 CPU @ 3.4 GHz, 8 GB RAM, 64-bit windows 7 operating system. 5. Conclusion Local energy oriented pattern encodes the relationship of pixels from 3-D grid which is extracted from image considering the first maxima and second maxima of pixel progression. The pixel level mapping of energy changes and local level spatial encoding of information between three 3
3 extracted sub grids enabled the proposed descriptor to show remarkable improvement in the performance. Proposed pattern does not directly encode local neighborhood relationship, but it encodes spatial information by monitoring pixel level energy orientations to find minute underlying texture information of an image. Our experimentations in different environments i.e. traditional CBIR and learning based CBIR, have proved the proposed LEOP as a robust feature descriptor. The significant growth in experimental results verifies the effectiveness of the proposed descriptor in image indexing and retrieval.

We would like to convey sincere thanks to Mr. Prashant W. Patil, Mr. Akshay A. Dudhane (Research Scholars), Computer Vision and Pattern Recognition (CVPR) laboratory, IIT Ropar, Punjab, India and Mr. Kuldeep M. Biradar (Research Scholar) MNIT Jaipur, Rajasthan, India, for their valuable technical discussions during this work. We would like to express our gratitude towards the anonymous reviewers whose expert insights has helped to improve the quality of this manuscript. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. References [1] T. Dharani, I.L. Aroquiaraj, A survey on content based image retrieval, in: 2013 International Conference on Pattern Recognition, Informatics and Mobile Engineering (PRIME), IEEE, 2013, pp. 485–490. [2] Y. Mistry, D.T. Ingole, Survey on content based image retrieval systems, Int. J. Innovative Res. Comput. Commun. Eng. 1 (8) (2013) 1828–1836. [3] M. Rehman, M. Iqbal, M. Sharif, M. Raza, Content based image retrieval: survey, World Appl. Sci. J. 19 (3) (2012) 404–412. [4] N. Singhai, S.K. Shandilya, A survey on: content based image retrieval systems, Int. J. Comput. Appl. 4 (2) (2010) 22–26. [5] G. Deep, L. Kaur, S. Gupta, Biomedical image indexing and retrieval descriptors: a comparative study, Procedia Comput. Sci. 1 (85) (2016 Jan) 954–961. [6] R.C. Veltkamp M. Tanase, Content-Based Image Retrieval Systems: A Survey, Dept. Computing Science, Utrecht University, Utrecht, The Netherlands, Tech. Rep., vol. 2, 2002, pp. 1–62. [7] Y. Rui, T.S. Huang, S.F. Chang, Image retrieval: Current techniques, promising directions, and open issues, J. Vis. Commun. Image Represent. 10 (1) (1999) 39–62. [8] A.W. Smeulders, M. Worring, S. Santini, A. Gupta, R. Jain, Content-based image retrieval at the end of the early years, IEEE Trans. Pattern Anal. Mach. Intell. 22 (12) (2000) 1349–1380. [9] M.J. Swain, D.H. Ballard, Color indexing, Int. J. Comput. Vision 7 (1) (1991) 11– 32. [10] M.J. Swain, D.H. Ballard, Indexing via color histograms, in: Active Perception and Robot Vision, Springer, Berlin, Heidelberg, 1992, pp. 261–273. [11] M.A. Stricker, M. Orengo, Similarity of color images, Storage and Retrieval for Image and Video Databases III, vol. 2420, International Society for Optics and Photonics, 1995, pp. 381–393. [12] J. Huang, S.R. Kumar, M. Mitra, Combining supervised learning with color correlograms for content-based image retrievalProceedings of the Fifth ACM International Conference on Multimedia, ACM, 1997, pp. 325–334. [13] E. Persoon, K.S. Fu, Shape discrimination using Fourier descriptors, IEEE Trans. Syst., Man, Cybern. 7 (3) (1977) 170–179. [14] Y. Rui, A.C. She, T.S. Huang, Modified Fourier descriptors for shape representation-a practical approach, in: Proc of First International Workshop on Image Databases and Multi Media Search, Citeseer, 1996, pp. 22–23. [15] A. Ahmadian, A. Mostafa, An efficient texture classification algorithm using Gabor wavelet, in: 2003. Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2003, pp. 930–933. [16] B.S. Manjunath, W.Y. Ma, Texture features for browsing and retrieval of image data, IEEE Trans. Pattern Anal. Mach. Intell. 18 (8) (1996 Aug) 837–842. [17] M. Kokare, P.K. Biswas, B.N. Chatterji, Texture image retrieval using rotated wavelet filters, Pattern Recogn. Lett. 28 (10) (2007 Jul 15) 1240–1249. [18] M. Kokare, P.K. Biswas, B.N. Chatterji, Texture image retrieval using new rotated complex wavelet filters, IEEE Trans. Syst., Man, Cybern., Part B (Cybern.) 35 (6) (2005) 1168–1178. [19] M. Kokare, P.K. Biswas, B.N. Chatterji, Rotation-invariant texture image retrieval using rotated complex wavelet filters, IEEE Trans. Syst., Man, Cybern., Part B (Cybern.) 36 (6) (2006) 1273–1282. [20] T. Ojala, M. Pietikäinen, D. Harwood, A comparative study of texture measures with classification based on featured distributions, Pattern Recogn. 29 (1) (1996) 51–59. [21] Z. Guo, L. Zhang, D. Zhang, Rotation invariant texture classification using LBP variance (LBPV) with global matching, Pattern Recogn. 43 (3) (2010) 706–719. [22] Z. Guo, L. Zhang, D. Zhang, A completed modeling of local binary pattern operator for texture classification, IEEE Trans. Image Process. 19 (6) (2010) 1657–1663. [23] B. Zhang, Y. Gao, S. Zhao, J. Liu, Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor, IEEE Trans. Image Process. 19 (2) (2010 Feb) 533–544. [24] M. Subrahmanyam, R.P. Maheshwari, R. Balasubramanian, Local maximum edge binary patterns: a new descriptor for image retrieval and object tracking, Signal Process. 92 (6) (2012 Jun 1) 1467–1479.

G.M. Galshetwar et al. / J. Vis. Commun. Image R. 64 (2019) 102615 [25] M. Subrahmanyam, R.P. Maheshwari, R. Balasubramanian, Expert system design using wavelet and color vocabulary trees for image retrieval, Expert Syst. Appl. 39 (5) (2012) 5104–5114. [26] S. Murala, R.P. Maheshwari, R. Balasubramanian, Directional binary wavelet patterns for biomedical image indexing and retrieval, J. Med. Syst. 36 (5) (2012) 2865–2879. [27] S. Murala, R.P. Maheshwari, R. Balasubramanian, Local tetra patterns: a new feature descriptor for content-based image retrieval, IEEE Trans. Image Process. 21 (5) (2012) 2874–2886. [28] N. Jhanwar, S. Chaudhuri, G. Seetharaman, B. Zavidovique, Content based image retrieval using motif cooccurrence matrix, Image Vis. Comput. 22 (14) (2004) 1211–1220. [29] C.H. Lin, R.T. Chen, Y.K. Chan, A smart content-based image retrieval system based on color and texture feature, Image Vis. Comput. 27 (6) (2009) 658–665. [30] M. Subrahmanyam, Q.J. Wu, R.P. Maheshwari, R. Balasubramanian, Modified color motif co-occurrence matrix for image indexing and retrieval, Comput. Electr. Eng. 39 (3) (2013) 762–774. [31] S.K. Vipparthi, S.K. Nagar, Directional local ternary patterns for multimedia image indexing and retrieval, Int. J. Signal Imag. Syst. Eng. 8 (3) (2015) 137– 145. [32] S. Murala, Q.J. Wu, Local ternary co-occurrence patterns: a new feature descriptor for MRI and CT image retrieval, Neurocomputing 7 (119) (2013) 399–412. [33] S.K. Vipparthi, S.K. Nagar, Expert image retrieval system using directional local motif XoR patterns, Expert Syst. Appl. 41 (17) (2014) 8016–8026. [34] X. Tan, B. Triggs, Enhanced local texture feature sets for face recognition under difficult lighting conditions, IEEE Trans. Image Process. 19 (6) (2010) 1635– 1650. [35] S. Murala, Q.J. Wu, Spherical symmetric 3D local ternary patterns for natural, texture and biomedical image indexing and retrieval, Neurocomputing. 3 (149) (2015) 1502–1514. [36] S. Murala, Q.J. Wu, Local mesh patterns versus local binary patterns: biomedical image indexing and retrieval, IEEE J. Biomed. Health. Inf. 18 (3) (2014) 929–938. [37] S.K. Vipparthi, S. Murala, A.B. Gonde, Q.J. Wu, Local directional mask maximum edge patterns for image retrieval and face recognition, IET Comput. Vision 10 (3) (2016) 182–192. [38] M. Heikkilä, M. Pietikäinen, C. Schmid, Description of interest regions with local binary patterns, Pattern Recogn. 42 (3) (2009) 425–436. [39] C.H. Yao, S.Y. Chen, Retrieval of translated, rotated and scaled color textures, Pattern Recogn. 36 (4) (2003) 913–929. [40] C.G. Baby, D.A. Chandy, Content-based retinal image retrieval using dual-tree complex wavelet transform, in: 2013 International Conference on Signal Processing Image Processing & Pattern Recognition (ICSIPR), IEEE, 2013, pp. 195–199. [41] S. Murala, R.P. Maheshwari, R. Balasubramanian, Directional local extrema patterns: a new descriptor for content based image retrieval, Int. J. Multimedia Inform. Ret. 1 (3) (2012 Oct 1) 191–203.

11

[42] L. Sorensen, S.B. Shaker, M. De Bruijne, Quantitative analysis of pulmonary emphysema using local binary patterns, IEEE Trans. Med. Imaging 29 (2) (2010) 559–569. [43] G.M. Galshetwar, L.M. Waghmare, A.B. Gonde, S. Murala, Edgy salient local binary patterns in inter-plane relationship for image retrieval in Diabetic Retinopathy, Procedia Comput. Sci. 31 (115) (2017) 440–447. [44] M. Kokare, B.N. Chatterji, P.K. Biswas, Comparison of similarity metrics for texture image retrieval, in: TENCON 2003. Conference on Convergent Technologies for the Asia-Pacific Region, IEEE, 2003, pp. 571–575. [45] K. Ashizawa, T. Ishida, H. MacMahon, C.J. Vyborny, S. Katsuragawa, K. Doi, Artificial neural networks in chest radiography: application to the differential diagnosis of interstitial lung disease, Acad. Radiol. 1; 6(1) (1999) 2-9. [46] M. Anthimopoulos, S. Christodoulidis, L. Ebner, A. Christe, S. Mougiakakou, Lung pattern classification for interstitial lung diseases using a deep convolutional neural network, IEEE Trans. Med. Imaging 35 (5) (2016) 1207– 1216. [47] MESSIDOR Database Available at . [48] VIA/I-ELCAP-CT Lung Image Database, Available from . [49] Corel-10K Image Database Available at . [50] J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, L. Fei-Fei, Imagenet: A large-scale hierarchical image database, in: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, IEEE, 2009, pp. 248–255. [51] G. Ding, J. Zhou, Y. Guo, Z. Lin, S. Zhao, J. Han, Large-scale image retrieval with sparse embedded hashing, Neurocomputing 27 (257) (2017) 24–36. [52] X. Lu, H. Yao, S. Zhao, X. Sun, S. Zhang, Action recognition with multi-scale trajectory-pooled 3D convolutional descriptors, Multimedia Tools Appl. 1–7 (2017). [53] S. Zhao, H. Yao, Y. Yang, Y. Zhang, Affective image retrieval via multi-graph learning, in: Proceedings of the 22nd ACM international conference on Multimedia, ACM, 2014, pp. 1025–1028. [54] K.M. Biradar, V. Kesana, K.B. Rakhonde, A. Sahu, A.B. Gonde, S. Murala, Local Gaussian difference extrema pattern: a new feature extractor for face recognition, in: 2017 Fourth International Conference on Image Information Processing (ICIIP), IEEE, 2017, pp. 1–5. [55] A.B. Gonde, P.W. Patil, G.M. Galshetwar, L.M. Waghmare, Volumetric local directional triplet patterns for biomedical image retrieval, in: 2017 Fourth International Conference on Image Information Processing (ICIIP), IEEE, 2017, pp. 1–6. [56] A.A. Dudhane, S.N. Talbar, Multi-scale directional mask pattern for medical image classification and retrieval, in: Proceedings of 2nd International Conference on Computer Vision & Image Processing, Springer, 2018, pp. 345–357. [57] A. Dudhane, G. Shingadkar, P. Sanghavi, B. Jankharia, S. Talbar, Interstitial lung disease classification using feed forward neural networks, ICCASP 2017 in Procedings of Advances in Intel-ligent Systems Research, vol. 137, 2017, pp. 515–521.