Quantitative evaluation of similar images with quasi-gray levels

COMPUTER

VISION,

GRAPHICS,

AND IMAGE

PROCESSING

38, 342-360 (1987)

NOTE Quantitative Evaluation of Similar Images with Quasi-Gray Levels MASAWKI Nagoya

Institute

of Technology,

I~ZUKA

Gokiso-cho,

Showa-ku,

Nagoya

466, Japan

Received November 28,1984; accepted April $1986 The mutual relationships between the visual appearance of an image and its quantitative evaluation based on various statistical computation results have not been clear, regardless of a large number of recent research efforts. The purpose of this study is to determine the most effective measures from a set of proposed texture features. In this paper, two kinds of portrait images were displayed on a CRT device by a direct gray level transform method and a modified and constrained average method. The contrast between the visual appearance and the numerical values of texture features obtained from the co-occurrence matrix and that of the Erms and SNR in an image is brought out and discussed. D I987 Academic press. hc. 1. INTRODUCTION

Many techniques for image analysis have been proposed to accomplish gray level transformations, restoration, and enhancement. On the other hand, a number of useful techniques for displaying a half tone image on a display device with limited gray levels have been investigated by Jarvis et al. [l]. The original constrained average method is more similar to the dither-coded method with regard to the idea of a threshold. But it happens that the visual appearance of computer-processed images is different from the results of the quantitative evaluation obtained by statistical computation. In this paper, the evaluation of similar and non-textural images with quasi-gray levels displayed on a CRT device is treated quantitatively. Two distinct display techniques are considered: the direct gray level transform method and the modified and constrained average method, a kind of a dither-coded method. Moreover, the influence of the contrast parameter which controls a positive or negative image is discussed, and the relationship between the visual appearance of similar color images and the histogram, the co-occurrence matrix, and the numerical values of statistical measures are investigated. 2. THRESHOLD

CONDITION FOR MODIFIED AVERAGE METHOD

AND CONSTRAINED

The mean or expectation of a random variable x with probability function f(x) is defined by the basic expression x=E(x)=(x)

= jrn x *f(x)

density

dx

(1) 342 0734-189X/87

$3.00

Copyright Q 1987 by Academic Press, Inc. All rights of reproduction in any form reserved.

EVALUATION

OF

SIMILAR

IMAGES

343

where, Pr( X = xi) = pi = H(xi)/N,; i’$ = CiH(xi), x: gray level X-: mean or expectation of gray level pi: probability f(x): probability density function N,: total number of pixel H(xi): histogram of gray level xi. We may derive the threshold condition for an original constrained average method from Eq. (1). Let us assume that a variable x is replaced by the limited maximum value X,,, and then the upper limit and the lower limit in the integral term of Eq. (1) are changed into X f K and x,, respectively. As a result, the cumulative distribution function is defined as follows:

“+Kf(x) dx = x/x,, JXl

(2)

where xt: threshold value x rnax’. maximum value of gray level K: contrast parameter. As a simple example, if the probability density function f(x) is uniform within the limited domain, i.e., f(x) = 1/(2K); X - K I x I X + K and K > 0, the following expression for the threshold value x, may be derived from the use of Es- (2). x, = K + f(i,

j) * [l - 2K/X,,]

(3)

where, f(i, j) = X: local average of gray level. This threshold value changes from K to X,, - K as the local average of gray levels in the original image data changes from 0 to X,,,,. The original constrained average method corresponds to a kind of dither-coded method, and has been introduced by Jarvis and Roberts as a new technique for displaying continuous tone images on a bilevel display device [2]. Table 1 shows a simple algorithm for the modified and constrained average method. In Step 4, there are two cases for displaying the final image on a green or color CRT device. For the case of Step 4a, i.e., 0 I f(i, j) < x,, it is possible to obtain a positive image on condition that the contrast parameter K has a negative value. On the contrary, for the case of Step 4b, x, < f(i, j) I X,,, a positive image is obtained directly under the same constraint on the contrast parameter. As pointed out in Table 1, many similar computer-generated images with different visual impressions may be displayed in marked contrast to subjective impressions of an original image. An image or picture of a positive-film type which contains the special effect of edge emphasis may be simply demonstrated from the procedure of Step 4. Note that the original constrained average method generates only one bit, i.e., a black or white dot for each pixel on a bilevel device and does so by comparing the gray level f(i, i) with the threshold value x,.

344

MASAYUKI

IIZUKA

TABLE 1 Simple Algorithm on Modified and Constrained Average Method Preprocessing operation (if necessary) gray level transform filtering technique Local average of gray level “for example”

[Step11

[Step21

f(i.

J) = (i)

X x:f(i

+ k, j + I)

Computation of threshold value “ for example” X, = K + j(i. j) X (1 - 2K/X,,) where K: contrast parameter” X,, : upper limit of gray level Representation of gray level:

[Step 31

wep 41 (4

0 0: negative image)

X, 0: positive image (K < 0: negative image)

@I

“According to a plus or minus value of the contrast parameter K. it is possible to display a positive image under the condition of (Step 4).

Table 2 shows the relation between the typical probability density function and its expression for estimating the threshold for the constrained average method. If a simple and special function f(x) satisfies the definite integral, i.e., jE,f(x) dx = 1: it is possible to analytically derive the threshold formula from Eq. (2) on the assumption that the contrast parameter K is not zero and negative value. But even for the case of a condition such as K I 0, a certain value can be numerically

Probability

TABLE 2 Density Function and Threshold Formula of Constrained Average Method

Probability density function

Threshold formula

(1) Rectangular function f(x) = 1/(2K), where 2 - K I x I X + K = 0 (else)

X, = K +

(2)

Triangularfunction

f(x) where (3)

X (1 - 2K/X,,}

= -(l/ZK*) x + (x + K)/2K2, E - K < x I jz + K = 0 (else)

Gauss distribution

function

f(x) = 1/(0&G) . exp(-(x where 0: standard deviation Note. X = f(i,

X,=K+x-(1-ZK/\IxT}

- X)‘/20*)

j): local average of gray level.

X, = x + uJT erffi(1 - 2x/X,,) = a&G/2 + z (1 - (e&)/X,,}

EVALUATION

OF SIMILAR

345

IMAGES

obtained by using Eq. (3) or the results of Table 2. As a result, we can display the computer-processed color images with quasi-gray levels under the constrained condition of Step 4 as shown in Table 1. 3. STATISTICAL

MEASURES

FOR QUANTITATIVE

EVALUATION

OF IMAGES’

Many useful approaches for extracting characteristics contained in an image data have been introduced in the book Image Modeling edited by Rosenfeld [3]. This book gives us a lot of useful information on the statistical measures. The co-occurrence matrix is essentially a 2-dimensional histogram corresponding to number of times that the pairs of gray levels occur in a specific spatial relationship. The main characteristics of the co-occurrence matrix are closely related to (a) the total number of gray levels, (b) the distance between the adjacent pixels, (c) the specific direction or angle. The co-occurrence matrix, however, bears no relation to the sampling numbers of a given digital image data in consequence. For simplicity in this study, we use a simple description M(x, y) in place of using a concrete description M( x, y; d, 8) or Me(x, y; d). The joint probability, i.e., the normalized co-occurrence matrix p(x, y) is defined by dividing each element of the co-occurrence matrix by the sum total of the same co-occurrence matrix as follows:

I+,

Y) = Mh, N=

Y; d/N

N8-1

Ng-1

c

c

= Mb,

M(x,y)=

x=0 y=o

YW 2 x=1

(4

?M(X,Y) y-1

where x; y: d: 8: N: Ng:

gray level distance angle sum total of M(x, y) number of gray levels.

For example, the numerical values of the texture features such as the contrast CON and correlation COR may be computed by using the following expression:

CON =cc [b -Y>’ vb, Y)] x c C[b- l-4*(Y -

(5)

Y

Py) *P(X, Y)]

(6)

COR =

P, = c cx x Y

Ax,

r>;

cLy= CCY’Pb,Y) x Y

‘Recently, M. D. Levine has reviewed that statistical measures for texture analysis and their problems under discussion in his book Vision in Man and Machine from the point of view of information processing.

346

MASAYUKI

IIZUKA

where p(x, y): joint probability II.,; Py: average or expected value of x and y. The contrast (or inertia) has a large numerical value when the apparent contrast of a computer-processed color image, i.e., the gray level difference between the neighboring elements (pixels) becomes high, regardless of the diagonal elements in a given

FIG. 1. Direct gray level transform method: (A) Lincoln image and histogram; (B) boy image and histogram.

EVALUATION

OF SIMILAR

IMAGES

347

co-occurrence matrix or joint probability. Note that a technical term called the inertia is used in the field of mechanics as a moment of inertia: J = Cd* . m, where d is the distance and m is the mass of an object. It seems that we may predict intuitively a large or small degree of the CON value from the distribution aspect of a gray level histogram or a co-occurrence matrix. The correlation COR is a measure of the linear relationship among the gray levels of the co-occurrence matrix in an image. It has the dimensionless value as a result, though the numerator and

FIG. 2.

llistogram.

Modified gray level transform method: (A) Lincoln image and histogram; (B) boy image and

EVALUATION

FIG.

4.

Perspective

OF

diagram

SIMILAR

IMAGES

of 8 x 8 co-occurrence

matrix

de1nominator in Eq. (6) have the special value corresponding to the square of E:w .els, respectively. The numerator in Eq. (6) is often called the second-order ten &al mcbment or the covariance, and is very important as one of the statistical measlures or descriptors which are invariant to variation in the translation, rotation, and size [41,. For further details with regard to the 14 kinds of texture features, see the E‘IG. 3. Modified gray level transform levc 4s; (B) 4 gray levels; (C) 2 gray levels.

method

based

on Fig.

2 with

a few gray

levels:

(A)

6

gray

350

MASAYUKI

IIZUKA

TABLE 3A Lincoln Data \

Y

x

0

1

2

3

4

5

6

I

0

255

9 50

4

1

80 11 10

59 398

6

1

17

19

16

51

152 37 32 20 17 4

32 48 241 51

23

21 29 15 12 2

31 250 52 20

1 7 9

10

13

36 440 80

* 3 5 3 6

5

7

15

2 3 4 5 6 1

7 3

1 1

18

10 12

13

IO 361 67

56

651

Notes. * : this symbol means “0.” 8 x 8 co-occurrence matrix (average of 4 directions) with Cont. = 1.305; the total numbers of CM. = 4009; and 14 hinds of statistical measures: (1) ASM = 0.073; (2) CON = 1.305; (3) COR = 0.875; (4) VAR = 5.558; (5) IDM = 0.800; (6) SMA = 9.867; (7) SMV = 20.767; (8) SME = 2.552; (9) ENT = 3.114; (10) DIV = 1.017: (11) DIE = 0.979; (12) IMCl = 0.473; (13) IMC2 = 0.925; (14) MCC =/////.

important and original paper by Haralick et al. [5]. It is possible to classify these texture features into the five kinds of categories from a physical point of view, though each computational formula differs from each other. It should be noted that Eq. (6) differs from the definition of the correlation coefficient: R,, = Ci[(xj - X) . (yj - J)]/ /!Zm * fim, where x, (and JQ and X (and j) are the

TABLE 3B Boy Data

0

1 2 3 4 5 6 I

377 92 22

1

7 8

22 67 95 47 22 8 4

3

19

19

11 4 3

84 556 72 25

15

13 31

5 14

2 9

5 7

44

22 55 91 55

11 17 59

11 16

16

58

13

19

129 50 22 6

12

104

20 67 241 62

1 7 8

1 9 12 17

1014

Nores. 8 X 8 Co-occurrence matrix (average of 4 directions) with Cont. = 1.553; the total numbers of CM. = 4005; and 14 kinds of statistical measures: (1) ASM = 0.103; (2) CON = 1.553; (3) COR = 0.894; (4) VAR = 7.185; (5) IDM = 0.780; (6) SMA = 9.651; (7) SMV = 27.242; (8) SME = 2.439; (9) ENT = 3.016; (10) DIV = 1.195; (11) DIE = 1.043; (12) IMCl = 0.437; (13) IMC2 = 0.903; (14) MCC =/////.

EVALUATION

OF SIMILAR

351

IMAGES

gray levels and its mean intensity value, respectively. This correlation coefficient R,, is a measure of the linear relationship between the two different images, and has a dimensionless quantity that varies between - 1 and 1. On the other hand, the special statistical measures: the root mean square (rms) error Erms and the rms value of SNR for image fidelity may be defined by the following expressions without relation to the co-occurrence matrix or the joint probability on condition that a given image data has the same sampling point NP for x and y directions,

EMU =(l/N,)d1 c [dkd - fk .dl’ i

j

SNR =,/c c [dipd*/(N,-Ems) V i i TABLE 4 Co-Occurrence Matrix (Average of 4 Directions) for Lincoln Data

1 x

Y 0

1

2

3

4

5 * 17 23 18 36 440 95 *

(4 0 1

*

*

*

*

1

791

2 3 4 5 6 7

*

62 31 35 18 15 *

58 152 37 32 20 21 *

23 37 250 52 20 15 *

;4 32 48 241 51 20 *

0

*

* * 1063 74 67 14 * *

*

*

* * * * * * * *

*

1

l0 250 52 34 * *

;s 48 241 II * *

* * 64 31 54 1752 * *

* * * * * * * *

* * * * * * * *

* * *

* * * 197 2130 * * *

* * * * * * * *

03

(4

* * * * *

2 3 4 5 6 7

*

0

*

1

*

2 3 4 5 6 I

* * * * *

* * * * * *

1446 221 * * *

6

7

*

83 1134 *

* * * * * * * *

* * * * * * * *

* * * * * * * *

* * * * * * * *

* * * * * * * *

11 14 13 18

Notes. *: this symbol means “0.” (a) Erms = 0.656; SNR = 6.599; CONT. = 1.074. (b) = 1.285; SNR = 3.101; CONT. = 0.588. (c) hs = 2.056; SNR = 1.161; CONT. = 0.106. Erms

352

MASAYUKI

IIZUKA

where NP: pixel number for x (or y) direction original (input) gray level processed (output) gray level.

f(i, j): g(i, j):

Both Erms and SNR have been widely used as a measure of system quality in some signal transmission systems. If we consider the difference between an original gray level and a computer-processed gray level to be noise or error, the right term of Eq. (7) or the denominator of Eq. (8) corresponds to the noise. Generally, there is a reciprocal relation between the two with regard to the numerical values. 4. SIMILAR

AND NON-TEXTURAL QUANTITATIVE

IMAGES WITH QUASI-GRAY FEATURE EVALUATION

LEVELS AND

In this study, the coded data with 64 x 64 pixels and 32 gray levels is used as the original and standard test data [6]. The coded characters relevant to each gray level are from “0” through “9” and “A” through “V,” and the number of gray levels becomes 32 in total. Figures 1A and B show the quasi-color portrait images and its histograms with 64 x 64 pixels and 8 gray levels. The gray scales of the displayed quasi-colors are composed of the black (0), blue (1) red (2), purple or magenta (3) green (4) light

aA t

co

Erms Erms

*7*I

(a)

FIG.

(b)

Cc) Cd) (e) Images

(a)

(b)

(cl Cd) Images

(e)

5. Statistical measure for five different images: (A) Lincoln data: (B) boy data.

EVALUATION

OF

SIMILAR

353

IMAGES

blue or cyan (5), yellow (6), and white (7), respectively. Each numerical vah ie in pare ntheses stands for the quasi-gray level of the computer-processed images. Fi gures 2A and B show the similar results in contrast with Fig. 1, but these resuits were : displayed after the direct gray level transform according to y = FIX(x, ‘4 + 0.5), where y is the output gray level and x (0 I x I 31) is the input gray level. The role of the FIX function is to transform a real number into an integer. It is pos sible to siimply change the visual aspects in relation to the dark (black) and/or light

FIG. 6. 4b.

Modified

and constrained

average

method:

(A) algorithm

of Step 4a; (B) algorithm

of Step

354

MASAYUKI

IIZUKA

(white) parts of the standard images by using a convenient function for the direction transformation of gray levels. Figures 3A, B, and C show the displayed results after the simple gray level transformation using the data of Fig. 2A as the standard image. In contrast with Fig. 1 or 2, the visual appearance and its features among the Lincoln images with quasi-gray levels are very intuitively clear, judging from each gray level histogram. The histogram of Figs. 3A and B is similar in distribution to that of Fig. lB, though the effective domain of gray levels differs from each other. In these cases both values of end sides of gray levels are distinct and each histogram shows a concave type. As a result, it seems that a spatial filtering operation using a 3 x 3 mask for the high pass filter is performed. Note that the effect of edge or contour line enhancement is highlighted by means of the direct gray level transform method in place of carrying out directly the discrete convolution processing such as g(4 j) = C,C,[WG 0 . f(i + k j + 01, where h( k, 1) is termed as a mask or spatial filter. Figures 4A and B demonstrate the perspective diagrams of the 8 X 8 co-occurrence matrix in connection with the results of Fig. 1A and Fig. 2A. In Fig 4 the scale of the vertical axis may be appropriately normalized for displaying this figure as a matter of convenience.

Lincoln

Data

SNR

I .-

.

(Broken

Line,

(Solid

Line)

.K -1 Contrast

FIG.

7.

Statistical

0

1

2

3.5

7

Parameter

measure

for similar

images.

EVALUATION

OF

SIMILAR

355

IMAGES

Tables 3A and B show the typical representation of the 8 X 8 co-occurrence matrix and its statistical measures containing the texture features for the case of Fig. 2. This matrix denotes the average of the four specified directions, i.e., 0 O, 45 O, 90 O, and 135 O, and an asterisk symbol “ * ” stands for an element with the numerical value “0.” The main abbreviations for a number of texture features contained in Table 3 are: (1) the angular second moment, (2) contrast, (3) correlation, (4) variance, (5) inverse difference moment, (9) entropy, etc. The physical meanings of the above texture features have been already commented by Iizuka et al. [7]. Tables 4A, B, and C demonstrate the similar co-occurrence matrices in connection with the results of Fig. 3. Each element in the matrix concentrates on the central part, and the numerical values of the statistical measures such as CON, Erms, and SNR differ from each other. In case the denominator of Eq. (8) becomes almost constant, the multiplication of Erms and SNR together approaches a similar value such as 4.33, 3.98, and 3.62 regardless of the results of each co-occurrence matrix shown in Table 4. Figures 5A and B show the characteristic curves for the typical statistical measures for the five kinds of quasi-color images. These computer-processed images correspond to the Lincoln (or boy) data of Figs. 1,2, and 3, respectively. Each curve of the eight kinds of statistical measures shows almost the same tendency and characteristics regardless of the two different and original images such as the

;>. 1

:.’ -1.

!-

Local

2

3

4

Average

5 of

Gray

6

7 Level

I

FIG.

8.

Threshold

function

for modified

and constrained

average

method.

356

MASAYUKI

IIZUKA

Lincoln and boy data. But the curves of the statistical measures change considerably in value among the similar images except for COR and IDM. The two curves of Erms and SNR for image fidelity have the contrary tendency as predicted by Eqs. (7) and (8). Figures 6A and B show the quasi-color portrait images displayed by the modified and constrained average method under the condition of Steps 4a and b in Table 1. The difference in appearance among the similar portraits is emphasized and the

FIG. 9. Modified and constrained average method by algorithm function; (B) triangular function.

of [Step 4b]: (A) rectangular

EVALUATION

OF SIMILAR

357

IMAGES

subjective appearance of each image is considerably different in comparison with the results of Figs. 1, 2, and 3. In regard to an edge or outline part, the noticeable features are discernible owing to the effects of the contrast parameter K. Figure 7 shows the characteristic curves for the special statistical measures of the change in the contrast parameter. It should be noted that the broken and solid lines show the results obtained by means of the different condition of Step 4 in Table 1. An alteration of the contrast parameter gives us the important influence on the visual appearance of similar images. The two kinds of curves for CON show the asymmetrical tendency according to a certain change in the contrast parameter. Both Erms and SNR are more similar except in contrast with CON. Figure 8 shows the special curves for the threshold value to the local average of gray levels which is indispensable in carrying out the modified and constrained average method. Note that the curves are plotted by using the results of Table 2, and the threshold value for K = 0 is equal to the local average of gray levels, and the threshold x, = X holds. Figures 9A and B show the results obtained by changing the computational formula for the threshold value. According to a plus or minus specified value of the

Lincoln

-7

Data

-3.5

-1 Contrast

FIG.

Step 4b.

0

1

2

3.5

7

Parameter

10. Statistical measures among similar images: (A) algorithm of Step 4a; (B) algorithm of

358

MASAYUKI

IIZUKA

B

[Step Lincoln

'

iAt

4-b] Data

I

\ CON

FIG. lo-Continued.

contrast parameter, the different hinds of images may be simply displayed. These displayed results contain the two types of images corresponding to the negative and positive prints in the same photo for all views. All the images are, moreover, more or less different from each other except for the contrast parameter K = 0. As far as the apparent and visual features of the similar portrait images are concerned, they are clearer in comparison to the displayed results as shown in Fig. 6. Figures 10A and B show the characteristic curves obtained by considering the condition of Step 4 in Table 1 for the case of the results of Fig. 9A. It is difKcult to predict the visual appearance of the similar portrait images from the results of Fig. 10 intuitively. But a few, special kinds of texture features such as CON, VAR, SMA, etc. are very useful to identify the distinction of the similar images with quasi-gray levels quantitatively, because the values of the statistical measures change along with that of the contrast parameter in a great degree. 5. CONCLUSIONS

The author performed his study on the quantitative evaluation techniques among the similar and non-textural portrait images with quasi-gray levels, and obtained the

EVALUATION

OF

SIMILAR

FIGURE

A.1

FIGURE

A.11

IMAGES

359

MASAYUKI

360

IIZUKA

following results: (1) It is possible to display similar images with a different image quality by altering the specified contrast parameter and/or the threshold formula for the modified and constrained average method. (2) It is very useful to make use of some formulas of the texture features and that of Erms and SNR for image fidelity in order to extract quantitatively a noticeable difference among the computer-processed images. (3) A few kinds of texture features, e.g., ASM and IDM; VAR and SMA, show almost the same property and tendency, though the computational formulas are distinct from each other in a physical dimension of gray levels. APPENDIX

The original photographs, i.e., Figs. 1-4, 6, and 9, are processed by using a color CRT device which may display the eight different quasi-color levels. Note that all of the red colors in color prints are changed to black in these figures. As a reference of monochrome prints, Figs. AI and AI1 are originally reproduced in black and white in comparison with Figs. 2A and 9A, and they are very different as far as the image quality. ACKNOWLEDGMENTS

I would like to thank Mr. K. Takeuchi at Fujistu Limited who has prepared a computer program for computing the texture features connected with a co-occurrence matrix, and last of all thank Professor L. G. Shapiro for giving me encouragement and valuable comments in order to accomplish this paper. REFERENCES 1. J. F. Jarvis, C. N. Judice, and W. H. Ninke, A survey of techniques for the display of continuous tone pictures on bilevel displays, Comput. Graphics Image Process. 5, 1976, 13-40. 2. J. F. Jarvis and C. S. Roberts, A new technique for displaying continuous tone images on a bilevel display, IEEE Trans. Comput. C-24,1976, 891-898. 3. A. Rosenfeld (Ed.), Image Modeling, Academic Press, New York, 1981. 4. M. K. Hu, Visual pattern recognition by moment invariants, IRE Trans. hf. Theory IT-S. 1962, 179-187. 5. R. M. Harahck, K. Shamnugam, and Its’hak Dinstein, Textural features for image classification. IEEE

Trans.

Syst.

Man

Cybern.

SMC-3,1973,610-621.

R. C. Gonzalez and P. Wintz, Digital Image Processing, Addison-Wesley, Reading, Mass., 1977. 7. M. Iizuka et al., Estimation of image quality of quasi-half tone image by means of co-occurrence matrix and its statistical amount of information. Bull. Nagoya Inst. Technol. 35, 1983, 171-183

6.