Image contrast, complexity, and stability

COMPUTER

VISION,

GRAPHICS,

AND

IMAGE

PROCESSING

26,

394-399 (1984)

NOTE Image Contrast, Complexity, and Stability* ROBERT RICH Received September 11,1982; revised March 21, 1983 Three quantities associated with an image are defined and studied. Contrast (C) and complexity (K) are defined in terms of the first and second derivatives of the image. K is a measure of the fine detail in the image. Stability is defined as C/K and is shown to measure the rate at which an image loses contrast when degraded. For the case of continuous images both local and global behavior is studied. Some discrete examples are also presented. 0. INTRODUCTION

We define and study three quantities associated with an image: contrast, complexity, and stability. Contrast and complexity are derived from the “size” of the first and second derivatives of the image, respectively. Complexity is a measure of the fine detail in the image. The quotient contrast/complexity is the stability and its reciprocal the instability. We see that high instability corresponds to a large amount of fine detail per unit of contrast. The term stability derives from a result (Section 2) showing that when an image is degraded by smoothing, images of high stability lose contrast at a relatively slow rate. In Section 3 we present some examples of discrete images. A number of photographic transparencies have been digitized and corresponding stability values computed. Several psychological experiments conducted by Mr. Vincent Gallagher of Comsis, Inc. have shown strong positive correlation between a subject’s difficulty in comprehending an image and the instability of that image. These results will be reported elsewhere (under FHWA contract no. DTFH 61-81-C00058). 1. LOCAL

THEORY

E will denote R”, Euclidean m space, and U an open subset of E. For x = (XI, . . . ,x,) in E define 1.~1~= Xx:. By an image we mean a real valued function fx defined on U and with continuous second-partial derivatives. We denote by D,fx and Djjfx the first and second partials off. The case of principal importance is that of two-dimensional images but no real simplification occurs in fixing m = 2. Intuitively we are dealing with black and white images and f(x) = a + blog B(x)

where B(x) is the brightness at x. In the case of a photographic would be the optical density at x. We define

transparency f(x)

Cfx = X~DJX~~ Kfx = XjDijfx12 Sfx =

Cfx/Kfx.

*The author acknowledges the support provided by the Federal Highway Administration, Schwab, and Mr. Gallagher. 394 0734-189X/84 $3.00 Copyright (0 1984 by Academic Press, Inc. All rights of reproduction in any form reserved.

Mr. Richard

IMAGE

CONTRAST,

COMPLEXITY,

395

AND STABILITY

We consider S to be defined (possibly infinite) except where both Cf and Kf equal 0. We call Cfx the contrast, Kfx the complexity, and Sfx the stability (off at x). We call l/Sfx the instability. Observe that Cfx is simply the norm square of the gradient off and Kfx the norm square of the Hessian off. We call a point x proper iff Sfx is defined at x. We make some observations about the above definitions. (1) Cf, Kf, and Sf are nonnegative. (2) If f is constant in an open set then Cf =

Kf =

0 and

Sfis undefined

(in that

set). (3) If f is linear and not constant then Cf # 0, Kf = 0, and Sf is infinite. (4) If g(x) = uf (x) + b then Cg = aCf, Kg = a@, and Sf = Sg. (5) If g(x) = f(x/t)

then Cg = tt*Cf,

Kg =

tb4Kf, and Sg =

t’Sf.

From (4) we conclude the stability is invariant under a linear transformation in the measurement of image density. From (5) we see that compression of the image decreases stability. Since Sf is nonnegative whenever it is defined, points for which Sfx = 0 are of particular interest. We define the unstable set off to be the closure (in U) of the set of zeros of Sf. Its points are called unstable points and its subsets are called unstable sets. We consider an example. Let E = R, V open in E, and f: V + R with f” continuous in V and f proper. (Note: for functions of one variable we will often use primes to denote differentiation.) Then the unstable points of f are exactly the relative maxima and minima of f. Also between these points will be points where f ’ z 0 and f” = 0 and at these points (inflection points) S is infinite. Now let g( x, y) = f(x) be defined on U = V x R. We can interpret f as an image consisting of vertical stripes. For example the diffraction pattern of a narrow slit. Then the lines of maximum and minimum brightness constitute the unstable set for g. We state without proof some theorems concerning stability. THEOREM. Let A # 0 be the boundary in U of f -l(c) A is an unstable set. (A is an “edge ” off .)

where c is any constant. Then

THEOREM. Assume that the gradient off is nowhere 0 in U. Let x E U and Z = f -‘fx, that is Z is a hypersurface through x on which f is constant. Let kx = sup kx( e) where e is any unit vector tangent to Z at x, kx( e) is the curvature in the direction e, and the sup is taken over all unit tangent vectors. Then (Ex)~ I l/Sfx and so high curvature (in any direction) of a surface of constant brightness implies low stability. THEOREM.

Let N(x) = 1x1,then CNx = 1

and l/SNx

= 1x1- 2( m - 1).

THEOREM. Let I/ be an open subset of (0,~) and assume g : V + R with g” continuous on V. Let U = N-‘(V), f = g . N and r = 1x1. Then l/Sfx = (l/Sg(r)) + (l/SNx) at all points where the expressions are dejmed.

The last two theorems show that if the proper points of g have 0 as a limit point then every neighborhood of 0 in E has points of arbitrarily high instability, for example, the diffraction pattern of a circular aperture. This may be compared with the theorem on curvature.

396

ROBERT 2. GLOBAL

RICH STABILITY

Our goal is to show that the rate at which an image loses contrast when degraded is determined by its (global) stability. We take an informal approach and assume conditions sufficient to make well defined any expressions formed. f will denote an image defined on all of E. We define

where the integrals are taken over E. We call cf, Ef, sf the (global) contrast, complexity, and stability off. We now let p be a radial probability density on E, i.e., p is constant on the spheres 1x1 = constant. For t > 0 define pt(x) = t-“&x/t) and observe that pt is a probability density for all t and that as t - 0, pt approaches a “delta” function. We now define L(t) = (c(f) - c(f *pt))/i?(f) w h ere * denotes convolution. L(t) is the relative loss of contrast when the image is “smoothed” by convolving it with pt. It can be shown that L’(0) = 0 and L”(0) = A/$(f) where A is a positive constant depending only on p and m. This shows that near 0 low stability implies a more rapid loss of contrast. An example of an image with low stability would be a resolution chart used for testing lens performance. Also, in many cases, obtaining optical alignment of images is equivalent to obtaining extreme values for c, K, and l/s. For example, registration of an image with itself locally maximizes these quantities. 3. DISCRETE

IMAGES

For the discrete case we consider R2 to be divided into square cells called pixels, and f is assumed to be constant on pixels. In the previous definitions, differences replace derivatives, sums replace integrals, and otherwise the definitions remain unchanged. Note that if the pixels are unit squares and f assumes only the values 0 or 1 where 0 represents white and 1 black, then c(f) is simply the perimeter of the black figure. Some examples for the discrete case follow. All figures are assumed to be black inside the boundary lines and white outside. In Example 2 potentially ambiguous outlines have been shaded to explicitly show the black areas. All figures are made up of unit square pixels and are represented by functions assuming only the values 0 and 1. For Examples 1 and 2 the nonzero values lie in a 3 x 3 square. The first figure of Example 1 corresponds to the diagonal matrix 1

0

0

0

1

0

0

0

1

Areas for the figures are given in the last column (A). For Examples 3 and 4 the

IMAGE

CONTRAST,

COMPLEXITY,

397

AND STABILITY

it

l/-i-

A

A

68

5.7

3

4

it

l/i-

A

44

3.1

5

32

2.7

9

PII

56

4.7

5

48

4.0

5

46

3.8

5

El

T

8 EXAMPLE

it 100

l/ii

A

6.3

4

All figures have c = 12.

1.

ii

l/s

A

it

l/s

A

70

4.4

7

62

3.9

7

66

4.1

6

60

3.8

8

M

cl0 q [7

88

5.5

4

;

:

EXAMPLE

All figures have c = 16.

2.

nonzero values lie in a 5 X 5 square. Fig. f of Example 3 corresponds to 1 1 1 1 1

1 0 0 0 1

1 0 0 0 1

1 0 0 0 1

1 1 1 1 1

Example 3 depicts a “closing” action. Note how Kincreases until closure is obtained

d

b

c

e

f

EXAMPLE 3

a

42

12

3.5

f

104

32

3.3

398

ROBERT

-

RICH

0

b

a

c

fig.

it

c

l/S

a

a4

24

3.5

b

84

24

3.5

c

94

24

3.9

d

36

14

2.6

EXAMPLE

d

4

at which time x decreases. Example 4 depicts another closing action. Note that E increases only when the two rectangles move close enough to “interact.” Note also the marked drop in g when contact is made. Also maximum instability occurs just before contact and maximum stability occurs upon contact. In the sample computation (for Fig. 2 of Example 2) which follows the upper matrix represents the image. The other matrices are obtained by the indicated forward (Do) and/or downward (D2) differencing operation. 0000000 0000000 0010100 0000000 0010100 0000000 0000000 8 8

16

c

28 32 28 88

K

5.5

l/S Dl

00

00

00

00

0

1

00 0 1 00 00

-1 1 00 -1 1 00 00

00 00

-1 0 00 -1 0 00 00

IMAGE

CONTRAST,

D2 00

00

00 0 0 00 0 0 00

10 -1 0 10 -1 0 00

0

0

0

1 -1 1 -1 0

0 0 0 0 0

0 0 0 0 0

Dll

0 0 1 0 1 0 0

00

00

00

00

-2 2 00 -2 2 00 00

-2 1 00 -2 1 00 00 D12

0

0

0

0

0 0 0 0 0

1 -1 1 -1 0

-1 1 -1 1 0

1 -1 1 -1 0

00

-1 1 -1 1 00

0 0 0 0

D22

00 0 0 00 0 0 00

10 -2 0 20 -2 0 10

1 -2 2 -2 1

0 0 0 0 0

0 0 0 0 0

COMPLEXITY,

AND

STABILITY

399