Preprocessing of Degraded Images to Augment Existing Restoration Methods

COMPUTER GRAPHICS AND IMAGE PROCESS/NO (1975)4, (25-39)

Preprocessing of Degraded Images to Augment Existing Restoration Methods M. J. MCDONNELL AND R. H. T. BATES Department of Electrical Engineering, University of Canterbury, Christchurch, New Zealand Communicated by T. S. Huang Received October 17, 1974 We report the restoration by inverse filtering of degraded images, truncated by finite recording frames. We smoothly extrapolate the edges of the truncated degraded images throughout the regions of image space which they would have occupied if there had been. no truncation. Our procedures rely on the circular property of the fast Fourier transform algorithm. We develop a comprehensive notation for image restoration. We present examples of faithful images recovered from degraded images whose distortions are due to onedimensional and two-dimensional point spread functions and to recording noise.

1. INTRODUCTION

Advances in computer technology together with the availability of efficient algorithms such as the fast Fourier transform (FFT) [1] have led to increased interest in the digital processing of degraded images. Recent progress is reviewed by Huang, Schreiber, and Tretiak [2] and Sondhi [3]. Applications of image processing are found in, for instance, medical technology [4], space exploration [5], electron microscopy [6] and astronomy [7]. Existing approaches to image restoration can be classed as either recursive [8] or nonrecursive [3]. This paper is concerned with the latter as applied to inverse filtering. Our interest is in images which have been degraded by processes that can be represented by convolution of the images with point spread functions (psf). We consider monochromatic, planar images. We define an image as a real, nonnegatire distribution of intensity over the image plane; and we restrict the psf to be a real, nonnegative, spatially invariant function of the image plane coordinates. In the real-world it is only possible to record things of finite size, i.e., an actual degraded image can only be recorded within a finite recording frame. It is usual for a degraded image to occupy a larger region of the image plane than that occupied by the original, undegraded image. As in a previous study [9], we consider the special class Y of degraded images which fit completely inside their recording frames; and the general class f# of degraded images which are truncated by their recording frames. Reported restoration methods [2,3,9] usually work adequately with images of class Sa. They often give unacceptable results with images of class ~, however. The truncation of the degraded image by the recording frame tends to introduce artefacts into the restored image. We introduce here a new technique (for restoring images of class f~) called the 25 Copyright© 1975by AeademlePress, Inc. All rightsof reproductionin any formreserved.

26

MCDONNELL AND BATES

"edge extension" method. The recording frame is enclosed by a larger frame called the preprocessing frame, and the space between between the edges of the frames is filled with a distribution of image intensity. We show how to choose this distribution so that the '*extended image," filling the preprocessing frame, is an image of class Sa. This is our "preprocessing." We now restore the original image by inverse filtering, using a Wiener filter [2]. In Section 2 we develop a comprehensive notation. We show in Section 3 why unaided inverse filtering is often inadequate. We present our edge extension procedures in Sections 4 and 5, where we also report examples of actual image restorations. In Section 6 we comment on the significance of our results. 2. PRELIMINARIES

As well as image space J , in which all our images are represented by nonnegative real functions, we require Fourier space a~-. As indicated in Fig. 1, 3' denotes an arbitrary point, with Cartesian coordinates x and y, in ~ ; and o~ denotes an arbitrary point, with Cartesian coordinates u and v, in ~ . We denote by f = f ( x , y ) the final degraded image appearing within the recording frame F, which is a simply connected domain of J . The original image, which we hope to recover from f, we denote by p = p ( x , y ) . We imagine the degradation of the image to take place in two steps. There is the preliminary degradation, which is the distortion (denoted here by = N ( p } ) i n t r o d u c e d by the physical process responsible for degrading the original image. Any nonlinear response of the recording medium is included in ~ . The second degradation is the addition to N of recording noise n = n (x,y). Here we only treat the type of degradation which can usefully be approximated by convolution of p with a psf. We define the ideal degraded image b -.= b ( x , y ) by b=p

(1)

®h,

where ® denotes convolution and h = h (x,y) is the psf. In certain situations the degradation is effectively one-dimensional, which means that the psf assumes the form h ( x , y ) = h ( x ) 8(y),

Y

(2)

vk

~.

(~,v)

X

Fro, 1. Image space J and Fourier space ~'.

PREPROCESSING DEGRADED IMAGES

27

where 8(.) denotes the Dirac delta function (we always take a one-dimensional degradation to exhibit its variation in the x-direction). In order to express f in a form convenient for later manipulation we introduce the truncated ideal image, t = t(x,y), which is the part of b within F: t

b, =0,

~,~r, ~,~F.

(3)

I f f were identical with b, we could restore p perfectly by inverse filtering (see Section 3). In Sections 4 and 5 we show how to estimate b from f , on the assumption that the latter is the best available estimate of t. Consequently, we refer to all parts o f f , other than t, as the contamination c = c(x,y), so that f = t + c,

(4)

where

c=~{p}-b+n,

3, ~ F

(5)

We introduce the classes 5a and ~¢ of degraded images, defined by f ~ ~T f f f ~¢

if if

t -~ b; t ~ b.

(6) (7)

The effect of h is to blur p in such a way that each point in p is spread out over a region of J which we call the "domain of influence." A primary characteristic of point-spread-invariant degradations is that the domains of influence of all points in p are congruent. Since the parts of [h I below some threshold level produce no sensible blurring, the domain of influence is effectively finite. We define the point spread frame Y as the domain of ~ throughout which h is significant (to ensure that Y has a convenient shape, it is sometimes necessary to stretch the definition of Y by including within it regions of 3¢ where h is negligible). We see that the domain of influence of each point in p is a domain congruent to Y but with its center transferred to the said point. We require two further frames for our preprocessing of degraded images. For defining these frames it is convenient to introduce a frame function .o-ca, itself defined by

2,{¢}

0,

(8)

where ~ is a finite domain of J . The first preprocessing frame f~ is the largest domain of J that can be occupied by an original image, the domains of all of whose points intersect F, i.e., [ ¢ O, 2 ' {r} ® ~e {Y} [= O,

.y ~ f~, • ~ a.

(9)

The second preprocessing frame A is the domain of~" occupied by b when p fills 12, i.e.,

2, (st} ® 2, (Y}

o,

A.

28

MCDONNELL AND BATES

We note that F C ~ C A.

(11)

We find it necessary to define the "extent" of a frame in an arbitrary direction. We take Cartesian coordinates ¢ and 7, inclined at an arbitrary angle to the coordinates x and y in J . We construct in g two straight lines, parallel to the T-axis. We say that the extent of ~ in the C-direction is the greatest separation of the lines such that they are tangent to, but do not intersect, ~. It is often convenient for frames to be rectangular. We can usually choose the sides of the rectangles to be parallel to the x- and y-axes, in which case we say a frame ~ is rectangular (a ×/3) if its extents in the x- and y-directions are a and /3 respectively. We say that ~ is centered-rectangular if its center coincides with the origin of the x,y-coordinates. Figure 2 shows 11 and A for two types of psf, with F centered-rectangular (A × A'). Figure 2a relates to a one-dimensional degradation, as in (2), and Y is centered-rectangular (L x O), where L is the extent of h in the x-direction. In Fig. 2b, Y is centered-rectangular (L X L'). w e identify the Fourier transform (FT) of an image by the upper case form of the English letter which denotes the image, e.g.,

F(u,v)= ( ( f ( x , y ) d FJ

exp(i2zr[ux+ vy]) dx dy.

(12)

Similar notation is used for the psf and its Fourier transform H = H(u,v). When the degradation is one-dimensional, so that (2) applies, H is a function of u only:

H(u) = (uz h(x) exp(i2erux) dx,

(13)

J--LI2

where the interval --L/2 ~ x <~L/2 is always used for Y when the degradation is one-dimensional (e.g. refer to Fig. 2a). We normalize the psf such that

at

--alN

&

_~_., ,

I

~xA'

@ I A*L A*2L

FIG, 2. Examples of the frames fl and A when F is centered-rectangular (A × A'). (a) Y is centered-rectangular (L x O). (b) Y is centered-rectangular (L x L').

29

PREPROCESSING DEGRADED IMAGES C

H(0,0) = ~ J h(x,y) dx dy--1 d

Y

(14)

d

for two-dimensional degradations, and

H(O) ~- f L/2 h(x) dx = 1

(15)

d--LI2

for one-dimensional degradations. 3. RATIONALE Given f, we wish to obtain an estimate/3-- ~(x,y) of the original image. In the procedure known as "inverse filtering" [2],/3 is taken to be the inverse F T of F H , where we call/:/the "inverse filter" for the following reason: When f E 5: and c = 0 it follows from (1), (3), (4), (6) and the convolution theorem that /3=p

if

H = 1/H.

(16)

We use h, which we call the "inverse psf," to denote the inverse FT of H. We define the inverse point spread frame -Y in terms of 7~, in the same way as we define Y in terms of h in Section 2. Since H is often small in regions of ~- where F is significant, the simple form (16) cannot usually be used for H in practice, since contamination is always present. A form for H which is often suitable is the Wiener filter [2]

H = H*/(HH* + W2),

(17)

where the asterisk denotes the complex conjugate and W = W(u,v) is an a priori estimate of the ratio of ICI to ]PI. The resulting :3 minimizes [3]

f n f lf~(x,Y) -P(x,y)l~ dx dy. When f ~ fY, there is more contamination near the perimeter than near the center ofF. This cannot be compensated for by W, which depends only upon the spatial frequency content of c and p. Huang, Schreiber, and Tretiak [2] argue that if the extent of Y was considerably less than the extent of F, in all directions, then/3 would be likely to represent p faithfully throughout F, except close to its perimeter. This would mean that p could be recovered throughout most of F by Wiener filtering. We define (see Section 2) h,c, and b such that they occupy finite domains of •, implying that H,C, and B are entire (integral) functions of exponential type, which necessarily possess analytical continuations for all finite, complex values of u and v [10]. We replace the Cartesian coordinates u and v by the polar coordinates p and ~b, so that we can write

H(u,v)=H(p;d))

; W(u,v)=W(p;qb)

X = X(p;~b) = HH* + W z.

;

(18) (19)

Since H and W are entire functions (of exponential type) of p, for any ~b in the range 0 ~< ~b < 27r, it follows that X is such a function. Consequently, X is zero at a denumerable infinity of points in the complex p-plane [10]: On referring to

0

_..4.L 0 2

~ 2

x

0

#

_~

,,

o

2 X

_A

o

2

~. 2

+ 0

Fro. 3. Inverse psf for a particular one-dimensional degradation. In each of our one-dimensional examples we take 128 sampling points in the x-direction. (a) h ( x ) , --L/2 < x < L/2. For --A/2 < x < A/2, (b) h(x), W = 0; (c) h(x), W-----0.003; (d) h(x), W = 0.03.

PREPROCESSING D E G R A D E D IMAGES

31

(17), it is therefore seen that H must possess complex poles, so that the area of Y is actually infinite, although it is often effectively finite because h can be negligible outside a finite domain of Y. However, the extent of Y must be much larger than the extent of Y in at least one direction. To illustrate this, we show in Fig. 3, for a particular one-dimensional degradation characterized by the h shown in Fig. 3a, the form of h for three different constant values of W. As W increases, h decreases markedly in parts of the range - A / 2 < x < A/2, but it is by no means all concentrated close to x = 0. This example confirms our other computational experience, which suggests strongly that few psf satisfy the conditions necessary for the aforementioned conjecture by Huang, Schreiber, and Tretiak [2] to be useful. We see therefore that a direct application of Wiener filtering is often inappropriate w h e n f E ~7. There are two further points which apply when F is rectangular, which is the case of most practical interest because the F F T can be used (with a consequent increase in computational efficiency): (i) Using the F F T to invert FH leads to a/3 existing in F; whereas p occupies a domain of J larger than F. In fact, p can fill ~. So, as well as being only an estimate of p, we see that /5 is necessarily incomplete. (ii) Before the F F T can be used to transform f , the latter must be sampled. Now, the F F T is said to be "circular," in that it is predicated upon the samples being taken from a periodic function which repeats itself throughout image space within contiguous rectangular domains congruent to F. This introduces no difficulties when f E S~, because f is everywhere zero (apart from contamination) on the perimeter of F. When f ~ f~, however, the circular property of the F F T can have the effect of m a k i n g f discontinuous on the perimeter of F, thereby distorting F. After we multiply by H and take the inverse FT, these spurious components of F introduce artefacts which tend to mask the true detail in p. In Sections 4 and 5 we show that we can largely overcome the disadvantages noted in (i) and (ii) above by constructing an estimate/3 = / ; (x,y) of b before attempting to recover/~ by inverse filtering. We must comment here on Gennery's [11] restoration of a class f¢ degraded image. He examines a one-dimensional degradation, for which the extent of h is much less than the extent of F, in the xdirection. So, the difficulty noted in (i) above is scarcely apparent. He side-steps the difficulty noted in (ii) above by assuming a large average value of W, thereby resigning himself to a restored image of inferior quality. 4. ONE-DIMENSIONAL DEGRADATION When the degradation is one-dimensional, (2) applies, which means that (1) can be written as

b(x,y) = p(x,y) ® h(x)

(20)

so that it is convenient to process the degraded image along straight lines (on each of which y is a constant) in image space. In this section we consider how, knowing h (x), we can obtain f r o m f ( x , y ) an estimate/~ (x,y) of b (x,y), for a par-

32

MCDONNELL

A N D BATES

ticular value "0 of y, given that F is centered-rectangular (A x A') and Y is centered-rectangular (L × O) ; w h e r e f E ~¢ and 1~71< A'/2. We now introduce our edge extension methods. We illustrate them with an example based on the truncated ideal degraded image shown in Fig. 4a. In this section our main purpose is to explain our methods, and we find it convenient to do this without introducing the extra complication of contamination on top of

I

I

_~ 2

o

-

I

2 x

b.~ o

I __A. 2

I 2

I

I

- 2A

0

A2

:

l-

.....

2

- -

I 2

~*1.

I 2

X

Fi•. 4, A n example of the edge extension method. (a) t(x,~), --A/2 < x < A/2, i.e., 3' E F. For - - A / 2 - - L < x < A / 2 + L, i.e.,3' ~ A, (b) - : b(x,'0);---: b(xaT). F o r - ( A + L)/2 < x < (A + L)/2, i.e., 3' ~ ft, (c) - : p ( x , ~ ) ; ---:/3(x,~).

PREPROCESSING DEGRADED IMAGES

33

t(x,'o). The example presented in Section 5 demonstrates how our methods behave when recording noise is present. We note that the curve shown in Fig. 4a is the intensity of t(x,~) along the part of the dashed line in Fig. 2a which lies in Y. We obtained t(x,~) by truncating the b(x,'o) shown as the solid curve in Fig. 4b. This b(x,'o) is the convolution of the p(x,~) shown as the solid curve in Fig. 4c with the h(x) shown in Fig. 3a. In practice, recording noise is always present, so it is convenient for us to think of the t(x,'o) shown in Fig. 4a as if it were our given data, i.e., f(x,'o). Since we know h(x), we know L, so that we must assume that b (x,~9) has significant value throughout - - A / 2 - - L < x
b(-A/2 -- L,~) = b(A/2 + L,~q) = O, b(-+-A/2,~l) -----f(+_A/2,,/),

(21) (22)

and, unless the intensity of the original image p(x,'o) and/or its derivatives with respect to x are discontinuous withinA/2 < Ixl < (A + L)/2 (i.e., in the part of 12 which lies outside F), then b(x,~) must be well behaved along the dashed line in Fig. 2a. We therefore assume, because it is reasonable to do so and also because there is no alternative available to us, that b (x,~) is an analytic function throughout A/2 < Ix I < A/2 + L. So, we extrapolatef(x,~q) smoothly (by which we mean that we match its first few derivatives at x =-- +_t_A/2)throughout A to obtain our estimate b(x,~) of b (x,'o). The dashed curve in Fig. 4b was calculated by fitting a polynomial of order 2 to each edge of the t(x,'o) shown i n Fig. 4a. Having obtained/9(x,~)) we are ready to employ inverse filtering to compute an estimate /3(x,~) of the original image. There is no recording noise on the degraded image shown in Fig. 4a, so that W would be arbitrarily small if b(x,~) were an arbitrarily accurate estimate of b (x,~). But b (x,~7) is not identical with b (x,~), so that we require W to be finite to ensure that IBHt does not become excessively large for those spatial frequencies for which H is very small (refer to (17)). Our computational experience suggests that a constant value of 0.03 for W is usually satisfactory. We are aware of the danger of attempting to "optimize" the form of W by altering it in some iterative manner until a "best" result is achieved. Consequently, we feel that the only "honest" procedure is to fix on a particular W a priori. W h e n the recording noise level o n f ( x , ~ ) is known then we are, of course, permitted to adjust W accordingly. The dashed curve in Fig. 4c shows the result of using the F F T to compute the inverse F T o f / } H , with W = 0.03. The dashed curve in Fig. 4c is a poor estimate of the solid curve at the edges. There are two reasons for this. The first is that, because we use the F F T for computational efficiency and economy, our p(x,~q) exists throughout A, whereas it should only exist throughout ~ . The second reason is that, although we obtain b(x,'o) by extrapolatingf(x,~)) smoothly, there is no guarantee of the existence of a function (of extent (A + L), such as the p(x,y) in (20)) which when convolved with h(x) gives b(x,y). a

34

M C D O N N E L L A N D BATES

We improve our estimate of the original image by making use of the circular property of the F F T (already discussed in Section 3). We introduce the periodic function e(x,~) defined by

e(x,~q) = ~

(23)

b(x-- m[A + L],~),

which is seen to be of period (A + L). We sample e(x,~) within the range --(A + L)/2 < x < (A + L)/2 and use the F F T to compute its one-dimensional FT, which we denote by E(ul~). Because e(x,~7) is periodic, E(ut~q) only exists when u = u,~, where

um= m/(A + L),

(24)

where m is any positive or negative integer or zero. Consequently, the restored image, which is obtained by using the F F T to compute the inverse F T of E(u[~) H(u), is periodic of period (A + L) ; i.e., our estimate of the original image now exists in ~2, thereby overcoming the first disadvantage of the simple edge extension method. We now recall that b (x,r/) is of extent (A + 2L), because it exists throughout A. But e(x,'o) is of period (A + L). So the edges of the repeated versions of b (x,~) overlap in e(x,~). Specifically, the parts of b (x,~7) along the dashed line in Fig. 2a (lying in A but outside l~) overlap within the intervals (2m+l)A/2
(2m+I)A/2+L,

where m is any positive or negative integer or zero. Consequently, we call e (x,a~) the ideal overlapped degraded image. We now note that the circular property of the F F T ensures the existence of a function, itself periodic of period (A + L), which when convolved with h(x) gives e (x,~). This allows us to overcome the second disadvantage of the simple edge extension method. If either p ( - ( A + L)/2,~) or h(L/2) is zero and if either p((A + L)/2,~) or h(-L/2) is zero then

Ob(+(A/2 + L),~)/Ox = O,

(25)

as inspection of (20) confirms. We assume that the intensities of all original images fall to zero on the perimeter of g~, so that (25) is always true (note that (25) is satisfied however large [Op/Ov[is on the perimeter of fZ, where the v-direction is the normal to the perimeter, provided that p = 0 everywhere on the perimeter). It then follows from inspection of (23) that e(x,~) and Oe(x,~q)/Ox are continuous at x = +__.4[2. Invoking the assumptions concerning b (x,y) which we make for the simple edge extension method, we see that

cg'*e(--(A + L)/2,~)/Ox" = O"e((A + L)/Z,~)/Ox n for all positive integers n. We obtain an estimate

b(x,~q)

of

e(x,~),

within

(26) the

interval

(/1 + L)/2 < x < (A + L)/2, by extrapolating f(x,'o) into the intervals --(A+L)/2
-

PREPROCESSING

0

I

I,

2

2

I, i . A(At.k) _ A 2 2

DEGRADED

,

35

IMAGES

~

J,,,

l o

i

2

2

t

t

_A. tA~.L) 2, 2

x

F l o . 5. A n e x a m p l e o f t h e o v e r l a p p i n g e d g e e x t e n s i o n m e t h o d . F o r - - ( A + L ) / 2 < x < (A + L ) / 2 ,

i.e., ~, E a, (a) - : e(x,'o), ---: ~(x,~) ; (b) - : p(x,'o), ---: ~(x,'~). first derivatives continuous atx = +__A/2. We satisfy (26) with e replaced by 8, for as many values of n as we feel is warranted by the quality of our data. The solid curve in Fig. 5a shows e(x,'o), which is computed from the b (x,~) shown in Fig. 4b. T h e dashed curve in Fig. 5a shows ~(x,~) obtained (with (26) satisfied for n = 1 only) from the t(x,~7) shown in Fig. 4a. The dashed curve in Fig. 5b is the estimate of the original image obtained by inverse filtering, using a value of 0.03 for W, of the ~(x,~) shown in Fig. 5a. T h e dashed and solid curves of Fig. 5b are gratifyingly similar, suggesting that our overlapping edge extension method overcomes the deficiencies of both direct inverse filtering and simple edge extension followed b y inverse filtering. 5. TWO-DIMENSIONAL DEGRADATION In this section we show how to apply our edge extension method to an image which has suffered a two-dimensional degradation. In Section 4 we indicate that the overlapping method is superior to the simple method for one-dimensional degradations. The same is true in two dimensions because the F F T retains its circular property, provided that the function being transformed is defined on a rectangular grid of points. We can therefore dispense with the preprocessing frame A.

36

MCDONNELL

AND BATES

In general, Y is not rectangular when it covers only that part of ,fi where h is significant, which means that f~ is not, in general, rectangular even if F is (refer to (9)). We could always choose F such that f~ is rectangular, but this would involve us in a tedious deconvolution process to determine F, given Y. We find it more c o n v e n i n e n t to take F to be rectangular and to include in Y parts of J where h is negligible, so that Y itself can be rectangular. It is then straightforward to extrapolate f into f~. We define F and Y to be centered-rectangular (A × A') and (L × L') respectively, as in Fig. 2b. We define our ideal overlapped degraded image e(x,y) by

e(x,y) =

~ m,ll~--

b ( x - m[A + L ] , y - - n[A' + L ' ] ) ,

(27)

co

which we estimate, within f~, by extrapolating f ( x , y ) into the part of f~ outside F. We make ~(x,y) and its first derivative (normal to the perimeter) continuous on the perimeter of F:

~(±A/2,y) = f(+-A/2,y) }--A']2 < y < A'/2; a~ (±A /2,y ) /ax = a,f(+---A/2,y ) lOx ~(x,+_A'/2 ) = f(x,+_~A'/2 ) a~(x,+_A'/2)/Ox = Of(x,±A'/Z)JOx] --A/2 < x < A[2.

(28)

(29)

W e assume that b(x,y) is analytic in the part of gl outside F, so that all derivatives o f e(x,y) exist on the perimeter of fL T o extrapolate f ( x , y ) we use as many o f these derivatives as we feel our data warrant. Figure 6a shows an original image before it suffers a computer-generated degradation which results in the truncated ideal degraded image shown in Fig. 6b. W e set

A'=A,

L'=L;

A/L=9.

(30)

T h e psf is

h(x,y) = 4/(zrLZ), = 0,

(x2 + y 2 ) 112 ~< L[2, (x 2 + y2)ljz > L[2,

(31 )

which is a fair approximation to the psf of an out-of-focus optical system imaging incoherent radiation [11]. We see from (31) that h is zero in the part of Y which lies outside the circle of radius L/2, centered at the origin of coordinates in .f. To restore the image shown in Fig. 6b, after recording noise is added to it (so that it becomes f ( x , y ) rather than t(x,y)), we extrapolatef(x,y) from F into in the following way. F o r a particular y in the range lY] <~A/2 we extrapolate f ( x , y ) into --(A + L)/2 < x <--A/2 and A/2 < x < (A + L)/2 using cubic polynomials. Having filled the two domains, --(A + L)/2 < x < --A/2 & --A/2 < y < A/2 and A/2 < x < (A + L)/2 & --A/2 < y < A/2, with our estimate ~ ( x , y ) we then extrapolate f ( x , y ) into --(A + L)[2 < y < - - A / 2 and A/2 < y < (A + L)/2 using cubic polynomials. Figure 6c shows the estimate of the original image which we obtain after we extend the edges of the image shown in Fig. 6b and then inverse filter it. Figure

37

PREPROCESSING DEGRADED IMAGES ,;"e~:e"~lp;=:';':'~"";~t,;

:...i !'i!i "!iii:!'i::iiNiNiiii:J!i,.:. ...:L:::i !:Ji: iiiii.i: i~ii'i!ii iiiiiillii:i!'il. ..ijjj ii.:i:i:iiil!i !!ili:ij L-'i;.i!ijil:::!: ":~!ii'IZ ...:ii:i ii.ii.! ii!i!i illl!::!.ii!Li!ir!i!i!:::i:'iiiii.:'.: .i , ' : : ' . : 1 : : : , : :::::::1 : : : : : : : : : : : : : : : : : : : : : : : : "1,:,"::::::::::

. . . . . . . . . . . . . . . .,........ , . . , . . , " " ". '. , ........,....... . : ,:: ::%::': ::::: :::::.:::::::::':

,if!!,iii:! ili:r!:iiiii::i!i'iri:::,

" • .!iiii}ii"'i"F ::=::'=::'::

::|,

11; "

:: : : : :

:

,,,, . . . . . .

.

.'

.

: ":

~::i" • i iR!-":iiil:::... ".'.::!. :':1::: : : : : " : ."

:,::::::,,:::,: :|:::::::I::::'-"

: : .... ::::":::

:u@;i.:ir~iii:

::

:i!! : . : !L:!'IIIF!'!L:;I":" . . . . . , , , . , , , , , , . ~ , , . . . .::: . a~:~ma

.,=,,,,,,,,...,'.~'.'.1£~,"=" "| " : ' Nl|;' ":'IL"

. . . .

?;:,III:...',~.~:llII,

=ii~:m:m~i.::.:: ii~i~i~:~!iiiiiiii~ilNdii[i!iN'i:ii:~,:...::~"

::'.. "::"

:|||:::::::::::" :::,::.:':::=::=:"i'::=":::: :: • •U:|~m.:=::. " ,~n~ m:::::.: :..::::.: ~ :~ !: "" ", ...... H~I!III,RI.'-'!::,,: • ", :

~.:~':

:

I

"I :'::

:

: ' ."

. i :ill_q,

|:Frl.':'.

~::::'-'::..:

.:.::~IN~!

. . .

..... .:'ll'n" .... i11:1111 :::....:i::,':,l| •:'" :'::;1:;I:1 : . ..:::::::u

• .."

,:iii!:, :,ii!iiilii I

,...

I.

:!liFi!!!!i s"! ::.....

:'

"i'.:,i'i:i~:i...:l'i:. ,~:

:... ":"

.:

:'.::" •

",: ..................... 'l , :::.:::,:: :::.::

•

::;:':::':1.:::::::1::::::::: , : , : I : : : : : :1 I:llhln:l:::::::lUi:::l::::::l.'::': i,:i : I:11¥:::1::;.:11|-' i i ' I | ' : : :: : ' . i " " : I:u|.::::l:.':::l:nl:lr:" |':..:".: : " . I 1 | : : : : : : . : : : : : : . : . : I . ". : . : ' : .:"

:"

"

i!i :i!!i:ii:':,:, i i !i : !i i::i:" .: !

: ::::.:::Ill::::::':

:::::.

• ":...:

:' i:iiii'ii|ii," :i i, .:."

,

•:i .!i.Lliiii!:!i:ii~l ". !~i iHII!!!III:i!I

""

• :'ii|i~F!H|ii::

: i

:.. :. ".

,

" • ": : : : : I I : , : : : : . : 1 : : :

,i:i,,:i,,i:,i,,,

• •

:: "'~:'" "-" ": "" :""": :": " "

...

It:N!:.

" "': ":" ::: "" "

:'L'Niii"

:

I : i

. • :: Ill I':T:!:"

:: ,.,:.,

..

,:.;

: .... :"

" I': ii

:'.: :.'i.

""

I.:

:

: :,",'.

"

:i: i'i "l

.: .::'i':i'..

~

•

: "

•

I : . " : ' : : ::1. " : ; . : . : .

:::" :~.:" : : :i~.i;,i: :~i:.::.~.': :,i : :::'::" ,::::: : : : : •

:.

!. ":i " : .iiiii';:':':.': ."

:

iJi!i'!]

;G~:...': :'.~ :.[ i. ~'::':!.i': i!: "i!!~":: :'ii~ |

:'i::

:" " : l :" : ; L : ' : • "

".

: :.;::::.:

"" : !

"..1

:,

.. •

i:: :i" :d" :i:.:!r!!il

:i

~':::': ~i:iiiiiB'::

""

i: :;

. . •

"~:: . : :

•

•

":i :. i : •

:!.i k" :': ." , . •

:

.: : '

i!'::~. : "J'."

"',, .

,

: :::"

: :.

"

. :.

i'i:

: "

." "L I!.L!I :'! "::iii!!i.: i i .:i" ! .!:!.! :.... "'. i" i!:ri:':'i :;" ::i i: :" i i:~ " r:'; :" :. "."

,.:%::['..

:ii i i!iili::iiiiii!

I I I

r"'.': .... .... :!,:i ,m', i:id.! :: I1:, i i!iu I'.

.~

:'::'":.:::.::'.

.

::. •

:

• ~:i..'.:i::.:~'i.-iii:Nr'iii~

I

•:":" ..'.-!':i÷'ii.iiiiiii'i..,.:: ..: ,i:irJ I:" i:ii:.:ii.i.:i:.~:.:i'~ 2 " "": • ! :" i :.ill i!! ii.: :...'! ." ":'4 ii "i iii:i!i!i:iiiiii~ili:i :.:':.:.::. i !.i: iiii!!ii!!ii~iiiiii.'.. "... i:ii:'::i:.:'ii'i!iliii"!:ii:ii.:i:.:

•

:',::!" i :," :"

"..:,:L..N!! :i:i. ' : :::i ii!"..i i:i L :.:" ."..: •:.:.1.::::.:|:':': : : • : : ::: I : " •:

• :.., " • : •. :

'":li! i:l:i"P'!" .:::.r: :':; :" i. i !i: ~ ': :! ::i ~"

•

i"

II;'!:PT::!~ i! "iii:i~!!:i! ii .~i ii!,F: :I, ~,

":'." .::.: ::.:1::. :, :, .

•

. .'l:ldl! ":" ".!':'::!'ill'"":":"

i

""

: l l'i'ik "i : l

ih:r rii-. ..:.....:=" !I,i!.!ii!iiiiii!ij.j N-. '['~ ',l!,r":" : •

• .: : : : . : .,,,,L.,,., ,,.. =::, , .::::::::::::'::::|:":

i :.':.:i:

.: : ' : :

:. :.:.:.,:1::':::,:

:: :, ::::::':::. : . . . . .

"':'E

.... , •

::'::.'

.:

: :':'" ::::""

".

" • :.

.... :"::: ........... I"'": " . . . . ~' . . . ... . . : : ' ."| '.:.:...:.'.:.:.:.: : : : " " • J"l ' " l'|:":::l:| ::.:

":" ":i "i"

"

:I :'. "! • :""

Ni , li

",2Rr!!,~r'r, il! ":"

~:: :::: • • :" i :~'~;.,:...

i.

i.,~.t'!= i.]1 iilili:,iil.:: :': ::':'""

:i!iii!iii!iiilll...;i-": :.q.:u~,:;~mu,:,:-" •

'. :

i!"ii~ i~kq ,,':uii ~" :: .'! .'!......:

i:::;::;l:':':.:.~H |i: ."=:::::==':''r": , l l , . m , , , . , , : , | ~ L ' :" [ ]

::" :

:':I!'::":L:

.'-.' ........ • ~ .... r, ........ '~il.!i" ."i:i#..Ij!' liu.::.t L'ii..." • i~li~i: :' ::ill "Tiii'i ~i.'.i .i .::.

•

!:: ":"

~'ll];i.i!"!~:.. "..':. L. . " : : : , : : : : ' : l : ' : , . . : r : : : ' : " : . , ::: " ~":':"

"

....

I. I: rl :::i.'i: ..:i:

:: ' • •

: :!EIIJ! I ! !l

P~i.':!::=":r:!Hili!

: ::'::;l:r:::-':::|::::

..

i[" :'.:.'::"

;~]~.:-iili "llii'llrldi...i!

• [ Ill ::::ll:I: :':I;:ll:.:::::.':.:: . . : ' : , : : ; 1 . : , . . : : 1: " ":, " : l l :" 1: ' " : ' : ' : : " " ",:" " ::" : I : i : : I. :

irii~:.".:.il.::~i::it~ii!':: ":::. :g

:~iiii!i,iiiiNilL'-.:,i

:, • . . . .

:

:::::'":" :=:.:::.: ::.l:l:r

ilill ,ill!fill;if:

.

!.

" i li i I i:iiii:lh'iiii~i.::

,..

::: !:="

,,,,,--',

"

":'|l"

:::'::" I : ::|::: ::l:,:l |" " ~:l::: " " : :" , . ~]~i:" , , h" :,: : :,., •: , l. ' : :": J i l l . . . . ' , " "" |: . . . . : I. . . . . . . . . . . . . . . . . . . . . .

:!i"i"i:!'!iiiii! "'. ::EN:'!!:!i:'I :'.

...............

:i~ii~nli~!lliiiiiii N'i!~iiiNi~!t.i!i ~:i;:~: ii~i ~:

:,l;::'l,::: : : :;:::1:" : : " : : 1 : : : . ,

~ .:~..

l n i l l u ; l : l , - - , ' : ,'~ ' , . ' ' ' ~

:1 ~':~t::'::'1: := ": ': "=r , : : : ::":"1 : : : l : : : : . , : : I 1::::1:::::: : ' : :" :~:'" • : • . :" :~. : .

"" '

;r,-'.":"..

: : = : :ii~ii~i:it =, ,:,=, , =, , ,:,.,., , :i N:iill

......

:,

I,::

II"l'plllill

~bi:: :::"'~.,,':~ :iL.':!:li:i, :.lq~,,::r" :! .:u'i:i!~!! .=u!d! !'.!!i! i ~ i!.r.'ii!~i~i':; i:-

•

':

:

:: : "ililli::'i:,::i-!:iiii',ilr illi:

:. : . : :.

II1~ I f l l l l

• ""

:' :

~::I i::~i~iiii:ii!;ii:r:i!

:. i~.i!!!iiiiii i~: i::: ~.~... . . . . . . . . . . . . . , ,..

: ...:.:"."

""L:'.: :" i:::.h:i

•

, ,.....,,, .......

I: :'::|.

::

: !

.,

,:: : : :

:" ! :'" L'.'.'F'I :

:

!i: :

;,. ::

: i :!!i!F.'iii'i!!! : ": " :

;F'i~'.:;il'i:.:ii i: :. !

i: i .:'iiKi!i!:ii~i:.!::."

::

:,,.,']l;,,m

"'. : : .::::.

: .. i.. ! ..:'ill'"

• .

=.:::' ::::'='":':,,::,:.: =::=:":':,. :.-:,:-.., :':: l::l, :FII::|:::,I:'::'::::' .:."--:':-,'.', h.,,,, . ., ., ' I L, |=, I .,.,|,.:..:.1., :.: .:..:..: .:... ..

" . : ! : ! "i!;i:

.

!i !" .iiii;!!il!i:.i:'!":" :"

".:

.-

"":

i

. ~i i ~!F.~:I.

:

m!i:~ ".

::':"!

,. : !i;,.';:'..:.iti;'i" "

:i

"

:i!".: i::

;: :

:

r !:Nii!i .:i~ ~:::.. i :ili:ii:i~ .

:

,,:

",1 ." ...:.:

!!::'i:" i':':l:" ~.' , . " :

• : .:1:: : : : ' : : " : , : : : 1",

~:

"i ":.',.r:.i" !: : " " "" ,'1: :, : ' : ' " ." " ".

i:ii !ii: i6

.:.::'.

:'::::

!1 i: !

:ii~ i i! ":::" :':: •

".

:. i

.

•

: : : : : ""

: : : : ":::;

:'i: "::if'i!" ::"

:::1

•

.:

F[o. 6. Restoration using edge extension followed by inverse filtering. In D. there are 128 sampling points in each of the x- and y-directions. (a) p(x,y), ~ ~ ~. (b) t(x,y), y ~ F. For 3' ~ fl, W = 0.03, (c) ~)(x,y), no recording noise on b(x,y); (d) ~(x,y), 1% recording noise on b(x,y).

6d shows the estimate obtained w h e n uniformly distributed recording n o i s e having an rms value of 1% o f the rms intensity occurring in b(x,y) is a d d e d to the image s h o w n in Fig. 6b before we carry out o u r processing. We use the s a m e W w h e n obtaining both of our restored images b e c a u s e the recording noise associated with the s e c o n d of these images is less than that theoretically d e m a n d e d b y a value o f 0.03 for W (we use this value for W because it seems n e c e s s a r y to

i

38

MCDONNELL AND BATES

c o m p e n s a t e for the inaccuracies u n a v o i d a b l y present in our estimate ~ (x,y) of e (x,y)). We n o t e that the images shown in Figs. 6c, 6d b o t h show details of a mouth, e v e n t h o u g h the m o u t h is not visible in Fig. 6b.

6. CONCLUSIONS We h a v e presented two methods of extending the edges of degraded images so as to m a k e t h e m m o r e suitable for restoration by inverse filtering. Both methods a r e simple to implement, since they only involve fitting polynomials of low order to given data. T h e o v e r l a p p i n g edge extension m e t h o d is the m o r e effective, and we intend to refer to it as the "edge extension m e t h o d " in future. T h e example presented in Section 5 suggests that the method c h o s e n for extrapolating f f r o m F into fZ is relatively uncritical. T h e method b y which w e fitted our cubic polynomials is incapable of faithfully reproducing variations of b parallel to the perimeter of 11, within the p a r t o f 12 outside F, and yet our restorations of the original image are encouraging. W h a t seems to be important about our method is that it forces/3 to o c c u p y a region o f ~¢ of the correct size and it avoids the artifacts which usually a p p e a r due to the truncation o f the degraded image. We h a v e r e p o r t e d here w h a t appears to us to be the first account in the open literature of a general m e t h o d of r e s t o r i n g the whole of a degraded image which is Of the class f~. T h e finite-filter-array methods of Arguello, Sellner, and Stuller [12] a n d N a t h a n [13] are only suitable for treating the central parts of such an image. We a r e currently combining the edge extension m e t h o d with our previously r e p o r t e d noise rejection methods [9] in an attempt to further i m p r o v e the fidelity of o u r r e s t o r e d images.

ACKNOWLEDGMENTS We wish to acknowledge the many helpful discussions we had with our colleague W. K. Kennedy. One of us (M. J. McD) acknowledges the support of a Postgraduate Scholarship from the University Grants Committee.

REFERENCES 1. J. W. Cooley and J. W. Tukey, An algorithm for the machine calculation of complex Fourier series, Math. Cornput. 19, 1965, 297-301. 2. T. S. lrluang, W. F. Schreiber, and O. J. Tretiak, Image processing, Proc. IEEE 59, 1971, 1586-1609. 3. M. M. Sondhi, Image restoration: the removal of spatially invariant degradations, Proc. IEEE 60, 1972, 842-~53. 4. E. L. Hall, R. P. Kruger, S. J. Dwyer, D. L. Hall, R. W. McLaren, and G. S. Lodwick, A survey of preprocessing and feature extraction techniques for radiographic images, IEEE Trans. Cornp. 20, 197l, 1032-1044. 5. Evaluation of motion-degraded images, in Proceedings of the Nasa/ERC Seminar (M. Nagel, Ed.), NASA, Cambridge, Mass., 1968. 6. H. P. Erickson and A. Klug, Measurement and compensation of defocussing and aberrations by

PREPROCESSING DEGRADED IMAGES

7. 8. 9. 10. 11. 12. 13.

39

Fourier processing of electron micrographs, Phil. Trans. Roy. Soc. London Ser. B 261, 1971, 105-118. B. L. McGlamery, Restoration of turbulence-degraded images, J. Opt. Soc. Am. 57, 1967, 293-297. F. S. Zimmerman and S. C. Gupta, A state variable approach to digital image processing, Comput. Elect. Engng. 1, 1973, 255-276. R. H. T. Bates, W. K. Kennedy, and M, J. McDonnell, Efficient digital restoration of images blurred by linear motion, Letters in Applied and Engineering Sciences, 2, 1974, 133-143. R. H. T. Bates, Contributions to the theory of intensity interferometry, Monthly Notices Roy. Astron Soc. 142, 1969, 413-428. ~ D. B. Gennery, Determination of optical transfer function by inspection of frequency-domaln plot, J. Opt. Soc. Am. 43, 1973, 1571-1577. R.J. Arguello, H. R. Sellner, and.J.A. StuUer, Transfer function compensation of sampled imagery, IEEE Trans. Comp. 21, 1972, 812-818. R. Nathan, Image processing for electron microscopy: 1, enhancement procedures, in Advances in Optical and Electron Microscopy, Academic Press, New York/London, Vol. 4, pp. 85-125.