A new block-based motion estimation algorithm

Signal Processing: Image Communication 4 (1992) 507 517 Elsevier

507

A new block-based motion estimation algorithm Kan Xie, Luc Van Eycken and Andr60osterlinck University of Leuven, ESA T-MI2, Kardinaal Mercierlaan 94, B-3001 Heverlee, Belgium Received 19 June 1991 Revised 2 December 1991

Abstract. The conventional motion estimation algorithms used in digital television coding can roughly be classified into two categories, namely the block-matching method and the recursive method. Each of them has its own strong points. In this paper, a new type of block-based motion estimation algorithm is presented, which is based on the block-recursive (gradient) method and makes use of some of the merits of the block-matching method. For a moderate translational motion, motion estimation with a subpel precision can conveniently be obtained with only a couple of recursive searches, and for a violent or complicated motion which cannot be estimated by any block-based algorithm, the local minimum of prediction errors can always be found. Our experiments show that the proposed algorithm is efficient and reliable, and obviously superior to the conventional block-recursive algorithms and the fast block-matching algorithms. The performance of the proposed algorithm tends almost to the optimum of the full search algorithm with the same estimation precision, but the computational effort is much less than that of the full search algorithm. Keywords. Motion estimation; motion-compensated prediction.

1. Introduction

In digital television coding, one attempts to minimize the temporal redundancy in a sequence of images so as to reduce the transmission data rate. Motion-compensated (MC) interframe prediction is a most efficient approach to realize this goal. In motion-compensated interframe prediction, the frame to frame displacement of a moving object in a television scene is estimated, and the prediction is formed by displacing the previous frame elements. It is well known that the success of motion-compensated prediction depends on the ability to estimate the displacement of moving objects. Several mathematical models have been proposed to estimate the motion vector fields in a sequence of television pictures. They can be classified basically into two groups : the block-matching motion estimation algorithms [4, 5, 7] and the recursive (gradient) motion estimation algorithms [ 1, 2, 9, 15]. The fundamental idea behind the block-matching algorithms is that the motion estimation can be

obtained by searching the position of the best match in the previous frame. The brute force search procedure (full search algorithm) is straightforward to find the optimal match in a search region, but an enormous amount of computation is required. Therefore, most of the efforts in the block-matching method are to seek an efficient search procedure to reduce the computational complexity but without an obvious effect on the estimation accuracy, such as some proposals like the 2-D logarithmic algorithm, the 3-step algorithm, the conjugate direction algorithm, etc. Most of these fast algorithms are based on the assumption that the cross correlation function in the search region is a single-peaked and quadrantmonotonic model. However, the assumption is not always true for actual TV signals. It has been noted that sometimes a fast search procedure slides to a wrong direction or is trapped by a false correlation peak, and as a result we get a non-optimum match or an erroneous estimate which could affect the visual perception and the coding efficiency. In

0923-5965/92/$05.00 © 1992 Elsevier Science Publishers B.V. All rights reserved

508

K. Xie et al. / A new motion estimation algorithm

addition, it is known that motion estimation with a subpel precision is effective to improve the estimation performance and to reduce the prediction errors, but the computational complexity will increase tremendously even more. For example, suppose that the possible maximum displacement in the horizontal and the vertical directions is Omax = 7 pel, the full search procedure with a subpel 1 precision P = z pel requires to evaluate a crosscorrelation function (or a mismatch function) at (2Dmax/P+l)2=3249 shifts and to calculate additionally the spatial interpolation at [ ( 2 D m a x / P + 1) 2 - (2Dmax+ 1) 2] =3024 points of subpel. Such a tremendous amount of computation is a big difficulty for the real time applications. The recursive algorithms are based on the gradient (differential) method: suppose that the luminance intensity changes in successive frames are due to the translation of an object, the motion estimation will iterate in the luminance gradient direction so that the consecutive approximations converge towards the actual displacement. The recursive motion estimation algorithms can be executed either block by block or pel by pel, and denoted as the block-recursive algorithms and the pel-recursive algorithms, respectively. For a slow motion, the performance of the recursive algorithms is quite good and the convergence speed is high. Moreover, it is very convenient for the recursive algorithms to estimate a motion vector field with a subpel precision. However, experiments show that the recursive algorithms are not very stable and reliable especially when there are some violent and complicated motions in television scenes. Starting from the investigation of the different motion estimation algorithms published, a new type of block-based motion estimation algorithm is developed. The algorithm is a hybrid version of the block-recursive method and the blockmatching method. 2. The proposed block-based motion estimation algorithm The proposed algorithm is based on the blockrecursive method. Let I(z, k) denote the luminance SignalProcessing:Image Communication

intensity of a pel at spatial location z"r= (x, y) in frame k, where superscript T denotes transpose. If lighting is uniform and luminance intensity changes are due to the translation of an object, then

I(z,k)=I(z- D,k-1), where D is the motion vector of a moving object during the time interval between two successive frames. If/~i is the/th estimate of the actual motion vector, the displaced frame difference D F D can be defined as DFD(x, b i) =I(z, k ) - I ( z - D

i, k - 1).

(1)

Suppose that the displacement of a moving object is constant within a block. The motion estimation of a block can be approximated by the following recursive equation:

[-~DFD(z,/~i 1) sign(Vlx(z_/~t-,,k_ 1)).]

b,=b,_l_|

~,lVl~(z-b'-l,k-1)l +o

L

ZlVly(z-D'-I,k-1)l+~r

I (2)

where VIx, VIy are the vector components of the luminance gradient VI with respect to z - / ~ i - 1 in frame k - 1, and

sign(x) =

0, 1, -1,

if x = 0 , if x > 0, if x < 0 ,

Equation (2) is similar to one of the block-recursive algorithms given by Netravali and Robbins [9]. A small positive constant ~ is included to account for the case of the denominator becoming zero (here ~ = 10 is used for the block size 8 x 8). Since (2) has a comparable performance with other blockrecursive algorithms but with a much simpler hardware implementation, it is adopted in this study. The performance of a gradient algorithm is good as long as the displacement D is small. Lamb and Murphy reported [6] that the gradient algorithm (without iteration) performs accurately for a moving object up to a speed of 2.5 pel per frame. However, as motion D increases, the performance of the

K. Xie et al. ,/A new motion estimation algorithm

gradient algorithm degenerates and errors increase because the quality of approximation becomes poor. Although the estimation errors can be alleviated to a certain extent by introducing the recursive method, the block-recursive algorithms are still unstable and unreliable for a large displacement. In addition, the effective range of estimation and the stability of a block-recursive algorithm are related to the size and the speed of the estimated moving object. It has been observed that sometimes an estimated vector is far away from the actual displacement, because the mathematical model of the gradient method fails to describe the real motion. For example, sometimes there is no reasonable luminance gradient due to fast motion of a small object, or sometimes the luminance gradient is very steep and the gradient region is very narrow, which often occurs in the neighborhood of edges and contours of moving objects, so that recursive estimates are not able to follow the correct gradient direction and converge to the real motion. As a result, an erroneous estimation or socalled 'oscillation' may occur, and the variance of prediction errors may be very large. In the proposed algorithm, an alternative approach similar to the block-matching method will be employed to the regions where the recursive method is invalid. For a recursive algorithm, a basic assumption is that an initial displacement estimate/)~-~ is used to produce a new improved estimate b i according to

b ' = b ' - ' +u,,

(3)

where ui is the updated term of the ith iteration. If the algorithm is convergent, then for a large enough number of iterations u; will tend towards zero. In actual computations, if ui is below a predetermined value, then the motion estimation is terminated and the v e c t o r / ) ; is considered as the estimated motion. However, it should be kept in mind that (ui-- 0) is only a necessary condition but not a sufficient condition for a correct convergence. It is not difficult to understand from (2) that a small u; does not guarantee a small variance value of prediction errors. In addition, it has been noted

509

that sometimes the prediction errors in the last estimate is slightly larger than that in the previous estimates, even though the recursive algorithm is convergent. Therefore, an additional test for evaluating MC prediction errors is measured at every iteration point in the proposed algorithm, 1

MAD(b;)= B Y

I I ( z , k ) - I ( z - D^i, k - 1)l,

(4)

xE~¢

where ~ is the subset of the integer pels in a block, and B ( = M x N ) is the total number of the pels in the block. The MAD is identical to the mean of the absolute displaced frame differences, a matching criterion in the block-matching algorithms. The purpose of measuring M A D is to locate the smallest mean of prediction errors over all operation points, especially when the block-recursive algorithm fails to estimate the real motion. That is, motion estimation is obtained by searching the optimal match along the prevalent luminance gradient direction rather than along a pre-determined search route. It is known that the convergent behavior of a recursive estimation algorithm depends strongly on the motion natures and the local image details. If the recursive method is not suitable to describe the real motion, oscillation may occur and the prediction errors may be very large. In the proposed algorithm, if the performance of the bloek,recursive algorithm is not good enough, a so-called 'force search procedure' which is similar to the block-matching method is applied to search the best match in the whole search region. Then, the block-recursive algorithm performs again from the position of the best match to get a higher estimation precision. This approach is efficient to remedy some erroneous estimated vectors. In addition, sometimes it is impossible to obtain a correct motion vector by any of block-based algorithms for the actual signals (for example, motion estima, tion will not make any sense if a block contains multiple motions or deformed objects), then the force search procedure can at least help to minimize the prediction errors. The reported blockmatching algorithms like the full search algorithm Vol. 4, No. 6, November 1992

510

K. X i e et aL / A new motion estimation algorithm

can be used as the force search procedure. Additional computation does not increase too much because the force search procedure is only used for a small number of blocks, while a considerable improvement can be obtained. It is known that a reasonable initial estimate is beneficial to improving the performance of a motion estimation algorithm. Since the motion vector field contains a considerable correlation in both the spatial and the temporal dimensions, one can expect that a motion estimation algorithm converges faster if a correlative motion vector from the previous nearby blocks is taken as the initial motion estimate of the present block. Our experiments have shown that the initial estimate is favorable to improving not only the convergent speed but also the estimation accuracy of the motion estimation algorithm. In summary, the proposed block-based motion estimation algorithm is implemented in the following steps:

1. Init&l estimation. The initial estimate is obtained by selecting a correlative motion vector from the previous nearby blocks. The motion vectors of the block above and the block to the left of the present estimated one, and of the block at the same position in the previous frame, if they are available and good-estimated, are taken as the candidates of the initial estimation. A good-estimated vector means the one which is within the maximum possible displacement region and the corresponding mean of absolute displace frame differences MAD(D) ~< T2 (a pre-determined threshold). These candidates are used to predict the present block, and the one corresponding to MADmin, the smallest value of MAD, is chosen as the initial motion estimate /~0, where the superscript 0 denotes the initial state. But if MAD(0)-%< MADmi°,/~0 is reset to zero. If MADmin is below a small pre-determined threshold T~, the corresponding vector is considered as the estimated motion vector of the present block, and no further computation is necessary. Otherwise, go to the next step. Signal Processing:Image Communication

2. Iteration. Starting from the initial motion estimate /~0, the block-recursive motion estimation algorithm (2) is executed iteratively until M A D ( / ) i) ~ T,, (a pre-determined threshold) after the block-recursive estimation algorithm, then go to Step 3. Otherwise, the optimal estimated vector Dop which corresponds to the minimum MADmin instead of the last estimate/~i is taken as the estimated motion vector of the current block. 3. Force search procedure. If an estimated vector exceeds Dmax or if MADmin is higher than Tin, the force search procedure is applied to find the best match by searching all integer pels in the searching region of the previous frame. If the minimal mismatch value MAD is smaller than the MADmin in Step 2, then go back to Step 2 and the recursive algorithm (2) is executed once again from the best matching point to get a subpel estimation precision. Otherwise, the computation is terminated and the optimal vector Dop estimated in Step 2 is taken as the output motion vector.

3. The performance evaluation The performance of the proposed algorithm is evaluated and compared with the conventional block-based motion estimation algorithms. Since the actual motion vector field of television signals is unknown, the accuracy of motion estimation obtained by any algorithm could not be determined directly. Several indirect measures which are usually used to evaluate the estimation performance are adopted in this investigation: 1. The peak-to-peak signal-to-noise ratio (SNR) of prediction image, SNR = 10 log10 x (peak to peak value of signal) 2 (dB), var[I(z, k) - I(z, k)]

511

K. Xie et al. / A new motion estimation algorithm

where I(z, k) is the actual luminance intensity and I(z, k) the predicted value. 2. The entropy of displaced frame differences DFDs (i.e., the MC prediction errors), sp H(b) = - Y. P(b) log2 P(b) (bits/pel), b----gp

where Sp is the possible maximum value of DFDs, and the absolute values of DFDs below the threshold 7"1 are lumped into zeros. 3. The number of blocks with the mean of absolute prediction errors MAD > Tp (a pre-determined threshold. The luminance components of two television sequences, 'Miss America' and 'Claire', which are typical teleconference scenes, are used in the computer simulation. The luminance signals are quantized uniformly using 8 bit/pel, and each frame has 288 × 352 samples. The frame frequency of 'Miss America' is 30 frame/s, and that of 'Claire' is 10 frame/s. The following block-based motion estimation algorithms have been investigated, where the block-matching algorithms are enhanced with a subpel precision for the purpose of comparison: Algorithm 1: A block-recursive algorithm (eq. (2)). Algorithm 2: A modified 3-step algorithm with a subpel precision (i.e., the 3-step procedure is used to estimate motion vector with 1-pel precision, and then the subpel precision is obtained by full-searching all subpel positions in the 2 x 2 region centered at the last integral estimate). Algorithm 3: The full search block-matching algorithm with a subpel precision. Algorithm 4: The proposed block-based motion estimation algorithm. Some implementation details in the simulation are described as follows: 1. The performance of the block-based motion estimation relates strongly to the block size and the motion estimation precision. In this investigation, the block size is 8 x 8 (64 pels) and the 1 motion estimation precision is P = z pel.

. The maximum displacement Dmax in the horizontal or the vertical direction is set to 7 pel per frame. . The maximum number of iteration limitation for the block-recursive algorithms is 10. 4. The pre-determined thresholds TI = 2, T2 = 15, Tp = 8 and Tm = 5 (out of 255). 5. The calculation of the horizontal and the vertical gradient components and the spatial interpolation are illustrated in Fig. 1. Let I(z, k - 1) = Ic be the luminance intensity of the present pel at position z in the previous frame k - 1 . Suppose that I ( z - [ D i - l ] , k - 1 ) = I e , where [D] denotes the integral approximation of a motion vector D by rounding both the components of the vector, then the spatial gradients of the displaced pel are given by

V/x(Z-- D i-1, k - 1) = Ih-- Ib, 2

V/Az-b'-l, ~ - 1)- h-/d 2

The corresponding luminance intensity of the displaced pel in frame k - 1 is obtained by a two-dimensional interpolation as I(Z

- D~ i - I , k - l ) =(1 - dy)[(1 -dx)Ie+dxlh] + dy[(1 -dx)Id+d~Ig],

--8

8 I(z - Di-l,k

g

8 - 1)

6

Fig. 1. Position of the picture elements in frame k - 1 used for the computation of interpolation and gradient values. Vol. 4, No. 6, November1992

512

K. X W et aL / A new motion estimation algorithm

where dx, dy are the fractional parts of the motion vector in the horizontal and vertical directions, respectively, and

dy By [By].

dx = Dx [Dx],

=

-

-

-

The following figures show in detail the performance of the proposed algorithm and the conventional motion estimation algorithms for the sequences of 'Miss America' and 'Claire'. Figure 2

illustrates the signal-to-noise ratios of prediction images of the different algorithms for 'Claire' and 'Miss America'. Figure 3 shows the entropies of the motion-compensated prediction errors and the numbers of blocks with MAD>Tp for the sequence 'Claire'. It can be observed that the performance of the conventional block-recursive algo-

.....

tl 1.4 i

..............algo.l algo.2 , , , , algo.3 - - algo.4

:\/

~

algo.2 algo.l . a]go.4 algo.3

....

•

~1"

t.3

42

~.

i

i2L~

: I

itI[i'~ ' q~:t¢,

,,,,~

::

40 ,

/ V

i! ....

i

:

38

,

i !!

:'V,~ ./ i/i: :iiL ,

,'~ il V

36

,:

i~

.

i

t

/ "i

~' \ :

~i/

i:

:ff~I!:l/~. .... '

,

?'i,",X/I, , ];"v: ,,'/,l

I ili/A\v ~/i :",.i",!Fii:ll VV~il

*~

iv::

34 0.7 32

10

(a)

20

30

40

50

I0

(a)

20

30

.............

41.5 1

No. of f r a m e

80

algo. 1

.............

..... algo.2 J lt algo.3 _ _ algo.4

4z[-

70

:

40

6O

j '.

J =

! :

i

i/i

/"

39 /

...

"'-./......... \/ ...... i : A(

38.5

,,t, algo.3 _ _ algo.4

{

:

A

algo.l

.....algo.2

,5

50

~

: ,'

i t

"

i.

38 I0

(b)

50

No, of frame

42 [

Z

40

20

30

40

No, of frame

Fig. 2. T h e S N R of prediction images of the different algorithms (a) for the sequence 'Claire' a n d (b) for the sequence 'Miss America'. Signal Processing: Image Communication

of"

50

(b)

10

20

30

4.0

50

No. of frame

Fig. 3. (a) The entropies of M C prediction errors and (b) the n u m b e r of unpredictable blocks o f the different algorithms for the sequence 'Claire'.

513

K. Xie et al. / A new motion estimation algorithm

rithm (Algorithm 1) is obviously not stable and varies with motions in the image scenes. For the scenes containing slow motions, the performance of Algorithm 1 is passable. However, if there is a relatively fast or complicated motion in image scenes, the estimation performance of Algorithm 1 will degenerate seriously. By contrast, the stability of the 3-step block-matching algorithm (Algorithm 2) is much better, but the prediction errors sometimes are still very large because the fast search is strongly related to the image features. The graphs show that the proposed algorithm has a pretty good performance which almost tends to the optimum (by the standard of prediction errors) of the full search algorithm with the same estimation precision. However, the amount of computation of the proposed algorithm is much lower than that of the full search algorithm (the average number of iterations per block of Algorithm 4 is only 2-5). It is also noted from Fig. 2(b) that the SNR of Algorithm 4 is a little bit higher than that of algorithm 3. The reason is that there are some small differences in the procedure of initial estimation in this investigation. For Algorithm 4, besides D = 0, some previous motion vectors are used in the initial test, and if MmDmin ~
Fig. 4. The prediction image of Algorithm 1.

Fig. 5. The prediction image of Algorithm 2.

The simulation is also performed on other television sequences which contain more complicated figures and motions. 'Baltimore' is a sequence of high detailed pictures with moderate motion (including camera zooming), and 'Teeny' is a Vol. 4, No. 6, November1992

514

K. X i e et al. / A new motion estimation algorithm 6,

f'i f,~,

5.5

i

5

,

.......... algo.l +.~+algo.3 __ algo,4(Dm=7) .......

algo.4(Dm=14)

4.5 4

¢zl

3.5 3 2.5 2 10

(a)

i

i

i

20

30

40

50

No, of f r a m e 3.4

Fig. 6. The prediction i m a g e o f A l g o r i t h m 3.

.... algo.l ....... algo.2

3.3

~ algo.3 - -

3.2

algo.4

3.1 J~

3 2.9 2.8 2.7 2.6 2.5 2.4

(b)

r

i

i

i

i

2

4

6

8

10

No.

of

12

frame

Fig. 8. The entropies o f M C p r e d i c t i o n errors o f the different a l g o r i t h m s for the sequences (a) 'Teeny' a n d (b) 'Baltimore'.

Fig. 7. The prediction image of Algorithm 4. sequence of pictures of a close-up view of the head of a young girl with some violent and complicated motions and brightness changes. The motion estimation is performed on the central part of 288 × 352 samples in the frames for the purpose Signal Processing: Image Communication

of comparison and saving computation. Figure 8 shows the entropies of D F D for the sequences 'Teeny' and 'Baltimore'. It can be observed that for the sequence 'Baltimore', the performance of the block recursive algorithm is pretty good. However, for the scenes containing some violent and complicated motions like those in the sequence 'Teeny', it becomes very difficult and sometimes

515

K. Xie et al. / A new motion estimation algorithm

impossible to estimate the real motion. It can be seen that the entropy of the proposed algorithm tends to the optimum of the full-search algorithm (the performance of Algorithm 2 is approximate to that of Algorithms 3 and 4 and is not included in Fig. 8(a) to simplify the graph). Since there are some very violent motions in the sequence 'Teeny', extending the search region, i.e., increasing D . . . . can effectively reduce the prediction errors of an estimation algorithm in some scenes. The behavior of the proposed algorithm with D m a x = 14 pel is also shown in Fig. 8(a). By the way, it should be pointed out that in the prediction coding applications it is not worthwhile to seek so sedulously for a correct estimate in the violent motion areas, since the image of violent moving objects is blurred by the integration effect of a TV camera and the correlation between neighboring pels becomes larger. On the contrary, an intraframe coder like intraframe prediction may have a better behavior in these areas. Therefore, an adaptive MC inter/intra scheme will be more efficient to cope with a large class of television signals.

the proposed block-based motion estimation algorithm is capable of working efficiently and stably with different block sizes. Figure 9 illustrates the SNR of prediction images of Algorithm 4 with different block sizes for 'Claire' and 'Miss America'. It can be observed that SNR is remarkably improved as the block size reduces, because a

44 '

42

.,'6 *V, \

,;'. 4 x 4

~

40 38

,', f / 8x8 16x16

,

z

32x32

i,'

I

' I'J

34 ri 32 i

i

J

i

10

20

30

40

30 (a)

No.

of

50

frame

4. E x p e r i m e n t a l results for different b l o c k s i z e and different e s t i m a t i o n precision

It is obvious that the performance of blockbased motion estimation algorithms is firmly related to the block size used to measure the motion vector. A large block size could result in an increase in the variances of prediction errors because the assumption of constant displacement in a block may no longer hold, whereas a small block size could lead to an unstable estimate or an erroneous match. It has been reported that, for the block-recursive (gradient) algorithm, the estimation of the motion components in the direction orthogonal to the prevalent gradient can be carried out only when the block size is 8 x 8 or larger [2, 3]. Experiments also show that the performance of the conventional block-recursive algorithms becomes very poor as the block size decreases to smaller than 8 x 8. However, it is of interest to note that

41

4x4 8x8

~"

40

16x16

Z; 39

' 38

',, !

,,. /

i....

',!~'

'~ A~ / l ', '

,' ' "

,,,' ,/.~i \ vi',..'

i/

'~,'

.

~,- 3 2 x 3 2

.

'J

37

36

(b)

i

i

i

i

10

20

30

40

No.

of

50

frame

F i g . 9. T h e S N R o f p r e d i c t i o n i m a g e s o f t h e p r o p o s e d a l g o r i t h m w i t h d i f f e r e n t b l o c k sizes ( a ) f o r ' C l a i r e ' a n d ( b ) f o r ' M i s s America'. Vol. 4, No. 6, November 1992

516

K. Xie et aL ,/A new motion estimation algorithm

smaller block size gives a better approximation for complicated motion like rotation or deformation. However, it should be kept in mind that in the forward motion-compensated prediction, a smaller block size also means requiring more overhead data for the motion information. As mentioned above, increasing the estimation precision can improve the estimation performance and reduce the prediction errors. Our experiments show that the performance of Algorithm 4 with different precisions approximates the optimum of the full search algorithm with the same estimation precisions. Figure 10 indicates the SNR of prediction images of Algorithm 4 with different estimation precisions for 'Claire' and 'Miss America'. The SNR of the full search algorithm with pelinteger estimation precision is also presented in Fig. 10 as a comparison. It can be seen that there is a significant improvement as the estimation pre1 1 cision increasing from 1-pel to ~ and z pel, but little further improvement is obtained as the estimation precision increases to higher than ~ pel. In addition, there is a trade-off between the motion estimation precision and the overhead data if the motion vector field must be transmitted. A more precise motion estimation results in a reduction of the prediction errors which leads to a reduction of transmission bit-rate, but it also results in an increase of the motion information overhead. Therefore, the estimation precision chosen in this study is ¼pel.

44 43 42

Signal Processing:Image Communication

i

V ~./ Li

1/8 p e l f 1/4 p e l 1/2 pel_

/ ....

'v

t

~ / ~ algo3( lp~ " ! pel _

/

39 i

J

i

38

/ ,i

37 3~

i

i

l0

20

(a)

i 3O

i

40

50

No. of frame

42 41"5 t 41 I~/.

~

40

Z

39.5

m

39

.

//'V/X

40.5

]~,

~,

1/8 pel

/X~ .- ,! " \\.,~.-.,~ t/4 pe~ .......

38.5

'

t,]

38 37.5

(b)

Based on the investigation of the conventional motion estimation algorithms, a new block-based motion estimation algorithm is developed. It is a combination of the block-recursive (gradient) method and the block-matching method. For a small or moderate motion, a block-recursive algorithm is used to estimate the motion vector with a subpel precision. For a fast or complicated motion for which the gradient method is not valid, the block-matching method is employed to search the

/i

40

37

5. Conclusion

A

41

i

i

i

i

10

20

30

40

No.

50

of frame

Fig. 10. The SNR of prediction images of the proposed algorithm with different estimation preeisions (a) for 'Claire' and (b) for 'Miss America'.

best match in the search region. Our experiments show that the proposed algorithm is superior to the conventional block-recursive algorithms and the fast block-matching algorithms. The performance of the proposed algorithm approximates the optimum of the full search algorithm with the same estimation precision, but the computational effort is much less than that of the full search algorithm.

K. Xie et al. / A new motion estimation algorithm

References [1] M. Bierling, "A differential displacement estimation algorithm with improved stability", SPIE Vol. 594 Image coding (1985), 1985, pp. 170-174. [2] C. Cafforio and F. Rocca, "The differential method for image motion estimation," in: T.S. Huang, ed., Image Sequence Processing and Dynamic Scene Analysis, NATO ASI Series, Vol. 12, Springer, New York, 1983, pp. 104-124. [3] C. Cafforio, F. Rocca and S. Tubaro, "Motion compensated image interpolation", IEEE Trans. Comm., Vol. COM-38, No. 2, February 1990, pp. 215-222. [4] J.R. Jain and A.K. Jain, "Displacement measurement and its application in interframe image coding", 1EEE Trans. Comm., Vol. COM-29, No. 12, December 1981, pp. 17991808. [5] T. Koga et al., "Motion-compensated interframe coding for video conferencing", NTC 81, Proc., New Orleans, LA, December 1981, pp. G5.3.1-G5.3.5. [6] J.O. Limb and J.A. Murthy, "Estimating the velocity of moving images in television signals", Comput. Graph. Image Process., 1975, pp. 311-327. [7] H.G. Musmann, P. Pirsch and H.J. Grallert, "Advances in picture coding", Proc. 1EEE, Vol. 73, No. 4, April 1985, pp. 523 548. [8] A.N. Netravali and B.G. Haskell, Digital Pictures, Plenum, New York, 1988.

517

[9] A.N. Netravali and J.D. Robbins, "Motion compensated television coding- Part I", Bell Syst. Tech. J., Vol. 58, 1979, pp. 631-670. [10] S. Sabri, "Movement compensated interframe prediction for NTSC color TV signals", IEEE Trans. Comm., Vol. COM-32, No. 8, August 1984, pp. 954-968. [11] K. Xie, L. Van Eycken and A.J. Oosterlinck, "Motioncompensated interframe prediction", Proc. SPIE Vol. 1349, Applications of Digital Image Processing )(111 (1990), San Diego, USA. [12] K. Xie, L. Van Eycken and A.J. Oosterlinck, "Some new techniques in displacement estimation algorithms", Proc. SPIE 1Iol. 1199 Visual Communication and Image Processing I V (1989), 1989, pp. 18-29. [13] K. Xie, L. Van Eycken and A.J. Oosterlinck, "Motioncompensated adaptive inter/intra frame prediction", Proc. SPIE Vol. 1360 Visual Communication and Image Processing I V (1990), 1990, pp. 1798 1809. [14] H. Yamaguchi, "Iterative method of movement estimation for television signals", IEEE Trans. Comm., Vol. 37, No. 12, December 1989, pp. 1350 1358. [15] H. Yamaguchi, T. Sugi and K. Kinuhata, "Movementcompensated frame-frequency conversion of television signals", IEEE Trans. Comm., Vol. COM-35, No. 10, October 1987, pp. 1069-1082.

Vol.4, No. 6, November1992