- Email: [email protected]
Optimized quantization for image subband coding
Signal Processing: Image Communication 4 (1991) 65 79 Elsevier
65
Optimized quantization for image subband coding L. Vandendorpe Universitb Catholique de Louoain, Laboratoire de TM~communications, B-1348 Louoain-la-Neuve, Belgium Received ! October 1990 Revised 10 April 1991
Abstract. Subband coding of images, which can be seen as an extension of orthogonal transformations, has been introduced by Woods and O'Neil. In their paper, the subbands are further encoded by DPCM. In this study, it is proposed to use subband coding as a stand alone coding technique. The emphasis is put on the question of subband quantization. A methodology is proposed where the quantization steps of the subbands are computed so as to minimize a quantization noise power weighted by a sensitivity function of the eye. The methodology which is proposed is also valid for unequal size subbands.
1. Introduction
Subband coding of images has been widely studied for some years. It has been introduced for the coding of pictures by Woods and O'Neil [22]. In this paper, it is proposed to split the picture in 16 bands. After that, the different subbands are further decorrelated using DPCM. In [21], the picture is also decomposed in 16 bands as in [22] but samples of subbands are grouped into vectors which are encoded using vector quantization. Actually, as subband coding is obviously related to a spectral analysis of the signal, it can be used as a stand alone coding technique. This means that by using only subband coding, a sufficient spectral analysis of the source can be obtained and each subband can be PCM coded. Such a spectral analysis is nothing but something similar to what is done in transform coding [~6, 19]. Subband coding is more ge~eral in the sense that it allows for 1. longer filters than the decimation factor, 2. unequal size band splitting, 3. non-critical sampling.
Subband coding has also other advantages over differ~,nt coding techniques. It provides direct progressivity, thanks to the possibility of progressive decoding of the bands. It is also the key element of compatible or hierarchical coding [17, 15], because sub-formats of a picture are intrinsically contained in a subband scheme performed for example with a tree decomposition. Recently, non-separable twodimensional subband schemes suitable for compatible coding have also been presented in [20]. Thanks to the spectral analysis performed by subband coders, it is possible to encode each subband taking into account the sensitivity that the eye has of this subband. In particular, the purpose of this paper is to compute the quantization steps of the different subbands so as to minimize a noise power weighted by a sensitivity function of the eye. This basic idea has already been experimented in [9] for transform coding. The aim of the present paper is not to discuss the choice of a particular weighting curve but to extend the methodology to the general case of unequal size subband splitting. In a first section, the formalism of subband coding will be briefly presented. As a particular case, the connection with non-overlapI,;vg a , d
0923-5965/91/$03.50 ~, 1991 Elsevier Science Publishers B.V. All rights reserved
66
L. Vandendorpe / Optimized quantization for imag~' subband coding
overlapping transformations will be mentioned [161• In a second section, the criterion used to assess the quality of pictures will be commented on. Using the sensitivity function of the eye, the optimization of the quantization steps in subband schemes will be presented. In a third section, different ways of splitting the signal baseband will be considered. Finally, pictures coded with optimized quantization will be compared to non-optimally quantized pictures, both for subband coding and orthogonal transform schemes. The performances of subband coders will also be compared to the ones of transform coders. Finally, simulations and results will be presented and commented on.
Before transmission, the subbands are coded• At the receiving end, the reconstructed subbands are interpolated by means of zero insertion and filtering in the synthesis stage. When the analysis and synthesis filters satisfy some relationships, the system can be made of the exact reconstruction type (free of amplitude and phase distortion, and of alias)• These relationships can be summarized in a matricial equation' "
/-/o(.(2 )
"'"
2. Subband coding
2.1. General system The idea of subband coding has been introduced in 1976 by Crochiere et al., for the coding of speech [3]. A general presentation of this technique can be found for example in [ 18]. An 8-band one-dimensional system is presented in Fig. 1. The signal x to be coded is first split up by the analysis stage in different bands by bandpass filters• The outputs of these filters are downsampled, generally so as to keep the initial amount of information. When the total of subband samples is equal to the initial amount of points, the system works at critical sampling. The general system sketched in Fig. 1 is a parallel system• A particular case of subband schemes is the hierarchical structure. In the latter, a basic cell is used, which is a 2-band subband system• This basic cell is iteratively used to further split the baseband obtained at each stage, or even other subbands• Such a structure, of course, also has a parallel equivalent representation. In this case, the decimation factors of the different subbands are only powers of 2. Signal Processing: Image Communication
exp -jl2p 0
GN-I(O)
"
m
,ms
Go(t2 ) G~(t2 )
H N - I(..Q )
(1)
0 i
where H~ and Gj are the analysis and the synthesis filters, respectively, and p is an integer• The problem of synthesis of hierarchical or parallel filter banks satisfying the exact reconstruction constraint has been widely addressed in the literature recently• In [5], Johnston presented a procedure for the synthesis of quadrature mirror filters, being almost of perfect reconstruction• In [13], Smith and Barnwell generalized the concept of QMF and introduced the CQF filters, which are of non-linear phase and of the perfect reconstruction type. Some properties of perfect reconstruction FIR filter banks are presented in [19, 14]. Infinite impulse response filters have also been considered, as an example in [10, 7].
2.2. Orthogonal transformations A particular case of FIR filters used in a parallel structure is the orthogonal transformation [1]. It can be shown [16, 19] that the direct transformation (multiplication of a block of pels by the
L. Vandendorpe / Optimized quanti:ation for image subband coding
67
I
ho(n)
- - - ~(n) -~ I
N -.-
~(nN)
I N
I m '--
yi(nN)
TN
-.
y,(.N)
Tw
...
u,(n) - - ~ m - -
v,(nN)
t N
...
h(n) --~ ~ N ~
h(nN)
Vi(n)
u,(n) - . L N
So(-) ...
H +
..o
hu_s(n)
. . - . . - - e .
if(n) -- ~ N
---
y,(n.'v)
I~(n) --. * N
- ~
~(nN)
v,(n) - - * m - -
v,(,m)
Fig.
(2)
The decimation factor is given by the distance between two successive blocks. When the blocks do not overlap, this factor is equal to the filter length, i.e. the row length. In the case of overlapping blocks, the filter length is greater than the decimation factor. The inverse transformation (multiplication of the block of transform coefficients by the inverse of the transformation matrix) is equivalent to a synthesis stage. The synthesis filters are related to the columns of the inverse matrix D - ' by g~(n) = D - i ( n , i).
£
-.~
ss.,(n)
I. 8 - b a n d s u b b a n d s y s t e m .
transformation matrix) is equivalent to an analysis stage in a subband coder. The analysis filters are composed of the rows of the direct matrix D taken in reverse order. This means h~(n) = D(i, N - l - n).
..--
The optimization of coding algorithms, either for fixed bit-rates or for variable bit-rates cannot be performed exclusively on the basis of subjective tests. Even if they are at the present time still very rough [l l, 8], criterions predicting the subjective quality have to be used, at least for two reasons: - they can replace costly subjective tests; - they give an analytical expression of the perceptual parameters. 3.1.1. Q u a l i t y prediction In Allnatt's approach [2], also described in the CCIR Recommendation 405, an objective impairment measure, d, (e.g. a weighted SNR) allows the quality prediction by the logistic function: 5 Q= l + ( d / d u )~ ,
(4)
(3)
In short, transform coding is just a particular case of subband coding when unsampling factors, number of branches and filter lengths are the same.
3. Optimum quantization
3.1. Q u a l i t y definition
The quality of a coded TV picture is defined by subjective tests. The well-known five-grade quality and impairment scale is generally used. The CCIR has drawn up specifications concerning observers, grading scales and viewing conditions in Recommendation 500.
where dM and G are specific to the impairment dimension investigated and must be estimated on the basis of specific subjective tests. 3.1.2. Frequency w e i g h t e d S N R Over the years, a lot of weighting curves for random noise have been proposed. These curves have been developed for specific applications. For TV pictures, the only weighting curve widely admitted is that of the CCIR Recommendation 451-2 (Report 959-2, Note 2), for random wideband noise affecting the quality of analog television transmission. In this recommendation the noise power is expressed as the integral along the video frequency fv, expressed in MHz, of the noise Vol. 4. No. I. November 1991
L. Vandendorpe / Optimized quantization for image ~,bband coding
68
noise power. This weight is produced when considering the sensitivity function of the eye. The use of spectral density fimctions means that the noise signal is supposed to be stationary. In particular this means that the sampling phase of the subbands has to be considered as random and uniformly distributed on the region
spectral density weighted by a function expressing the eye sensitivity. The weighted noise power allows the definition of a weighted signal to noise ratio (SNRw). The quality-SNRw relationship is given in the CCIR Report 959: 5 Q - I + (SNRw/37.5) -7"
[0, No- l] x [0, N , - II
(5)
if No and N~ are the decimation factors of the subband in the horizontal and the vertical directions. In order to be able to speak about additive white noise, it has to be assumed that I. the quantization noise is not correlated with the values to be quantized, 2. the quantization errors of successive samples in a given subband are not correlated. Under these assumptions, the quantization noise spectral density of one band (i,, (,.) is given by
The CCIR weighting curves have been extended to two-dimensional random noise [9]. With f, and f~, being spatial frequencies expressed in cycle per picture height, those curves have been shown to be: 1. for luminance: W(f.,,fv) = 15.32 l +
(3.952 arctan( l / 2 L ) ) ' /
' (6)
2. for the chrominances:
r,,.,,tA,f,) W(f,.,f,.)-0.96 1 +(3.952arctan(l/2L))2
rxrv
;
-- N,,,,, " [G,,.,,(r.,.f,., ,',.f,.)l 2 O'q,i,,i 2 •
(8)
(7) where L is the distance from the screen, expressed in screen height H.
3.2. Methodology
In this equation - N,.,. is the product of the decimation factors of the subband i.,., (,. in the horizontal and vertical directions, r.,. and ry are the sampling intervals in the horizontal and vertical directions, - Gi,,g, is the synthesis filter of subband ix, iy. 2 is the quantization noise variance in the -- O'q.ix.iv subband, f,. and f,. are the horizontal and vertical spatial frequencies. A weighted noise power can be computed for each subband, by means of -
The basic idea of the optimization is to consider that the quantization adds white noise to the subbands. This noise signal is interpolated and colouted by the synthesis filters. For each subband, a quantization noise power can be computed by integrating the noise power spectral density. If this noise spectral density function is weighted by the sensitivity of the eye, a weighted noise power can be calculated. The quantization steps will be chosen so as to minimize the mean weighted noise power.
-
t~ 2 . q,w,t~ ,i~
W ( f ~ ,
f y ) ) t i , , i , . ( f x . , ~ , ) dfx dfy,
~D~,.
3.3. Weighting f a c t o r s The aim of this subsection is to compute the weights by which the noise power of the different subbands will be multiplied in the total weighted Signal Processing: Image
Commungcation
(9) where D, and D:. are the basebands of the spatial frequencies. The total weighted noise power is obtained when considering the contributions of the
L. Vandendorpe / Optimized quantication for image subband coding
different subbands" 2
~
Ml-I
0",w- E i, = 0
M2-1
E
9
(lO)
With such an assumption applied to all subbands, the weighted noise power can be expressed by
iv =0
M l - - I M2-1 0. 2
The total noise power can equivalently be written as 2 0"q,w =
Ml-I
M2-1
~ i~ =0
~
W,
2
i,,i,.0"q,i,,ir,
(11)
iv=O
W(f,.,f,.)
x IG,,.,,.(r,f,.. r,.f.,.)l 2 dr,.
(12)
l,V~,..i,, can be seen as the weighting factor of the (~,., iy)th subband and x/Ni,j,W~,.(, is its visibility factor. It depends on 1. the distance of the observer from the screen, 2. the sampling parameters of the picture, 3. the decimation factor of this subband, 4. the equivalent synthesis filter.
3.4. F i x e d bit allocation In the fixed bit allocation problem, one has to find the best decision and reconstruction levels of each subband so as to minimize the total quantization error variance under the constraint of a given average number of bits. In this study, the goal is to minimize the weighted error variance, also called weighted noise power. In [4] as an example, one can find a quantizer performance factor which provides a ratio between the quantization error variance and the signal variance. Such a relationship can be written for each subband: 2
_ _ ( ~ I ~ - R , i /2 __2
0"q,ix,iv - -
,'.v
OSB,i~. iv ,
(13)
vv
OSB,ix.!~ .
iv = 0
(14) The constraint of a given average number of bits is
i.~=0
~
,." (,. = 0
where IV,..,.
69
%' R"-R iv=O Ni.~iv
,,5,
With the Lagrange multiplier method, one can find that the optimum choice for the bit allocation (in bits per sample) is provided by .
i~,i~ W i j v C T S B , i ~ , i~
Ri,a,. = R + 2 logaL..i-[.
- - - ~ q T - - - = l ~x, ./,I • ( N j , , j v W.l,,jvO'SB.j,.j, )
(16) Equation (16) shows how the bit allocation has to be matched to the weighted subband variances. It has to be mentioned that this formula leads to real valued numbers of bits, and also to possible negative values. The bit allocation therefore has to be corrected and one has to compensate for the negative number of bits by subtracting bits allocated to other subbands. This can be done by deciding to subtract the bits one by one, so as to minimize the increase in distortion at each step. In [12], Perkins and Lookabaugh propose a bit allocation method derived from that idea: one starts with a number of bits allocated to the subbands which is zero. Each time a bit is allocated, it is to the subband which maximizes the reduction of the weighted distortion. With the bit allocation defined by 16, the weighted noise power becomes O'2SB,q,w = ~ f l - R / 2 H
(Nj, ,
W. [ , . j v 0 "2S B . j , . j , )
~' ( N , , . , ,
./x ,Jl
(17) where 0"sB.i,,~, is the variance of subband (i.,., (,.). The performance factor depends on the input signal pdf and the type of quantization law.
This expression shows that the weighted noise power is only a function of the geometric mean Vol. 4. No. I. November 1991
L. Vandendorl~e / Optimized quantization for image subband coding
70
of the weighted subband variances. Therefore, the distortion cannot only be minimized by means of an appropriate bit allocation, but also by choosing
the filter banks so as to minimize this geometric mean. Stated in other words, a second way of optimizing o.2sB,q,w should be the minimization of this geometric mean. This means that the filters have to be adapted to the signal statistics. This question is beyond the scope of this paper. It is interesting to compute the coding gain over PCM. The distortion in full band PCM is of the type
2
o.PCM,q
=afl-r/2,,r2 '-'x s
(18)
where erx2 is the full band variance. The ratio with the term provided by (17), assuming the same performance factor, becomes 2 o.x
Gsnc= H (Nj,.,j,.W~ o.2 . : ~ , j y
~I/(N,.j,.)" " "
(19)
SB.jx,./v/
J'x ,Jy
3.5. Entropy coding o f quantizer output
(20)
~'~--o ~,.=0 N~,.,6. W e denote by q~,~,, the quantization step of subband ix, iy and this parameter has to be optimized for the different subbands. Using the Lagrange m e t h o d again, one obtains 0 2 N:,~, Wi, i, o.q'i"'iy/Oqi"'6'--~l ,
(21)
where ~,i is constant. In [9], it has been s h o w n that Laplacian pdf's, the fraction with partial derivatives is proportional Signal Processing:
Image Communication
2 = $z N~.,,6W,~,.6qi,,6
(22)
or
1
q,.,.,6, or..4Jv~'"'~.6. W~,6,
(23)
This means that each subband should be quantized with a step which is inversely proportional to the square root of the decimation factor and the weighting factor. One way to proceed is to multiply each subband by the visibility factor ~/Ni,,6, W~,,6, and to use after that the same quantization step. This multiplication can be incorporated in the analysis stage, and a new analysis filter bank can be defined by multiplying each one of the analysis filters by the corresponding factor. The same quantization step for all subbands has to be used after that.
3.6. Comments
It has been shown [4] that for a signal which will be entropy coded, the optimum quantizer is very close to a linear quantizer. In this case, the quantization steps have to be determined so as to minimize the weighted noise power under the constraint of a given entropy. The constraint is now
ta~ l M~-I Hi,,6_ H.
2 to qi,.i,.. Then, one should have
Some comments can be made a b o u t these results and the interpretation of these correction factors. Let us consider the case without weightint; .'unct i o n , i.e. W ( f ~ , fy) = 1. Let us also consider the simplified situation of ideal bandpass filters for H and G. In this case, one can consider that the frequency response of the H~x.i, and Gix,6, filters is flat on an interval of area [r:,ryN~.~,]-~. But due to the decimation operation, if the H and G filters have a gain of I in linear (0 dB), an additional gain of N~x,6, has to be added somewhere. If the latter is mixed with the synthesis filters, the weighting factors for a flat sensitivity function [ W ( f x , f y ) = 1] are equal to 1, and the correction factors are equal to ~/-Ni~,6.. This additional N~,.6, gain can also be distributed between analysis and synthesis stages by multiplying each one of the 0 dB-normalized H and G filters by Vt-K~,,6. In this case, with a flat sensitivity function again, the correction factors are equal to 1.
L. Vandendorpe / Optimized quantization for image subband coding
4. Band splitting In order to see the performances of such a quantization strategy, three different splitting schemes have been considered. The first one is splitting the baseband in 8 x 8 equalsize bands. This way of splitting has been considered because it corresponds to the splitting obtained with an 8 x 8 orthogonal transformation. It is shown by the left-hand part of Fig. 2. Such splitting allows us also to make a comparison of coding efficiency and the artefacts with what is obtained with the classical DCT algorithm. Another orthogonal transformation has also been considured. It is the so-called modified symmetric DCT~ introduced by Kitajima in [6] for its computational efficiency. It is defined by Y(i)= ~ N 2 _ 1 ~ ' c(k) cos F irck ] x(k), k=o LN-11
(24) with
c(O)=c(N- 1)=0.5.
(25)
c(i#O. N - 1) = 1.
(26)
As the first and last sample of the block contribute with a weight which is half the weight of the other sami,!es, this transformation should be applied to overlapping blocks. Therefore, as the transformation matrix is square, the number of
71
analysis filters is equal to the decimation factor plus one, which means that we do not have critical sampling. The second case of splittin~g (in the middle of Fig. 2) is the octave decomposition. The basic cell is 2-band splitting of the signal. This cell recursively acts on the baseband of the signal produced at the previous step. !t is the basic decomposition which is needed for hierarchical coding in the sense that it produces directly downsampled versions of the original picture. A three-stage decomposition has been tested here. As a modification of this scheme, the one showed by the right-hand part of Fig. 2 has also been considered. The basebands containing the purely hodzontal and pure vertical frequencies which are closest to the baseband are further analysed, similarly to the baseband. Table 1 provides the values of Ni,j, W~,.,. which have been computed for the three cases of splitting in Fig. 2 for the 16 tap QMF filters given in [5]. Those QMF have been used as an example. They are not of the perfect reconstruction type, but the overall amplitude distortion is negligible with respect to the noise introduced by the quantization process. The distance L from the screen is equal to 4H. The weighting factors have been computed by means of (12), and the weighting curve has been provided by (6). The coefficients are in the same order as the position of one subband on the frequency axis when
Fig. 2. Baseband splittings. Vol. 4. No. I. November 1991
L. Vandendorpe / Optimized quantization for #nage subband coding
72 Table !
3 stages 16 tap QMF--splitting I 1024.00 117.70 103.44 43.58 8.04 6.53 25.68 16.90 0.89 0.85 1.40 1.31 4.75 4.08 2.40 2.16
9.63 7.44 3.09 5.13 0.67 0.97 2.27 !.41
30.28 18.30 4.7q 9.91 0.78 !.17 3.22 1.83
1.08 i.01 0.76 0.91 0.35 0.43 0.67 0.53
!.69 1.55 !.07 1.34 0.42 0.54 0.91 0.69
5.71 4.72 2.34 3.54 0.61 0.85 !.80 1.19
2.89 2.53 1.52 2.06 0.50 0.67 !.24 0.89
3 stages 16 tap QMF--splitting 2 1024.00 1 1 7 . 7 0 103.44 43.58 3.57 3.57 3.57 3.57 0.12 0.12 0,12 0.12 0.12 0.12 0.12 0.12
4.10 4.10 1.43 !.43 0.12 0.12 0.12 0.12
4.10 4.10 1.43 !.43 0.12 0.12 0.12 0.12
0.14 0.14 0.14 0.14 0.05 0,05 0.05 0.05
0.14 0.14 0.14 0.14 0.05 0.05 0.05 0.05
0.14 0.14 0.14 0.14 0.05 0.05 0.05 0.05
0.14 0.14 0.14 0.14 0.05 0.05 0.05 0.05
0.33 0.33 0.25 0.25 0.05 0.05 0.05 0.05
0.33 0.33 0.25 0.25 0.05 0.05 0.05 0.05
0.99 0.99 0.59 0.59 0.05 0.05 0.05 0.05
0.99 0.99 0.59 0.59 0.05 0,05 0.05 0.05
3 stages 16 ta~ QMF--splitting 3 1024.00 !17.70 103.44 43,58 8.04 6.53 25.68 16.90 0.28 0.28 0.28 0.28 0.84 0.84 0.84 0.84
9.63 7.44 !.43 !.43 0.22 0.22 0.54 0.54
30.28 18.30 !.43 !.43 0.22 0.22 0.54 0.54
the splitting is produced by a hierarchical structure. Let us assume an N-stage 2-band splitting producing 2N bands. If H0 is the analysis Iowpass filter, and H~ the highpass filter, one N-bit number can be given to each output of the complete analysis stage. The value of bit number j is 0 (respectively 1) when at stagej the signal is lowpass (respectively highpass) filtered. As an example, the MSB is 0 if the first step is lowpass filtering, and so on. If this number is k, it can be shown that the position of this band on the frequency axis is given by I where I is the binary to Gray converted value of k. It is provided by (27)
IN-I = k u - i , li=ki(~ki+l
for 0~
Signal Processing: Image Communwation
(28)
where 0) denotes modulo 2 addition. In particular, in a three-stage splitting, the position of the band produced by 3 consecutive highpass filters is given by k = ( 1 , 1, 1)=7 and l = ( 1 , 0 , 1)=5.
5. Simulation results
The following situations have been simulated, for an entropy coding of the quantized outputs, and for the same entropy (0.64 bit/pixel): 1. 8 x 8-DCT, 2. 9 x 9-MSDCT, 3. subband coding with separable QMF filters of order 16 [5], for the first and the third splitting presented in Fig. 2. Moreover, they have been stimulated with and without weighting. The following comments can be made: 1. the artefacts producted with 'real' subband systems are different from the effects obtained with transform coders: (a) on the subband coded pictures, the blocking effect is not present at all. The synthesis filtering operation can be seen as an averaging operation over several neighbouring subband samples. Due to this averaging operation, the blocking effect associated with a pattern at the subsampling frequency is removed. But it is also required to choose filter banks with bandpass filters (all of them except the filter around the DC) having exact zero valued transmittance at the zero frequency. Otherwise, due to the coding of those subbands, the highpass components can be encoded with insufficient accuracy for the Iowpass subband reconstruction and may produce blocking-like artefacts. (b) due to the quite long equivalent impulse responses of the reconstruction filters, one artefact runs over a greater distance with subband coding,
L. Vandendorpe / Optimized quantization for image subband coding
73
!
J
•
~:
9
Fig. 3. Original DICK.
Fig. 4. Weighted and non-weighted DCT at 0.64 bit/pixel. Vol. 4. No. I. November 1991
L. Vandendorpe / Optimized quantization for image subband coding
74
Fig. 5. Weighted and non-weighted M S D C T at 0.64 bit/pixel.
tj <
if,-
C:
/
t
Fig. 6. Non-weighted subband (first splitting) at 0.64 bit/pixel. Signal Processing: Image Comnmnication
L. Vandendorpe / Optimi:ed quanti:ation for image subband coding
75
Fig. 7. Weighted subband (first splitting) at 0.64 bit/pixel.
i
,
*
ll; ¸
i
P
i
Fig. 8. Non-weighted subband (third splitting) at 0.64 bit/pixel. Vol. 4. No. !. November 1991
76
L. Vandendorpe / Optimized quantization for image subband coding
Fig. 9. Weighted subband (third splitting)
at
0.64 bit/pixel.
Fig. 10. Original LENA, weighted DCT coded, first splitting subband coding with and without weighting at 0.64 bit/pixel. Signal Processing: Image Communication
L. Vandendorpe / Optimized quantization for image subband coding
77
Fig. ! 1. Original LENA, weighted DCT coded, third splitting subband coding with and without weighting at 0.64 b;t/pixel.
(c) the pictures are cleaner with orthogonal transformations; there is more granular noise with subban.d coding, but also the texture rendition is better. 2. the weighting modifies the noise structure; the most general impression for the 'real' subband case is that it reduces the visibility of the impulse responses of high frequency bands. Actually, it is obvious that the quantization introduces noise which prevents the perfect reconstruction relationships from being achieved. Weighting modifies the amount of aliasing produced in the different frequency intervals. 3. the M S D C T is as efficient as the DCT, even if the former is not of the critical sampling type. Due to the overlap of one pel in both directions in the reconstruction stage of the MSDCT, the blocking artefacts are significantly reduced. The following pictures are presented: 1. Fig. 3 is the original ' D I C K ' picture,
2. Fig. 4 is a comparison between weighted and non-weighted DCT at 0.64 bit/pixel, 3. Fig. 5 is a comparison between the weighted and non-weighted M S D C T at 0.64 bit/pixel, 4. Fig. 6 shows subband coding with the first way of splitting of Fig. 2 and non-weighted quantization at 0.64 bit/pixel, 5. Fig. 7 shows subband coding with the first way of splitting of Fig. 2 and weighted quantization at 0.64 bit/pixel, 6. Fig. 8 shows subband coding with the third way of splitting of Fig. 2 and non-weighted quantization at 0.64 bit/pixel, 7. Fig. 9 shows subband coding with the third way of splitting of Fig. 2 and weighted quantization at 0.64 bit/pixel, 8. Fig. 10 shows the original ' L E N A 256 x 256' (upper left corner), the weighted DCT coded picture (0.64 bit/pixel) (lower left), the subband result (first way of splitting) with weighting Vol. 4. No. I. November 1991
L. Vandendorpe / Optimized quantization for image subband coding
78 Table 2
Weighted noise power in dB (0.64 bit/pixel) DICK
Splitting 1 Splitting 3
LENA256
weighted q
non-weighted q
weighted q
non-weighted q
0.9 -2.6
3.5 2.4
5.0 1.9
8. I 7.6
(upper right) and without weighting (lower right), 9. Fig. 11 shows the original 'LENA' (upper left corner), the weighted DCT coded picture (0.64 bit/pixel) (lower left), the subband result (third way of splitting) with weighting (upper right) and without weighting (lower right). As an example, the weighted noise powers (in dB) have been computed for the simulations with QMF filters, for DICK and LENA256 and the first and third splitting cases. The results are summarized in Table 2. Obviously, the weighted noise powers are smaller with weighted quantization. The difference from a non-weighted quantization is of a few dB for the range of entropy measured here. 6. Conclusions This paper has presented a method for optimizing the quantization in a subband coder. It includes the case of orthogonal transformations and nonequalsize subband systems. The optimization has been done by taking into account a sensitivity function of the eye and defining a new distortion measure. The cases of fixed bit allocation and entropy coded quantizer outputs have been considered. The so-called weighted quantization allows for a smoother noise structure and less visible impulse responses. 7. Acknowledgments The author would like to thank Prof Delogne (UCL) and Dr Macq (Philips Research Laboratory of Belgium) for their help with this work. Signal Processing: Image Communication
References [!] N. Ahmed and K.R. Rao, Orthogonal Transformsjot Digital Signal Processing, Springer, Berlin, 1975. [2] J. AUnatt, "Television measurements through psychophysics to subjective picture quality", Radio and Electronic Engineer, Vol. 49, !979. [3] R.E. Crochiere, S.A. Webber and J.L. Flanagan, "Digital coding of speech in subbands", Bell. System Techn. J., Voi. 55, October 1976, pp. 1069-1085. [4] N.S. Jayant and P. Noll, Digital Coding of Waveforms, Signal Processing Series, Prentice-Hall, Englewood Cliffs, NJ, 1984. [5] J.D. Johnston, "A filter family designed for use in quadrature mirror filter banks", Proc. 1980 IEEE lnternat. Conf. Acoust. Speech Signal Process., Hartford, CT, May 1977, pp, 191- 195. [6] H. Kitajima, "A symmetric cosine transform", I E E E Trans. Comput., Vol. C-29, No. 4, April 1980. [7] T. Kronander, "A new approach to recursive mirror filters with a special application in subband coding of images", I E E E Trans. Acoust. Speech Signal Process., Vol. ASSP36, No. 9, September 1988. [8] F. Lukas and Z. Budrikis, "Picture quality prediction based on a visual model", I E E E Trans. Comm., Vol. 7, 1982. [9] B. Macq, Perceptual transforms and universal entropy coding for an integrated approach tu picture coding, Ph.D. Thesis, Universit~ Catholique de Louvain, September 1989. [I0] P.C. Millar, "Recursive quadrature mirror filtersCriteria specification and design method", IEEE Trans. Acoust. Speech Signal Process., Vol. ASSP-33, No. 2, April 1985. [11] M. Miyahara, "Quality assesments for visual services", IEEE Commun. Mag., Voi. 10, 1988. [12] M.G. Perkins and T. Lookabaugh, "A psychophysically justified bit allocation algorithm for subband image coding systems", Proc. 1989 IEEE Internat. Conf. Acousi. Speech Signal Process., Glasgow, Scotland, May 1989, pp. 1815 1818. [13] M.J.T. Smith and T.P. Barnwell, "Exact reconstruction techniques for tree-structured subband coders", IEEE Trans. Acoust. Speech Signal Process., Vol, ASSP-34, No. 34, June 1986. [14] P.P. Vaidyanathan, "Quadrature mirror filter banks, Mband extensions and perfect reconstruction technique", IEEE ASSP Mag., Vol. 4, July 1987, pp. 4-20.
L. Vandendorpe / Optimized quantization for image subband coding [15] L. Vandendorpe, "Hierarchical encoding of HDTV by transform coefficients splitting: A comparison between DCT and MSDCT", Picture Coding Symposium, 26-28 March 1990, MIT, Cambridge, USA, pp. 3.2-1, 3.2-2. [ 16] L. Vandendorpe and P. Deiogne, "Hierarchical encoding of HDTV by transform coefficients splitting", SPIE/ SPSE Symposium on Electronic Imaging, Conference on Image Processing ,41gorithms and Techniques, February 1990, Santa Clara, USA, pp. 343-354. [ i 7] L. Vandendorpe and B. Macq, "Optimum quality and progressive resolution of video signals", Les Annales des TblEcommunications, Vol. 45, Nos. 9-10, 1990, pp. 487-502. [18] M. Vetterli, "A theory of multirate filter banks", IEEE Trans. ,4coust. Speech Signal Process., Vol. ASSP-35, No. 3, March 1987.
79
[19] M. Vetterli and D. Le Gall, "Perfect reconstruction FIR filter banks: Some properties and factorizations", IEEE Trans. ,4coust. Speech Signal Process., Vo!. ASSP-37, No. 7, Jaly 1989. [20] M. Vetterli, J. Kova~evi/: and D.J. LeGall. "Perfect reconstruction filter banks for HDTV representation and coding", Signal Processing: Image Communication, Vol. 2, No. 3, October 1990, pp. 349-363. [211 P.H. Westerink, D.E. Boekee, J. Biemond and J.W. Woods, "Subband coding of images using vector quantization", IEEE Trans. Comm., Vol. 36, No. 6, June 88. [22] J.W. Woods and S. O'Neil, "Subband coding of images", IEEE Trans. Acoust. Speech Signal Process., Vol. ASSP34, No. 5, October 1986.
Vol. 4. No. I. November 1991
65
Optimized quantization for image subband coding L. Vandendorpe Universitb Catholique de Louoain, Laboratoire de TM~communications, B-1348 Louoain-la-Neuve, Belgium Received ! October 1990 Revised 10 April 1991
Abstract. Subband coding of images, which can be seen as an extension of orthogonal transformations, has been introduced by Woods and O'Neil. In their paper, the subbands are further encoded by DPCM. In this study, it is proposed to use subband coding as a stand alone coding technique. The emphasis is put on the question of subband quantization. A methodology is proposed where the quantization steps of the subbands are computed so as to minimize a quantization noise power weighted by a sensitivity function of the eye. The methodology which is proposed is also valid for unequal size subbands.
1. Introduction
Subband coding of images has been widely studied for some years. It has been introduced for the coding of pictures by Woods and O'Neil [22]. In this paper, it is proposed to split the picture in 16 bands. After that, the different subbands are further decorrelated using DPCM. In [21], the picture is also decomposed in 16 bands as in [22] but samples of subbands are grouped into vectors which are encoded using vector quantization. Actually, as subband coding is obviously related to a spectral analysis of the signal, it can be used as a stand alone coding technique. This means that by using only subband coding, a sufficient spectral analysis of the source can be obtained and each subband can be PCM coded. Such a spectral analysis is nothing but something similar to what is done in transform coding [~6, 19]. Subband coding is more ge~eral in the sense that it allows for 1. longer filters than the decimation factor, 2. unequal size band splitting, 3. non-critical sampling.
Subband coding has also other advantages over differ~,nt coding techniques. It provides direct progressivity, thanks to the possibility of progressive decoding of the bands. It is also the key element of compatible or hierarchical coding [17, 15], because sub-formats of a picture are intrinsically contained in a subband scheme performed for example with a tree decomposition. Recently, non-separable twodimensional subband schemes suitable for compatible coding have also been presented in [20]. Thanks to the spectral analysis performed by subband coders, it is possible to encode each subband taking into account the sensitivity that the eye has of this subband. In particular, the purpose of this paper is to compute the quantization steps of the different subbands so as to minimize a noise power weighted by a sensitivity function of the eye. This basic idea has already been experimented in [9] for transform coding. The aim of the present paper is not to discuss the choice of a particular weighting curve but to extend the methodology to the general case of unequal size subband splitting. In a first section, the formalism of subband coding will be briefly presented. As a particular case, the connection with non-overlapI,;vg a , d
0923-5965/91/$03.50 ~, 1991 Elsevier Science Publishers B.V. All rights reserved
66
L. Vandendorpe / Optimized quantization for imag~' subband coding
overlapping transformations will be mentioned [161• In a second section, the criterion used to assess the quality of pictures will be commented on. Using the sensitivity function of the eye, the optimization of the quantization steps in subband schemes will be presented. In a third section, different ways of splitting the signal baseband will be considered. Finally, pictures coded with optimized quantization will be compared to non-optimally quantized pictures, both for subband coding and orthogonal transform schemes. The performances of subband coders will also be compared to the ones of transform coders. Finally, simulations and results will be presented and commented on.
Before transmission, the subbands are coded• At the receiving end, the reconstructed subbands are interpolated by means of zero insertion and filtering in the synthesis stage. When the analysis and synthesis filters satisfy some relationships, the system can be made of the exact reconstruction type (free of amplitude and phase distortion, and of alias)• These relationships can be summarized in a matricial equation' "
/-/o(.(2 )
"'"
2. Subband coding
2.1. General system The idea of subband coding has been introduced in 1976 by Crochiere et al., for the coding of speech [3]. A general presentation of this technique can be found for example in [ 18]. An 8-band one-dimensional system is presented in Fig. 1. The signal x to be coded is first split up by the analysis stage in different bands by bandpass filters• The outputs of these filters are downsampled, generally so as to keep the initial amount of information. When the total of subband samples is equal to the initial amount of points, the system works at critical sampling. The general system sketched in Fig. 1 is a parallel system• A particular case of subband schemes is the hierarchical structure. In the latter, a basic cell is used, which is a 2-band subband system• This basic cell is iteratively used to further split the baseband obtained at each stage, or even other subbands• Such a structure, of course, also has a parallel equivalent representation. In this case, the decimation factors of the different subbands are only powers of 2. Signal Processing: Image Communication
exp -jl2p 0
GN-I(O)
"
m
,ms
Go(t2 ) G~(t2 )
H N - I(..Q )
(1)
0 i
where H~ and Gj are the analysis and the synthesis filters, respectively, and p is an integer• The problem of synthesis of hierarchical or parallel filter banks satisfying the exact reconstruction constraint has been widely addressed in the literature recently• In [5], Johnston presented a procedure for the synthesis of quadrature mirror filters, being almost of perfect reconstruction• In [13], Smith and Barnwell generalized the concept of QMF and introduced the CQF filters, which are of non-linear phase and of the perfect reconstruction type. Some properties of perfect reconstruction FIR filter banks are presented in [19, 14]. Infinite impulse response filters have also been considered, as an example in [10, 7].
2.2. Orthogonal transformations A particular case of FIR filters used in a parallel structure is the orthogonal transformation [1]. It can be shown [16, 19] that the direct transformation (multiplication of a block of pels by the
L. Vandendorpe / Optimized quanti:ation for image subband coding
67
I
ho(n)
- - - ~(n) -~ I
N -.-
~(nN)
I N
I m '--
yi(nN)
TN
-.
y,(.N)
Tw
...
u,(n) - - ~ m - -
v,(nN)
t N
...
h(n) --~ ~ N ~
h(nN)
Vi(n)
u,(n) - . L N
So(-) ...
H +
..o
hu_s(n)
. . - . . - - e .
if(n) -- ~ N
---
y,(n.'v)
I~(n) --. * N
- ~
~(nN)
v,(n) - - * m - -
v,(,m)
Fig.
(2)
The decimation factor is given by the distance between two successive blocks. When the blocks do not overlap, this factor is equal to the filter length, i.e. the row length. In the case of overlapping blocks, the filter length is greater than the decimation factor. The inverse transformation (multiplication of the block of transform coefficients by the inverse of the transformation matrix) is equivalent to a synthesis stage. The synthesis filters are related to the columns of the inverse matrix D - ' by g~(n) = D - i ( n , i).
£
-.~
ss.,(n)
I. 8 - b a n d s u b b a n d s y s t e m .
transformation matrix) is equivalent to an analysis stage in a subband coder. The analysis filters are composed of the rows of the direct matrix D taken in reverse order. This means h~(n) = D(i, N - l - n).
..--
The optimization of coding algorithms, either for fixed bit-rates or for variable bit-rates cannot be performed exclusively on the basis of subjective tests. Even if they are at the present time still very rough [l l, 8], criterions predicting the subjective quality have to be used, at least for two reasons: - they can replace costly subjective tests; - they give an analytical expression of the perceptual parameters. 3.1.1. Q u a l i t y prediction In Allnatt's approach [2], also described in the CCIR Recommendation 405, an objective impairment measure, d, (e.g. a weighted SNR) allows the quality prediction by the logistic function: 5 Q= l + ( d / d u )~ ,
(4)
(3)
In short, transform coding is just a particular case of subband coding when unsampling factors, number of branches and filter lengths are the same.
3. Optimum quantization
3.1. Q u a l i t y definition
The quality of a coded TV picture is defined by subjective tests. The well-known five-grade quality and impairment scale is generally used. The CCIR has drawn up specifications concerning observers, grading scales and viewing conditions in Recommendation 500.
where dM and G are specific to the impairment dimension investigated and must be estimated on the basis of specific subjective tests. 3.1.2. Frequency w e i g h t e d S N R Over the years, a lot of weighting curves for random noise have been proposed. These curves have been developed for specific applications. For TV pictures, the only weighting curve widely admitted is that of the CCIR Recommendation 451-2 (Report 959-2, Note 2), for random wideband noise affecting the quality of analog television transmission. In this recommendation the noise power is expressed as the integral along the video frequency fv, expressed in MHz, of the noise Vol. 4. No. I. November 1991
L. Vandendorpe / Optimized quantization for image ~,bband coding
68
noise power. This weight is produced when considering the sensitivity function of the eye. The use of spectral density fimctions means that the noise signal is supposed to be stationary. In particular this means that the sampling phase of the subbands has to be considered as random and uniformly distributed on the region
spectral density weighted by a function expressing the eye sensitivity. The weighted noise power allows the definition of a weighted signal to noise ratio (SNRw). The quality-SNRw relationship is given in the CCIR Report 959: 5 Q - I + (SNRw/37.5) -7"
[0, No- l] x [0, N , - II
(5)
if No and N~ are the decimation factors of the subband in the horizontal and the vertical directions. In order to be able to speak about additive white noise, it has to be assumed that I. the quantization noise is not correlated with the values to be quantized, 2. the quantization errors of successive samples in a given subband are not correlated. Under these assumptions, the quantization noise spectral density of one band (i,, (,.) is given by
The CCIR weighting curves have been extended to two-dimensional random noise [9]. With f, and f~, being spatial frequencies expressed in cycle per picture height, those curves have been shown to be: 1. for luminance: W(f.,,fv) = 15.32 l +
(3.952 arctan( l / 2 L ) ) ' /
' (6)
2. for the chrominances:
r,,.,,tA,f,) W(f,.,f,.)-0.96 1 +(3.952arctan(l/2L))2
rxrv
;
-- N,,,,, " [G,,.,,(r.,.f,., ,',.f,.)l 2 O'q,i,,i 2 •
(8)
(7) where L is the distance from the screen, expressed in screen height H.
3.2. Methodology
In this equation - N,.,. is the product of the decimation factors of the subband i.,., (,. in the horizontal and vertical directions, r.,. and ry are the sampling intervals in the horizontal and vertical directions, - Gi,,g, is the synthesis filter of subband ix, iy. 2 is the quantization noise variance in the -- O'q.ix.iv subband, f,. and f,. are the horizontal and vertical spatial frequencies. A weighted noise power can be computed for each subband, by means of -
The basic idea of the optimization is to consider that the quantization adds white noise to the subbands. This noise signal is interpolated and colouted by the synthesis filters. For each subband, a quantization noise power can be computed by integrating the noise power spectral density. If this noise spectral density function is weighted by the sensitivity of the eye, a weighted noise power can be calculated. The quantization steps will be chosen so as to minimize the mean weighted noise power.
-
t~ 2 . q,w,t~ ,i~
W ( f ~ ,
f y ) ) t i , , i , . ( f x . , ~ , ) dfx dfy,
~D~,.
3.3. Weighting f a c t o r s The aim of this subsection is to compute the weights by which the noise power of the different subbands will be multiplied in the total weighted Signal Processing: Image
Commungcation
(9) where D, and D:. are the basebands of the spatial frequencies. The total weighted noise power is obtained when considering the contributions of the
L. Vandendorpe / Optimized quantication for image subband coding
different subbands" 2
~
Ml-I
0",w- E i, = 0
M2-1
E
9
(lO)
With such an assumption applied to all subbands, the weighted noise power can be expressed by
iv =0
M l - - I M2-1 0. 2
The total noise power can equivalently be written as 2 0"q,w =
Ml-I
M2-1
~ i~ =0
~
W,
2
i,,i,.0"q,i,,ir,
(11)
iv=O
W(f,.,f,.)
x IG,,.,,.(r,f,.. r,.f.,.)l 2 dr,.
(12)
l,V~,..i,, can be seen as the weighting factor of the (~,., iy)th subband and x/Ni,j,W~,.(, is its visibility factor. It depends on 1. the distance of the observer from the screen, 2. the sampling parameters of the picture, 3. the decimation factor of this subband, 4. the equivalent synthesis filter.
3.4. F i x e d bit allocation In the fixed bit allocation problem, one has to find the best decision and reconstruction levels of each subband so as to minimize the total quantization error variance under the constraint of a given average number of bits. In this study, the goal is to minimize the weighted error variance, also called weighted noise power. In [4] as an example, one can find a quantizer performance factor which provides a ratio between the quantization error variance and the signal variance. Such a relationship can be written for each subband: 2
_ _ ( ~ I ~ - R , i /2 __2
0"q,ix,iv - -
,'.v
OSB,i~. iv ,
(13)
vv
OSB,ix.!~ .
iv = 0
(14) The constraint of a given average number of bits is
i.~=0
~
,." (,. = 0
where IV,..,.
69
%' R"-R iv=O Ni.~iv
,,5,
With the Lagrange multiplier method, one can find that the optimum choice for the bit allocation (in bits per sample) is provided by .
i~,i~ W i j v C T S B , i ~ , i~
Ri,a,. = R + 2 logaL..i-[.
- - - ~ q T - - - = l ~x, ./,I • ( N j , , j v W.l,,jvO'SB.j,.j, )
(16) Equation (16) shows how the bit allocation has to be matched to the weighted subband variances. It has to be mentioned that this formula leads to real valued numbers of bits, and also to possible negative values. The bit allocation therefore has to be corrected and one has to compensate for the negative number of bits by subtracting bits allocated to other subbands. This can be done by deciding to subtract the bits one by one, so as to minimize the increase in distortion at each step. In [12], Perkins and Lookabaugh propose a bit allocation method derived from that idea: one starts with a number of bits allocated to the subbands which is zero. Each time a bit is allocated, it is to the subband which maximizes the reduction of the weighted distortion. With the bit allocation defined by 16, the weighted noise power becomes O'2SB,q,w = ~ f l - R / 2 H
(Nj, ,
W. [ , . j v 0 "2S B . j , . j , )
~' ( N , , . , ,
./x ,Jl
(17) where 0"sB.i,,~, is the variance of subband (i.,., (,.). The performance factor depends on the input signal pdf and the type of quantization law.
This expression shows that the weighted noise power is only a function of the geometric mean Vol. 4. No. I. November 1991
L. Vandendorl~e / Optimized quantization for image subband coding
70
of the weighted subband variances. Therefore, the distortion cannot only be minimized by means of an appropriate bit allocation, but also by choosing
the filter banks so as to minimize this geometric mean. Stated in other words, a second way of optimizing o.2sB,q,w should be the minimization of this geometric mean. This means that the filters have to be adapted to the signal statistics. This question is beyond the scope of this paper. It is interesting to compute the coding gain over PCM. The distortion in full band PCM is of the type
2
o.PCM,q
=afl-r/2,,r2 '-'x s
(18)
where erx2 is the full band variance. The ratio with the term provided by (17), assuming the same performance factor, becomes 2 o.x
Gsnc= H (Nj,.,j,.W~ o.2 . : ~ , j y
~I/(N,.j,.)" " "
(19)
SB.jx,./v/
J'x ,Jy
3.5. Entropy coding o f quantizer output
(20)
~'~--o ~,.=0 N~,.,6. W e denote by q~,~,, the quantization step of subband ix, iy and this parameter has to be optimized for the different subbands. Using the Lagrange m e t h o d again, one obtains 0 2 N:,~, Wi, i, o.q'i"'iy/Oqi"'6'--~l ,
(21)
where ~,i is constant. In [9], it has been s h o w n that Laplacian pdf's, the fraction with partial derivatives is proportional Signal Processing:
Image Communication
2 = $z N~.,,6W,~,.6qi,,6
(22)
or
1
q,.,.,6, or..4Jv~'"'~.6. W~,6,
(23)
This means that each subband should be quantized with a step which is inversely proportional to the square root of the decimation factor and the weighting factor. One way to proceed is to multiply each subband by the visibility factor ~/Ni,,6, W~,,6, and to use after that the same quantization step. This multiplication can be incorporated in the analysis stage, and a new analysis filter bank can be defined by multiplying each one of the analysis filters by the corresponding factor. The same quantization step for all subbands has to be used after that.
3.6. Comments
It has been shown [4] that for a signal which will be entropy coded, the optimum quantizer is very close to a linear quantizer. In this case, the quantization steps have to be determined so as to minimize the weighted noise power under the constraint of a given entropy. The constraint is now
ta~ l M~-I Hi,,6_ H.
2 to qi,.i,.. Then, one should have
Some comments can be made a b o u t these results and the interpretation of these correction factors. Let us consider the case without weightint; .'unct i o n , i.e. W ( f ~ , fy) = 1. Let us also consider the simplified situation of ideal bandpass filters for H and G. In this case, one can consider that the frequency response of the H~x.i, and Gix,6, filters is flat on an interval of area [r:,ryN~.~,]-~. But due to the decimation operation, if the H and G filters have a gain of I in linear (0 dB), an additional gain of N~x,6, has to be added somewhere. If the latter is mixed with the synthesis filters, the weighting factors for a flat sensitivity function [ W ( f x , f y ) = 1] are equal to 1, and the correction factors are equal to ~/-Ni~,6.. This additional N~,.6, gain can also be distributed between analysis and synthesis stages by multiplying each one of the 0 dB-normalized H and G filters by Vt-K~,,6. In this case, with a flat sensitivity function again, the correction factors are equal to 1.
L. Vandendorpe / Optimized quantization for image subband coding
4. Band splitting In order to see the performances of such a quantization strategy, three different splitting schemes have been considered. The first one is splitting the baseband in 8 x 8 equalsize bands. This way of splitting has been considered because it corresponds to the splitting obtained with an 8 x 8 orthogonal transformation. It is shown by the left-hand part of Fig. 2. Such splitting allows us also to make a comparison of coding efficiency and the artefacts with what is obtained with the classical DCT algorithm. Another orthogonal transformation has also been considured. It is the so-called modified symmetric DCT~ introduced by Kitajima in [6] for its computational efficiency. It is defined by Y(i)= ~ N 2 _ 1 ~ ' c(k) cos F irck ] x(k), k=o LN-11
(24) with
c(O)=c(N- 1)=0.5.
(25)
c(i#O. N - 1) = 1.
(26)
As the first and last sample of the block contribute with a weight which is half the weight of the other sami,!es, this transformation should be applied to overlapping blocks. Therefore, as the transformation matrix is square, the number of
71
analysis filters is equal to the decimation factor plus one, which means that we do not have critical sampling. The second case of splittin~g (in the middle of Fig. 2) is the octave decomposition. The basic cell is 2-band splitting of the signal. This cell recursively acts on the baseband of the signal produced at the previous step. !t is the basic decomposition which is needed for hierarchical coding in the sense that it produces directly downsampled versions of the original picture. A three-stage decomposition has been tested here. As a modification of this scheme, the one showed by the right-hand part of Fig. 2 has also been considered. The basebands containing the purely hodzontal and pure vertical frequencies which are closest to the baseband are further analysed, similarly to the baseband. Table 1 provides the values of Ni,j, W~,.,. which have been computed for the three cases of splitting in Fig. 2 for the 16 tap QMF filters given in [5]. Those QMF have been used as an example. They are not of the perfect reconstruction type, but the overall amplitude distortion is negligible with respect to the noise introduced by the quantization process. The distance L from the screen is equal to 4H. The weighting factors have been computed by means of (12), and the weighting curve has been provided by (6). The coefficients are in the same order as the position of one subband on the frequency axis when
Fig. 2. Baseband splittings. Vol. 4. No. I. November 1991
L. Vandendorpe / Optimized quantization for #nage subband coding
72 Table !
3 stages 16 tap QMF--splitting I 1024.00 117.70 103.44 43.58 8.04 6.53 25.68 16.90 0.89 0.85 1.40 1.31 4.75 4.08 2.40 2.16
9.63 7.44 3.09 5.13 0.67 0.97 2.27 !.41
30.28 18.30 4.7q 9.91 0.78 !.17 3.22 1.83
1.08 i.01 0.76 0.91 0.35 0.43 0.67 0.53
!.69 1.55 !.07 1.34 0.42 0.54 0.91 0.69
5.71 4.72 2.34 3.54 0.61 0.85 !.80 1.19
2.89 2.53 1.52 2.06 0.50 0.67 !.24 0.89
3 stages 16 tap QMF--splitting 2 1024.00 1 1 7 . 7 0 103.44 43.58 3.57 3.57 3.57 3.57 0.12 0.12 0,12 0.12 0.12 0.12 0.12 0.12
4.10 4.10 1.43 !.43 0.12 0.12 0.12 0.12
4.10 4.10 1.43 !.43 0.12 0.12 0.12 0.12
0.14 0.14 0.14 0.14 0.05 0,05 0.05 0.05
0.14 0.14 0.14 0.14 0.05 0.05 0.05 0.05
0.14 0.14 0.14 0.14 0.05 0.05 0.05 0.05
0.14 0.14 0.14 0.14 0.05 0.05 0.05 0.05
0.33 0.33 0.25 0.25 0.05 0.05 0.05 0.05
0.33 0.33 0.25 0.25 0.05 0.05 0.05 0.05
0.99 0.99 0.59 0.59 0.05 0.05 0.05 0.05
0.99 0.99 0.59 0.59 0.05 0,05 0.05 0.05
3 stages 16 ta~ QMF--splitting 3 1024.00 !17.70 103.44 43,58 8.04 6.53 25.68 16.90 0.28 0.28 0.28 0.28 0.84 0.84 0.84 0.84
9.63 7.44 !.43 !.43 0.22 0.22 0.54 0.54
30.28 18.30 !.43 !.43 0.22 0.22 0.54 0.54
the splitting is produced by a hierarchical structure. Let us assume an N-stage 2-band splitting producing 2N bands. If H0 is the analysis Iowpass filter, and H~ the highpass filter, one N-bit number can be given to each output of the complete analysis stage. The value of bit number j is 0 (respectively 1) when at stagej the signal is lowpass (respectively highpass) filtered. As an example, the MSB is 0 if the first step is lowpass filtering, and so on. If this number is k, it can be shown that the position of this band on the frequency axis is given by I where I is the binary to Gray converted value of k. It is provided by (27)
IN-I = k u - i , li=ki(~ki+l
for 0~
Signal Processing: Image Communwation
(28)
where 0) denotes modulo 2 addition. In particular, in a three-stage splitting, the position of the band produced by 3 consecutive highpass filters is given by k = ( 1 , 1, 1)=7 and l = ( 1 , 0 , 1)=5.
5. Simulation results
The following situations have been simulated, for an entropy coding of the quantized outputs, and for the same entropy (0.64 bit/pixel): 1. 8 x 8-DCT, 2. 9 x 9-MSDCT, 3. subband coding with separable QMF filters of order 16 [5], for the first and the third splitting presented in Fig. 2. Moreover, they have been stimulated with and without weighting. The following comments can be made: 1. the artefacts producted with 'real' subband systems are different from the effects obtained with transform coders: (a) on the subband coded pictures, the blocking effect is not present at all. The synthesis filtering operation can be seen as an averaging operation over several neighbouring subband samples. Due to this averaging operation, the blocking effect associated with a pattern at the subsampling frequency is removed. But it is also required to choose filter banks with bandpass filters (all of them except the filter around the DC) having exact zero valued transmittance at the zero frequency. Otherwise, due to the coding of those subbands, the highpass components can be encoded with insufficient accuracy for the Iowpass subband reconstruction and may produce blocking-like artefacts. (b) due to the quite long equivalent impulse responses of the reconstruction filters, one artefact runs over a greater distance with subband coding,
L. Vandendorpe / Optimized quantization for image subband coding
73
!
J
•
~:
9
Fig. 3. Original DICK.
Fig. 4. Weighted and non-weighted DCT at 0.64 bit/pixel. Vol. 4. No. I. November 1991
L. Vandendorpe / Optimized quantization for image subband coding
74
Fig. 5. Weighted and non-weighted M S D C T at 0.64 bit/pixel.
tj <
if,-
C:
/
t
Fig. 6. Non-weighted subband (first splitting) at 0.64 bit/pixel. Signal Processing: Image Comnmnication
L. Vandendorpe / Optimi:ed quanti:ation for image subband coding
75
Fig. 7. Weighted subband (first splitting) at 0.64 bit/pixel.
i
,
*
ll; ¸
i
P
i
Fig. 8. Non-weighted subband (third splitting) at 0.64 bit/pixel. Vol. 4. No. !. November 1991
76
L. Vandendorpe / Optimized quantization for image subband coding
Fig. 9. Weighted subband (third splitting)
at
0.64 bit/pixel.
Fig. 10. Original LENA, weighted DCT coded, first splitting subband coding with and without weighting at 0.64 bit/pixel. Signal Processing: Image Communication
L. Vandendorpe / Optimized quantization for image subband coding
77
Fig. ! 1. Original LENA, weighted DCT coded, third splitting subband coding with and without weighting at 0.64 b;t/pixel.
(c) the pictures are cleaner with orthogonal transformations; there is more granular noise with subban.d coding, but also the texture rendition is better. 2. the weighting modifies the noise structure; the most general impression for the 'real' subband case is that it reduces the visibility of the impulse responses of high frequency bands. Actually, it is obvious that the quantization introduces noise which prevents the perfect reconstruction relationships from being achieved. Weighting modifies the amount of aliasing produced in the different frequency intervals. 3. the M S D C T is as efficient as the DCT, even if the former is not of the critical sampling type. Due to the overlap of one pel in both directions in the reconstruction stage of the MSDCT, the blocking artefacts are significantly reduced. The following pictures are presented: 1. Fig. 3 is the original ' D I C K ' picture,
2. Fig. 4 is a comparison between weighted and non-weighted DCT at 0.64 bit/pixel, 3. Fig. 5 is a comparison between the weighted and non-weighted M S D C T at 0.64 bit/pixel, 4. Fig. 6 shows subband coding with the first way of splitting of Fig. 2 and non-weighted quantization at 0.64 bit/pixel, 5. Fig. 7 shows subband coding with the first way of splitting of Fig. 2 and weighted quantization at 0.64 bit/pixel, 6. Fig. 8 shows subband coding with the third way of splitting of Fig. 2 and non-weighted quantization at 0.64 bit/pixel, 7. Fig. 9 shows subband coding with the third way of splitting of Fig. 2 and weighted quantization at 0.64 bit/pixel, 8. Fig. 10 shows the original ' L E N A 256 x 256' (upper left corner), the weighted DCT coded picture (0.64 bit/pixel) (lower left), the subband result (first way of splitting) with weighting Vol. 4. No. I. November 1991
L. Vandendorpe / Optimized quantization for image subband coding
78 Table 2
Weighted noise power in dB (0.64 bit/pixel) DICK
Splitting 1 Splitting 3
LENA256
weighted q
non-weighted q
weighted q
non-weighted q
0.9 -2.6
3.5 2.4
5.0 1.9
8. I 7.6
(upper right) and without weighting (lower right), 9. Fig. 11 shows the original 'LENA' (upper left corner), the weighted DCT coded picture (0.64 bit/pixel) (lower left), the subband result (third way of splitting) with weighting (upper right) and without weighting (lower right). As an example, the weighted noise powers (in dB) have been computed for the simulations with QMF filters, for DICK and LENA256 and the first and third splitting cases. The results are summarized in Table 2. Obviously, the weighted noise powers are smaller with weighted quantization. The difference from a non-weighted quantization is of a few dB for the range of entropy measured here. 6. Conclusions This paper has presented a method for optimizing the quantization in a subband coder. It includes the case of orthogonal transformations and nonequalsize subband systems. The optimization has been done by taking into account a sensitivity function of the eye and defining a new distortion measure. The cases of fixed bit allocation and entropy coded quantizer outputs have been considered. The so-called weighted quantization allows for a smoother noise structure and less visible impulse responses. 7. Acknowledgments The author would like to thank Prof Delogne (UCL) and Dr Macq (Philips Research Laboratory of Belgium) for their help with this work. Signal Processing: Image Communication
References [!] N. Ahmed and K.R. Rao, Orthogonal Transformsjot Digital Signal Processing, Springer, Berlin, 1975. [2] J. AUnatt, "Television measurements through psychophysics to subjective picture quality", Radio and Electronic Engineer, Vol. 49, !979. [3] R.E. Crochiere, S.A. Webber and J.L. Flanagan, "Digital coding of speech in subbands", Bell. System Techn. J., Voi. 55, October 1976, pp. 1069-1085. [4] N.S. Jayant and P. Noll, Digital Coding of Waveforms, Signal Processing Series, Prentice-Hall, Englewood Cliffs, NJ, 1984. [5] J.D. Johnston, "A filter family designed for use in quadrature mirror filter banks", Proc. 1980 IEEE lnternat. Conf. Acoust. Speech Signal Process., Hartford, CT, May 1977, pp, 191- 195. [6] H. Kitajima, "A symmetric cosine transform", I E E E Trans. Comput., Vol. C-29, No. 4, April 1980. [7] T. Kronander, "A new approach to recursive mirror filters with a special application in subband coding of images", I E E E Trans. Acoust. Speech Signal Process., Vol. ASSP36, No. 9, September 1988. [8] F. Lukas and Z. Budrikis, "Picture quality prediction based on a visual model", I E E E Trans. Comm., Vol. 7, 1982. [9] B. Macq, Perceptual transforms and universal entropy coding for an integrated approach tu picture coding, Ph.D. Thesis, Universit~ Catholique de Louvain, September 1989. [I0] P.C. Millar, "Recursive quadrature mirror filtersCriteria specification and design method", IEEE Trans. Acoust. Speech Signal Process., Vol. ASSP-33, No. 2, April 1985. [11] M. Miyahara, "Quality assesments for visual services", IEEE Commun. Mag., Voi. 10, 1988. [12] M.G. Perkins and T. Lookabaugh, "A psychophysically justified bit allocation algorithm for subband image coding systems", Proc. 1989 IEEE Internat. Conf. Acousi. Speech Signal Process., Glasgow, Scotland, May 1989, pp. 1815 1818. [13] M.J.T. Smith and T.P. Barnwell, "Exact reconstruction techniques for tree-structured subband coders", IEEE Trans. Acoust. Speech Signal Process., Vol, ASSP-34, No. 34, June 1986. [14] P.P. Vaidyanathan, "Quadrature mirror filter banks, Mband extensions and perfect reconstruction technique", IEEE ASSP Mag., Vol. 4, July 1987, pp. 4-20.
L. Vandendorpe / Optimized quantization for image subband coding [15] L. Vandendorpe, "Hierarchical encoding of HDTV by transform coefficients splitting: A comparison between DCT and MSDCT", Picture Coding Symposium, 26-28 March 1990, MIT, Cambridge, USA, pp. 3.2-1, 3.2-2. [ 16] L. Vandendorpe and P. Deiogne, "Hierarchical encoding of HDTV by transform coefficients splitting", SPIE/ SPSE Symposium on Electronic Imaging, Conference on Image Processing ,41gorithms and Techniques, February 1990, Santa Clara, USA, pp. 343-354. [ i 7] L. Vandendorpe and B. Macq, "Optimum quality and progressive resolution of video signals", Les Annales des TblEcommunications, Vol. 45, Nos. 9-10, 1990, pp. 487-502. [18] M. Vetterli, "A theory of multirate filter banks", IEEE Trans. ,4coust. Speech Signal Process., Vol. ASSP-35, No. 3, March 1987.
79
[19] M. Vetterli and D. Le Gall, "Perfect reconstruction FIR filter banks: Some properties and factorizations", IEEE Trans. ,4coust. Speech Signal Process., Vo!. ASSP-37, No. 7, Jaly 1989. [20] M. Vetterli, J. Kova~evi/: and D.J. LeGall. "Perfect reconstruction filter banks for HDTV representation and coding", Signal Processing: Image Communication, Vol. 2, No. 3, October 1990, pp. 349-363. [211 P.H. Westerink, D.E. Boekee, J. Biemond and J.W. Woods, "Subband coding of images using vector quantization", IEEE Trans. Comm., Vol. 36, No. 6, June 88. [22] J.W. Woods and S. O'Neil, "Subband coding of images", IEEE Trans. Acoust. Speech Signal Process., Vol. ASSP34, No. 5, October 1986.
Vol. 4. No. I. November 1991