- Email: [email protected]
Chapter 7 Coding examples and comparisons
Chapter 7
Coding Examples and Comparisons In Chapter 5 we have focused on theoretical measures for the expected performance of various filter banks and transforms employed in subband coders. In this chapter we present examples of decoded images when using subband coders with different choices of filter banks. The aim is to identify the types of visual degradations associated with particular choices of filter banks. As representative images in the presentation of decoded images we have selected three very different test images, all of which are reproduced in Appendix D: o
.
.
"Lenna': A low complexity image with textured as well as very smooth areas. "Boat": A quite detailed image with sharply defined masts and rope against a smooth background. "Kiel": An image containing both extremes: High detail area in the bottom half of the image and all-smooth skyline in the upper half. Loosely described as a "subband coder's nightmare image", this suggests the problems of representing it as linear combinations of basis functions corresponding to long filter unit pulse responses. In fact, this is an ideal image for demonstrating a transform coder: All smooth sky region (not a cloud in sight) gives no problems with blocking effects. The extremely detailed lower half is very well suited for the short unit 205
206
Ct-IAP'I'ISR 7. C O D I N G E X A M P L E S A N D C O M P A R I S O N S
pulse responses of a transform coder. Blocking effects pose no problems in this region either, because they are efficiently masked.
An investigation of the visual performance of different image coding methods must necessarily be incomplete as well as subjective. Acknowledging this, we try to highlight pros and cons of some of the possible filter bank choices by showing selected image subregions that demonstrate the main classes of image artifacts, namely blurring, ringing and blocking effects. In addition a comparison of the two signal extension methods discussed in Section 2.4 is presented. First we present some examples of blurring distortion for one filter bank only, realizing that this distortion does not significantly depend on the exact choice of filter bank. When showing examples of ringing and blocking distortion we employ several filter banks, namely the DCT, a 16-tap LOT designed as shown in Section 5.4.1, a Johnston type FIR filter bank I17] and one of the IIR filter banks of [51]. All these filter banks are treated in Chapters 3 or 5. In all the examples to follow we use a uniform 8 x 8 band decomposition of the image. When using the Johnston type FIR filter banks and the IIR filter banks, this decomposition is obtained by organizing two-channel building blocks in a tree-structure as explained in Chapter 3. All experiments were carried out using the same quantization procedure: The lowpass band was DPCM coded with 5 bits per pel using an optimal 3rd order predictor, applying the configuration due to Gharavi and Tabatabai [84l, and computed according to the covariance method [79]. A Laplace quantizer was used inside the prediction loop [51]. The quantization algorithm for the upper bands, previously described in Section 6.2.3, is based on thresholding with "dead-zone" twice as large as the quantization step. This is followed by run-length-entropy coding, following the method of [51]. Limiting ourselves to "traditional" subband filter banks in the present chapter, the next chapter will present some approaches to the design of filter banks leading to less disturbing coding artifacts. We repeat that this demonstration is in no way exhaustive, as it would be possible to include a larger selection of filter banks and combine them with a wider selection of coding strategies.
7.1. B L U R R I N G D I S T O R T I O N
7.1
207
Blurring Distortion
In studying the blurring phenomenon, we focus on the choice of threshold value and quantization step, denoted t and A in Figure 4.2. Both A and t are obviously related to the desired total bit rate. If t is too small, many high frequency information samples in higher bands will be quantized to values different from zero. This is disadvantageous for the following reasons: The total bit rate will be high because the high frequency samples need many bits for representation. The run-length code used here will be inefficient in this situation. The quantized high frequency samples may generate unwanted artifacts in visually smooth image areas. When such a region contain weak, high-frequency textures, some subband samples in the higher bands are quantized to nonzero values. Even though the synthesis low pass response reconstruct a smooth underlying surface, some spurious artifacts results from the nonzero higher band samples. On the other hand, if t is too large, too much image detail will be lost, giving the decoded image a very blurred appearance. Choosing a suitable threshold therefore entails a trade-off: Avoid high bit rate and artifacts in smooth regions, while maintaining fine image details. To demonstrate the importance of using a suitable value for the threshold level, we code an image using fixed quantization step and a varying threshold level. The value of A is chosen to give high image q u a l i t y - thus highlighting the effect of increased threshold level. As an example, consider the image "Lenna" coded using the IIR filter bank/_2_2_06 given in Appendix C, with t = A, 2A, 3A and 4A. The bit rates and corresponding SNRs are given in Table 7.1, and in Figure 7.1 selected image subregions are shown. The LP-LP band was coded with DPCM at 5 bits per pel. As evident from Table 7.1, A is small enough to give a signal-to-noise ratio of 36.1 dB when the dead-zone is twice the size of the quantization step (t = A). We have picked two image subregions for demonstrating the above mentioned trade-off. For t = A the details of the hat texture are well preserved. As t is increased~ more texture detail is lost and the shoulder area becomes
208
CHAPTER
7.
CODING EXAMPLES
AND COMPARISONS
Figure 7.1" Varying the threshold: Original (top), t - A to t - 4A (bottom).
7.2. RINGING NOISE
209
[! Threshold t [ A Bit rate SNR(dB)
0.81 36.1
2A [ 3A I 4A 0.36 33.4
0.23 31.9
0.16 30.8
Table 7.1: Bit rates and SNRs for coding of "Lenna" with filter bank f_2_2_06
at several bit rates. slightly smoother. In addition, this filter bank exhibits excessive ringing noise. Note the difference in bit rate between t = A and t = 2A. Clearly, the expense in terms of bit rate is not worthwhile. In general, for low bit rate coding of images, the quantization step will be higher than for this experiment, and as a rule of thumb a threshold level equal to the quantization step is often used.
7.2
Ringing Noise
Ringing noise in subband coders is generated in regions with sharp transitions. This artifact is closely related to the Gibbs phenomenon [78] that arises in the convergence analysis of Fourier series. Since ringing noise is masked in textured regions as well as in regions with many image details, we show decoded image subregions that have transitions in the vicinity of smooth areas. Good examples of the nature of the ringing distortion can be seen by coding the "Boat" image. Figure 7.2 shows the original image excerpt, whereas Figures 7.3 and 7.4 depict the coding results obtained when using t = A in conjunction with several filter banks. The bit rates are in the 0.43 to 0.46 bits per pel range. From these figures it is evident that the ringing distortion is most prominent in traditional subband coders employing filter banks with long unit pulse responses- in this case the ones based on the f16b FIR and the f_2_2_06 IIR filters 1. The DCT and the 16-tap LOT exhibit very small amounts of ringing distortion. 1See Appendix C for filter coefficients.
210
CHAPTER 7. CODING E X A M P L E S AND COMPARISONS
Figure 7.2: Original "Boat" image (excerpt).
7.3
Blocking Distortion
In transform coders blocks of the image to be coded are processed and represented independently. In traditional subband coders the image is processed as one single signal entity. The LOT may be considered as an intermediate case in this context. The independence in processing and representation of adjacent blocks in a transform coding scheme manifests itself as blocking distortion. As this independence is reduced in LOTs and even more so in traditional subband coders (i.e. subband coders employing channel filters with long unit pulse responses) the blocking distortion is reduced and, for the case of traditional subband coders, it has completely vanished. These statements are illustrated in Figure 7.6 and Figure 7.7 (as well as in Figure 7.3, top). For comparison the excerpt of the original of "Lenna" used in these coding experiments is shown in Figure 7.5.
7.4
Comparing Circular and Mirror Extension
In Figure 7.8 we show two coding examples taken from the image "Lenna", where we compare the circular extension method with the mirror extension
7.4. COMPARING CIRCULAR AND M I R R O R E X T E N S I O N
211
Figure 7.3" Ringing noise for various filter banks" DCT (top), and L O T (bottom).
212
CHAPTER 7. CODING E X A M P L E S AND COMPARISONS
Figure 7.4: Ringing noise for various filter banks: Johnston FIR filter bank,
f16b (top), and IIR filter f_2_2_06 (bottom).
7.4. COMPARING CIRCULAR AND MIRROR EXTENSION
213
Figure 7.5: Original "Lenna" image (excerpt). method. The complete image is filtered with the f32d filters due to Johnston [17]. The total bit rate is 0.25 bit per pel. The improved result of the mirror extension method is clearly evident in the left, right, and top edge regions. These are the regions affected by the generation of artificial steps when using the circular extension method. Similar problems affects the bottom edge region but this is omitted from the excerpts shown here. Although there is a clear quality difference in this case, we remark that the difference is not very significant for higher bit rates.
214
CHAPTER 7. CODING E X A M P L E S AND COMPARISONS
Figure 7.6" Blocking distortion for various filter banks: DCT (top), and L O T (bottom). Bit rate" Approximately 0.4 bits per pel.
7.4. COMPARING CIRCULAR AND MIRROR EXTENSION
215
Figure 7.7" Blocking distortion (or lack thereof) for various filter banks:
Johnston FIR filter bank, f16b (top), and IIR filter f_2_2_06 (bottom). Bit rate" Approximately 0.4 bits per pel
216
CHAPTER 7. CODING E X A M P L E S AND COMPARISONS
Figure 7.8" Comparison of signal extension techniques'Circular extension (top), and mirror extension (bottom).
Coding Examples and Comparisons In Chapter 5 we have focused on theoretical measures for the expected performance of various filter banks and transforms employed in subband coders. In this chapter we present examples of decoded images when using subband coders with different choices of filter banks. The aim is to identify the types of visual degradations associated with particular choices of filter banks. As representative images in the presentation of decoded images we have selected three very different test images, all of which are reproduced in Appendix D: o
.
.
"Lenna': A low complexity image with textured as well as very smooth areas. "Boat": A quite detailed image with sharply defined masts and rope against a smooth background. "Kiel": An image containing both extremes: High detail area in the bottom half of the image and all-smooth skyline in the upper half. Loosely described as a "subband coder's nightmare image", this suggests the problems of representing it as linear combinations of basis functions corresponding to long filter unit pulse responses. In fact, this is an ideal image for demonstrating a transform coder: All smooth sky region (not a cloud in sight) gives no problems with blocking effects. The extremely detailed lower half is very well suited for the short unit 205
206
Ct-IAP'I'ISR 7. C O D I N G E X A M P L E S A N D C O M P A R I S O N S
pulse responses of a transform coder. Blocking effects pose no problems in this region either, because they are efficiently masked.
An investigation of the visual performance of different image coding methods must necessarily be incomplete as well as subjective. Acknowledging this, we try to highlight pros and cons of some of the possible filter bank choices by showing selected image subregions that demonstrate the main classes of image artifacts, namely blurring, ringing and blocking effects. In addition a comparison of the two signal extension methods discussed in Section 2.4 is presented. First we present some examples of blurring distortion for one filter bank only, realizing that this distortion does not significantly depend on the exact choice of filter bank. When showing examples of ringing and blocking distortion we employ several filter banks, namely the DCT, a 16-tap LOT designed as shown in Section 5.4.1, a Johnston type FIR filter bank I17] and one of the IIR filter banks of [51]. All these filter banks are treated in Chapters 3 or 5. In all the examples to follow we use a uniform 8 x 8 band decomposition of the image. When using the Johnston type FIR filter banks and the IIR filter banks, this decomposition is obtained by organizing two-channel building blocks in a tree-structure as explained in Chapter 3. All experiments were carried out using the same quantization procedure: The lowpass band was DPCM coded with 5 bits per pel using an optimal 3rd order predictor, applying the configuration due to Gharavi and Tabatabai [84l, and computed according to the covariance method [79]. A Laplace quantizer was used inside the prediction loop [51]. The quantization algorithm for the upper bands, previously described in Section 6.2.3, is based on thresholding with "dead-zone" twice as large as the quantization step. This is followed by run-length-entropy coding, following the method of [51]. Limiting ourselves to "traditional" subband filter banks in the present chapter, the next chapter will present some approaches to the design of filter banks leading to less disturbing coding artifacts. We repeat that this demonstration is in no way exhaustive, as it would be possible to include a larger selection of filter banks and combine them with a wider selection of coding strategies.
7.1. B L U R R I N G D I S T O R T I O N
7.1
207
Blurring Distortion
In studying the blurring phenomenon, we focus on the choice of threshold value and quantization step, denoted t and A in Figure 4.2. Both A and t are obviously related to the desired total bit rate. If t is too small, many high frequency information samples in higher bands will be quantized to values different from zero. This is disadvantageous for the following reasons: The total bit rate will be high because the high frequency samples need many bits for representation. The run-length code used here will be inefficient in this situation. The quantized high frequency samples may generate unwanted artifacts in visually smooth image areas. When such a region contain weak, high-frequency textures, some subband samples in the higher bands are quantized to nonzero values. Even though the synthesis low pass response reconstruct a smooth underlying surface, some spurious artifacts results from the nonzero higher band samples. On the other hand, if t is too large, too much image detail will be lost, giving the decoded image a very blurred appearance. Choosing a suitable threshold therefore entails a trade-off: Avoid high bit rate and artifacts in smooth regions, while maintaining fine image details. To demonstrate the importance of using a suitable value for the threshold level, we code an image using fixed quantization step and a varying threshold level. The value of A is chosen to give high image q u a l i t y - thus highlighting the effect of increased threshold level. As an example, consider the image "Lenna" coded using the IIR filter bank/_2_2_06 given in Appendix C, with t = A, 2A, 3A and 4A. The bit rates and corresponding SNRs are given in Table 7.1, and in Figure 7.1 selected image subregions are shown. The LP-LP band was coded with DPCM at 5 bits per pel. As evident from Table 7.1, A is small enough to give a signal-to-noise ratio of 36.1 dB when the dead-zone is twice the size of the quantization step (t = A). We have picked two image subregions for demonstrating the above mentioned trade-off. For t = A the details of the hat texture are well preserved. As t is increased~ more texture detail is lost and the shoulder area becomes
208
CHAPTER
7.
CODING EXAMPLES
AND COMPARISONS
Figure 7.1" Varying the threshold: Original (top), t - A to t - 4A (bottom).
7.2. RINGING NOISE
209
[! Threshold t [ A Bit rate SNR(dB)
0.81 36.1
2A [ 3A I 4A 0.36 33.4
0.23 31.9
0.16 30.8
Table 7.1: Bit rates and SNRs for coding of "Lenna" with filter bank f_2_2_06
at several bit rates. slightly smoother. In addition, this filter bank exhibits excessive ringing noise. Note the difference in bit rate between t = A and t = 2A. Clearly, the expense in terms of bit rate is not worthwhile. In general, for low bit rate coding of images, the quantization step will be higher than for this experiment, and as a rule of thumb a threshold level equal to the quantization step is often used.
7.2
Ringing Noise
Ringing noise in subband coders is generated in regions with sharp transitions. This artifact is closely related to the Gibbs phenomenon [78] that arises in the convergence analysis of Fourier series. Since ringing noise is masked in textured regions as well as in regions with many image details, we show decoded image subregions that have transitions in the vicinity of smooth areas. Good examples of the nature of the ringing distortion can be seen by coding the "Boat" image. Figure 7.2 shows the original image excerpt, whereas Figures 7.3 and 7.4 depict the coding results obtained when using t = A in conjunction with several filter banks. The bit rates are in the 0.43 to 0.46 bits per pel range. From these figures it is evident that the ringing distortion is most prominent in traditional subband coders employing filter banks with long unit pulse responses- in this case the ones based on the f16b FIR and the f_2_2_06 IIR filters 1. The DCT and the 16-tap LOT exhibit very small amounts of ringing distortion. 1See Appendix C for filter coefficients.
210
CHAPTER 7. CODING E X A M P L E S AND COMPARISONS
Figure 7.2: Original "Boat" image (excerpt).
7.3
Blocking Distortion
In transform coders blocks of the image to be coded are processed and represented independently. In traditional subband coders the image is processed as one single signal entity. The LOT may be considered as an intermediate case in this context. The independence in processing and representation of adjacent blocks in a transform coding scheme manifests itself as blocking distortion. As this independence is reduced in LOTs and even more so in traditional subband coders (i.e. subband coders employing channel filters with long unit pulse responses) the blocking distortion is reduced and, for the case of traditional subband coders, it has completely vanished. These statements are illustrated in Figure 7.6 and Figure 7.7 (as well as in Figure 7.3, top). For comparison the excerpt of the original of "Lenna" used in these coding experiments is shown in Figure 7.5.
7.4
Comparing Circular and Mirror Extension
In Figure 7.8 we show two coding examples taken from the image "Lenna", where we compare the circular extension method with the mirror extension
7.4. COMPARING CIRCULAR AND M I R R O R E X T E N S I O N
211
Figure 7.3" Ringing noise for various filter banks" DCT (top), and L O T (bottom).
212
CHAPTER 7. CODING E X A M P L E S AND COMPARISONS
Figure 7.4: Ringing noise for various filter banks: Johnston FIR filter bank,
f16b (top), and IIR filter f_2_2_06 (bottom).
7.4. COMPARING CIRCULAR AND MIRROR EXTENSION
213
Figure 7.5: Original "Lenna" image (excerpt). method. The complete image is filtered with the f32d filters due to Johnston [17]. The total bit rate is 0.25 bit per pel. The improved result of the mirror extension method is clearly evident in the left, right, and top edge regions. These are the regions affected by the generation of artificial steps when using the circular extension method. Similar problems affects the bottom edge region but this is omitted from the excerpts shown here. Although there is a clear quality difference in this case, we remark that the difference is not very significant for higher bit rates.
214
CHAPTER 7. CODING E X A M P L E S AND COMPARISONS
Figure 7.6" Blocking distortion for various filter banks: DCT (top), and L O T (bottom). Bit rate" Approximately 0.4 bits per pel.
7.4. COMPARING CIRCULAR AND MIRROR EXTENSION
215
Figure 7.7" Blocking distortion (or lack thereof) for various filter banks:
Johnston FIR filter bank, f16b (top), and IIR filter f_2_2_06 (bottom). Bit rate" Approximately 0.4 bits per pel
216
CHAPTER 7. CODING E X A M P L E S AND COMPARISONS
Figure 7.8" Comparison of signal extension techniques'Circular extension (top), and mirror extension (bottom).