- Email: [email protected]
Channel error propagation in adaptive tv coders
Signal Processing l0 (1986) 171-183 North-Holland
171
C H A N N E L ERROR P R O P A G A T I O N IN ADAPTIVE TV CODERS* C.J. H W A N G and K.R. RAO Department of Electrical Engineering, The University of Texas at Arlington, Arlington, TX 76019, U.S.A. Received 31 October 1984 Revised 14 February 1985 and 14 August 1985
Abstract. DPCM coders with adaptive predictors are used and compared with nonadaptive DPCM coders for processing composite color television signals. The transmission error propagation for different predictors both adaptive and fixed is investigated. Propagation of transmission noise is dependent on the type of prediction and on the location of noise, i.e., whether in a uniform region or in an active region. By introducing leak in predictor output and/or predictor function, it is shown that error propagation can be significantly reduced. The leaky predictors not only attenuate and/or terminate the channel error propagation but also improve the predictor performance based on quantitative evaluation such as peak value and mean-square-error. In order to protect the high picture quality from error propagation, an error-correcting code is needed. Zusammenfassung. Der Einsatz von DPCM-Codierverfahren mit adaptivem Prfidiktor bei der Verarbeitung zusammengesetzter Videosignale wird untersucht; diese Verfahren werden mit adaptiven DPCM-Codierverfahren verglichen. Insbesondere wird fiir verschiedene--adaptive wie nichtadaptive--Pr~idiktoren die Fehlerfortpflanzung bei 0bertragungsstSrungen untersucht. Die Fortpflanzung rauschartiger UbertragungsstSrungen h~ingt v o n d e r Art des Pr~idiktors ab, aul~erdem besteht eine Abh~ingigkeit yon der Stelle, an der das Rauschen auftritt, d.h., davon, ob das Bild an dieser Stelle wenig oder stark strukturiert ist. Wie gezeigt wird, kann die Fehlerfortpflanzung dutch die Einfiigung einer k/instlichen D~impfung in das Ausgangssignal des Pr~idiktors u n d / o d e r die zur Ubertragung anstehende Pr~idiktionsfunktion erheblich reduziert werden. Pr~idiktoren dieser Art d~impfen bzw. beenden nicht nur die Fortpflanzung von l'_'lbertragungsfehlern, sic ver~esseru dariiber hinaus auch das Verhalten des Gesamtsystems, wie sich durch quantitative Messungen mit Hilfe des Kriteriums der maximalen Abweichung oder des mittleren quadratischen Fehlers nachweisen l~il3t. Um die hochwertige Bildqualit~it vor 0bertragungsfehlern zu sch/itzen, ist allerdings ein fehlerkorrigierender Code bei der Ubertragung notwendig. R6sum6. Les codeurs MICD avec des pr6dicteurs adaptatifs sont utilisgs et compar6s avec les codeurs non adaptatifs MICD pour traiter les signaux composites de t616vision couleur. La propagation de l'erreur de transmission pour les diff6rents prgdicteurs, adaptatifs et non adaptatifs, sont investigugs. La propagation du bruit de transmission est d6pendant du type de pr6diction et de la position du bruit, c'est-b.-dire si le bruit se trouve dans une rggion uniforme ou une r6gion active. II esl: montr6 que la propagation des erreurs peuvent ~tre r6duite consid6rablement en introduisant une distribution ~t la sortie du pr6dicteur et/ou fi la fonction de prgdiction. Les pr6dicteurs distributifs non seulement attenuent et/ou arr~tent la propagation des erreurs de voie mais am61iorent aussi les performances des pr6dicteurs bas6es sur des gvaluations quantitatives telles que valeur de pic et erreur quadratique moyenne. Pour prot6ger la haute qualit6 des images de la propagation d'erreur un code correcteur d'erreur est n6cessaire. Keywords. Predictive coding, TV signals, channel error propagation.
1. Introduction Predictive techniques for digital processing of signals can be broadly classified as (i) delta modu* This paper is based on the research by Mr. C. J. Hwang for his Master Thesis as a partial requirement for the M.S. Degree from the University of Texas at Arlington, TX.
lation (DM) [1], and (ii) differential pulse code modulation (DPCM) [2,3]. The latter involves prediction (adaptive or nonadaptive) of the present sample based on previous sample(s). The error signal representing the difference between the actual sample and its predicted value is quantized, coded and transmitted over a digital link for recon-
0165-1684/86/$3.50 (~ 1986, Elsevier Science Publishers B.V. (North-Holland)
C.J, Hwang, ICIL Rao / Channel error propagation in adaptive TV coders
172
struction at the receiver end (Fig. 1). In a good prediction scheme, the range and entropy of the error signal are small compared to those of the
, Prwdtctor
'
error sensitivity of adaptive DPCM coders for composite color TV signals is now analyzed. It is hoped that this investigation can contribute to the design of channel encoder for incorporation into the codec built by Cox [11].
+
2. Data base (a) ~'rom Channel
(b) Fig. 1. Block diagram of a DPCM system: (a) transmitter, (b) receiver.
original signal. By incorporating appropriate quantization and coding schemes, the error signal can be transmitted at a reduced bit rate thereby achieving significant band-width compression. DPCM and other image coding techniques are described elsewhere [15, 16, 31]. The DPCM concept has been utilized by several researchers for transmitting broadcast quality TV signals [4-13]. For color image sequences, various DPCM techniques have been proposed, simulated and some have been implemented in hardware [11, 14-17]. In particular, Cox [11] has designed and built an encoder which implements the DPCM simulator (predictor, quantizer, buffer, etc.,) for digital processing of composite color TV signal sampled at 10.7 MHz. This encoder, however, has no error detection and correction capability. In previous papers [11, 17, 32], some adaptive methods based on contour prediction were reported. Their effectiveness in redundancy reduction was analyzed in terms of both quantitative and subjective criteria. Initial results on channel noise propagation in adaptive predictors have been reported [33]. By introducing leaks in the predictor function and/or predictor output [18], it has been shown that the transmission error propagation can be considerably reduced. Their ability to combat the channel Signal Processing
The data base is obtained from the NTSC color television signal, which has a bandwidth of 4.2 MHz. Such a signal has 30 frames per see, where each frame consists of 525 horizontal lines arranged in two interlaced fields. This signal was sampled at 10.74 MHz (three times the color subcarder frequency, f~c = 3.58 MHz) and was digitized to 8 bit PCM. Only a 512 x 512 array of every single frame, corresponding to most of the visible portion of the video signal was used for simulation. Four static pictures were used for simulation (see Fig. 2): PictureA: Lady head and shoulders (Fig. 2(a)). Picture B: Newscaster with a news slide in the background (Fig. 2(b)). Picture C: Fred and Wilma Flintstone (Fig.
2(c)). PictureD: An interviewer in a crowded background (Fig. 2(d)).
3. Predictor
Contour predictors originally developed by Zschunke [12] for monochrome images have been extended to color images [11, 17,32,33]. These adaptive predictors can track co~atours, slopes, and transitions and therefore can better predict a pel in an active region compared to the fixed predictors. The contour predictors 1 and 2 [17] (described below) together with the 'nearest neighbor' predictor [13] and fixed predictors are simulated. Pel locations are described in Fig. 3. Pels with the same notation have similar subcarrier phase.
173
C.J. Hwang, K.R. Rao / Channel errorpropagation in adaptive TV coders
Fig. 2. (a) Original picture A. (b) Original picture B. (c) Original picture C. (d) Original picture D.
Vth ( Vth
is the vertical threshold), and
Is2- $71<1s - $1ol. •S 6
S5
¢ : o # S 4 S 3 S 2 Sj
:
:
o
Fig. 3. Notation for contour prediction.
Predictor 1
(I) Using the last reconstructed sample $2, ( S 2 - S s ) is obtained. If IS2-$51 is less than some horizontal threshold Hth, a uniform region or horizontal contour is assumed and S~ is predicted as $4, i.e., $1 = $4. (II) However, if Is=-ssl> Hth, a contour is assumed. (a) $ 1 = S 6 if IS7-S,oI>SL (SL is the minimum slope requirement), sign ( $ 7 - S~o) = s i g n ( S 2 - $5), IS2- $71 <
(b) S~ -- $9 if IS~o- S~3[ > SL, sign(S~o$13) = sign(S2 - $5), IS2 - S~ol < Vth, and IS2- $1o1 < IS2- S7I. (III) If both tests (II)(a) and (II)(b) fail, it is difficult to determine a contour direction. Hence, $1 = $4. The thi'esholds chosen are Hth = 8, SL = 8, and Vth = 64. Predictor 2
Predictor 2 is identical to predictor 1 with the following exception. If both tests (II)(a) and (II)(b) fail, and if IS2 - $51/> Wth, then $1 = $2. Wth is the wideband or large threshold transition (Wth = 32). In a prototype e n c o d e r / s i m u l a t o r [11] that has been built, both predictors 1 and 2 have been implemented in real time. Vo[. 10, No, 2, March 1986
174
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders
Predictor 3
Predictor 3 is the composite equivalent of the 'nearest neighbor' predictor [13] and is defined as follows:
The contour predictors (predictors 1 and 2) have been utilized in composite color TV image coding [11, 17, 34].
rS6 if IS2-&l< &-S:o[ 4. Quantizer
and I S 2 - $71 < I S 2 - $51 ,
~ = S~ if ls2- S:ol < ls2- s71 $4
and [Sz- $1o1< [Sz - $51, otherwise.
Predictor 4
Predictor 4 is the 2-D-II composite predictor proposed by Dinstein [10], i.e., ,~,=S9+
½(S~- Slo). Predictor 5
Predictor 5 is the composite planar predictor [19]. Sl = $ 2 + $ 9 - S 1 o . Predictor 6
Predictor 6 is the simple S -3 predictor. $1 = $4.
Based on a statistical study of prediction errors, a max symmetric quantizer [20] for minimizing the mse was designed by Cox [17]. This quantizer was later adjusted subjectively based on the visibility thresholds using the DPCM simulator built by Cox [ 11]. Reflecting a compromise between the quantizer complexity and minimizing the number of bits per pel (BPP), a dual word length quantizer (Table 1) with corresponding code representation (Table 2) was adopted [11, 17]. To match this code to a constant bit rate, provision has been made for a fixed word length mode whenever the number of long words along a horizontal scan line exceed a preset limit.
Table 1 29/15 level symmetricquantizer based on visibilitythresholds [17] (only positive levels are shown) Dual-Length Input
Output
Fixed-Length Code
Input
Output
0
0
0 - 1
0
1
1
2 - 5
3
2
2
6 - II
8
3
3
12 - 19
15
4 bit
4-
6
5
20 - 28
24
7-
9
8
29 - 38
33
14
12
39 - ~o
51 - 255
io
-
20
17
21 - 28
24
29 - 38
33
39 - 50
44
51 - 63
57
64 - 77
70
15 -
78
-
93 -
Signal Processing
Mode
92
255
85 IOO
8 bit
57
Mode Code
4 bit
175
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders
Table 2 4/8 bit dual-word-length code construction [17]
but also corrected. However, there is a limit to the burst error correction of cyclic codes [30].
1111 1110 1101
5.1. Effect o f channel errors 4 bit words .......... a s s l g n e d to 15 most-probable levels
O6Ol oooo
.......... long word prefix
00001111
.......... frame sync code
ooooliiO 00001101 t oooolioo
8 bit long words ....... assigned to 14 least-probable levels
ooo6oool
00000000
.......... not a s s l K n e d
5. Channel error propagation
Error protection, for the D P C M coded video signals can be provided in many ways. One method is by predictor leak, that is, using a fraction [15] of the previous element value for prediction. By using a recent idea of prediction function leak [ 18], the transmission error propagation can be significantly reduced. This is the so-called 'leak factor' or 'leaky integrator' concept. This technique has the advantage of maintaining the transmission bit rate, but the predictor may become less efficient. A second method adds redundancy in the form of periodic PCM updates. Another technique is to add error-detection codes which enable single or multiple errors to be detected at the receiver. When an error appears, the erroneous sample is replaced with an estimate which is close to the actual value. This is called error c o n c e a l m e n t [22]. When high error rates are to be expected, this method is preferred. This is because more errors can be detected than corrected, given the amount of added redundancy. In the forward error correction scheme [23-27], error correcting codes are encoded in the transmitter and are decoded in the receiver such that single or multiple errors are not only detected,
The channel perturbations were introduced between the transmitter and the receiver for investigating the sensitivity of the predictor to transmission errors. An error propagation is effectively terminated when the receiver output is refreshed by the PCM even though this technique leads to an increase in the overall bit rate. This additional bit rate can, however, be avoided by a technique called hybrid D-PCM proposed by Van Buul [28]. In this method, by a clever combination of the PCM and DPCM, the advantages of both, i.e., the fast error recovery of the former and the bit-rate reduction of the latter, are retained. Another technique for counteracting the error propagation in a DPCM coder is the difference detection and correction ( D D C ) scheme [29]. In this method the difference criterion is adaptively set and an error detection is followed by the correction of the DPCM output sequence. The error propagation not only depends on the type of predictor, but also on the location of error. The areas of the picture chosen for error examination were categorized as uniform, moderately active, or highly active defined as follows [17]: uniform moderately active: highly active :
0.0 <~cc/pel < 0.05, 0.05 <~cc/pel < 0.4, 0 . 4 ~< cc/pel --z-1.0,
where cc/pel was computed by dividing the number of contour checks (cc) (predictors 1 and 2 describe the contours) performed within the region by the number of pels within the region. Contour check refers to # of contours for predictors 1 and 2. Exceeding the horizontal threshold Hth or the vertical threshold V~h counts as a contour check. The error patterns for predictors 4-6 are shown in Fig. 4. The Z 3 predictor propagates the error in horizontal direction until it disappears at the end of line. The planar predictor distributes error propagation in both the vertical and horizontal V o l 10, No. 2, March 1986
176 0 0 0 0 0 0 0 0
C.J. Hwang, ICR. Rao / Channel error propagation in adaptive TV coders 0 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0
o o o oo 00
O 18 18 18 18 0 0 0 0 0
0 18 18 18 18 18 0 0 0 0
0 18 18 18 18 18 18 0 0 0
0 18 18 18 18 18 18 0 0 0
0 18 18 18 18 18 18 18 0 0
0 18 18 18 18 18 18 18 o
0 0 1 2 4 18 0 0 0 0 0
0 0 0 1 2 9 18 0 0 0 0
0 0 0 1 1 4 9 0 0 0 0
0 0 0 0 I 2 4 18 0 0 0
0 0 o 00 1 2 9 18 0 o
0
(a) 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0 0 0
0 18 18 0 0 0 0 0 0 0
0 18 18 0 0 0 0 0 0 0
0 18 18 18 0 0 0 0 0 0
0 18 18 18 18 0 0 0 0 0
18
(b) 0 0 0 0 0 0 0 0 0 0 0
0 0 0 C 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0 0 0 0
0 9 18 0 0 0 0 0 0 0 0
0 4 9 0 0 0 0 0 0 0 0
0 2 4 18 0 0 0 0 0 0 0
0 1 2 9 18 0 0 0 0 0 0
0 1 i t, 9 0 0 0 0 0 0
by the Hth, SL, and Vth parameters, to find a general error propagation pattern is very tedious. As the amount of luminance activity increases, the opportunity for the error propagation increases. As Hth is increased, errors will propagate further in the horizontal direction, but less in the vertical direction. In summary, the propagation of transmission errors is influenced by (a) type of predictor, i.e., fixed, leaky fixed, or adaptive predictor, (b) one-, two-, or three-dimensional, i.e., intraline, intrafield, or interframe prediction, (c) the location of the error, i.e., whether in uniform region or highly active region, and (d) in the case of the contour predictor, it depends mainly on the activity of the local region, the duration of the contours, and the magnitude of the error. The characteristic of the adaptive predictor is very similar to that of a Z -3 predictor in the uniform region and is worse than that of a fixed predictor in the highly active region. Some examples of error propagation for adaptive predictors are shown in [33].
(c) Fig. 4. Error patterns whose decoded value is 18. (a) Predictor 6, (z-3). (b) Predictor 5, planar. (c) Predictor 4, 2D-II.
directions. The 2-D-II predictor is monotonically decreasing the error propagation in the horizontal direction. However, the error extends diagonally to each successive line. In the case of adaptive predictors, a transmission error can happen both due to the wrong value for the prediction and due to selecting a wrong predictor. The error propagation is similar to the Z -3 predictor, as long as the signal slopes of the previous line remain below SL (over a 3-pel distance). Some conditions will terminate the error propagation. If the error has the same sign for the slope which lasts for the next three pels, the horizontal propagation error will end. If the error has the opposite sign, horizontal propagation error must last for three pels. In either case, if the error is small enough so that it does not affect the selection of the contour direction, then no propagation occurs. Because the contour direction is affected Signal Processing
5.2. Predictor output attenuation [18] The predictor output leak and predictor function leak can provide significant reduction in channel error propagation without decreasing the transmission rate [18]. The basis for the technique is a generalization of the notion of predictor output attenuation which includes attenuation of the adaptive prediction function.
5.3. Predictor output leak The effect of transmission errors can be reduced by attenuating the predictor output F ( . ) by a constant a, 0 < a < 1, such that g = c~F(') + ( 1 - a)r/, where S is the new predicted value of the predictor output and r/is a constant in the span of possible picture values. The prediction error has the form
S - S= S - ( ~ F ( " )+ (1-u)~I) = a ( S - F ( ' ) ) + (1 - ~ ) ( S - r/),
177
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders 0 0
0 0 -1
-108 0-108 0-114 0
-2
-35 0 -35 17 -14 69 -35 17
-21
-35 12 -26 3 -14 -21
5
0 0 0 0 0 0 3 O-lO8 0 0-108 0-108 -1 0-108 -1 0 -91 -7 O-lO8 -7 0-111 -1 0-111 -1 -5 0 -2 -5 0 -2 0 - 3 5 -21 0 - 3 5 -21 0 -35 21 0 - 3 5 21 - 1 2 25 - 2 6 -21 17 -23 - 2 1 17 - 2 6 -21 17 -23 69 - 1 4 -21 17 -14 -21 -21 17 - 1 4 -21 17 -14 5 -21 11 5 -21 17 5 -28 17 5 -28 17
0
0
3
O-lO8
0
0 0 0 19 21 2 0-108 -1 -1 21 -91 -1 0-111
0 19
0 0-108 -1 -1 -7 21 -91 -1 -1 0-111 o -2 -5 -5 0 -2 -5 0 0 - 3 5 -21 0 - 3 5 -21 o -35 21 21 0 -35 21 17 - 2 3 -21 -21 17 -23 -21 -21 17 -23 -21 17 -23 -21 17 -14 -4 51 - 2 3 -4 51 -21 17 -14 -4 51 - 2 3 -4 5 -21 51 17 5 -21 51 5 -21 -28 17 5 -21 17
Fig. 5. Difference between receiver output with and without transmission error, picture C line 10-23 error value -100, a = 1,/3 = 1, predictor 1.
where /3 is a constant, 0
where S is the input data. The error consists of both D P C M a n d . P C M information. As a approaches zero, the system changes from D P C M to PCM. The reduction in error p r o p a g a t i o n for picture C from line 10 to line 23 is shown in Fig. 6 for a = ~ 7 (compare with Fig. 5). Although an i m p r o v e m e n t is obtained with this approach, the error is still intolerable. Further reduction in a leads to visible degradation in the output caused by the quantizing noise. Hence, a is chosen to be 7 15 31 a constant ~>~ such as Tg, ~5.4. Prediction f u n c t i o n leak [18]
Since the effect of channel errors in an adaptive codec is to make the value o f the prediction function F ( . ) uncertain, the receiver loop must in fact estimate not only the input data S but also the function F ( . ). The prediction is set by S=/3F(.) +(1-/3)F(. 0
-105 7 -1 -43 10 0 0 0 0 0 0 o
Using this method, Fig. 7 (in highly active region)
),
0 0 0 0 0 -92 0 -71 -8 -4 - 5 5 -1
6 -18 -37 5 o 9 8 0 o 0 0 o 0
,S=oq3F(')+a(1-/3)F(')+(l--o~)'O.
14 18 0 0 0 o o
2 0 0 0 0 o o
0 0 o 0 0 -81 o -63 -7
-4 -48 4 5 -31 -31 4 o 8 7 3 o 0 o o o 0 o o 0 0 o
o -25
0 0
0 0 0 -70
o -55
-6
-4 -42 3 5 -27 -28 3 o 7 6 3 o 0 0 o 0 0 0 - 9 16 - 1 5
-36
0 o 0 0 o -62 0 -/s8 - 6 -4 -37 3 4 -23 -24
3
o
6
5 2 0 0 0-100
0 19
0 - 3 6 -32
0
0 -28 -36 - 1 9 -31 - 3 1 -7 -6 0 -34 -22 - 1 5 -2 - 6 - 1 3 - 4 0
-8 8
-3
0 0 0 o o -42 -3 -32 3 -20 0 3 4 2 0 -9 0 -9 0 -9 -12 0 -13 -6 -12 35
Fig. 6. Reduction of transmission error propagation using predictor output leak (a = g,7/3 = 1), picture C line 10-23, error value -100, predictor 1 (compare with Fig. 5). Vol. 10, No. 2, March 1986
o
II
II
@
z2
II
~'~
O.z
~'~
00
o
N
,O
.m
I
I
I
I
I
I
I
I
I I
I I I I I ~-~ ~ , ~..~ h.) I.-~ 0
I
I
I
!
I
I
I
I
I
I
I
I I ~ 0
0 I
I ~
I
I
0
o
O
O O
!
!
r
I I I i%) I-.~ 4~-i.~ I ~ t.~
I
! I i'~o O x O
I I O ~.-~..~o
I ! O i-~ ~-.~ O
O k ~
I
I I t~ 0',4=" Ox~'-~%~00
f
o
i-.~ o
o
! ! r.-~ o % . m t,o o to o
! I ! O I..~ O ~-~ 4 = - k ~ ~-~ ~-~ r o ~.-~
O
I I ~-~ ~
o
O O
I
!
o
I I I I O ~-~, ~.-~ O K,~ t~O O ~-~ . ~ - r o
I
~-~ 0
I ~-~ ~
0
O
OO
. ~ I'o Ix) ~-'~ ~-~ t'o
O t,O I--s O k ~ ' , . ~ O O k.o r o
I
I I I I I O i-~ O I-~ I.-~ t~O O I~O I-~ I ~
0
I I I I ' 0 1 -L I"0 ~'~ ,'4='I'0 I '0 0 ~ -'L
I
I
I ~0 ~-~ 0
0
I
0
0
0
I
0
I
~) O
I ~-~.O
' oo
0
t r~. O
I ~-,,.O
0
O
o
I
I
0
0
I
I l
I
I
I ro h-~.O
I
I ~ i'~O 0
I ~-~ 0 I.-~ 0
Ik) O
o
O
o
i
i
O
I i-~
i
!
! l.~o.4~-o
! l,o i-~ O
C,xk.o l.-*O0 h-~
I I~ ~-" h..~.I-~ ~,0 po i-~ -.,.]o
I I I h.~.i-~ -..40 k.m ~-~.O
I ~-~ 4 ~ - 0
I-.,.O
I I f r~ t-* i-.,,k.0 h-,,
! i-~ t i-...r,0 t..-,.~.~ .4~-o
I h-,.o
I ~,. 0
I I I I I'~ I--,~ I-~ ..~- I.~
I
t,O O
I ! I-.,.l-~ b O O
t.o ~-b i-~.O
O
O
0
I h..~O
l..OI,Ok.O ~-~ O i
O
I
0
I I O k , . ) . ~ , , ~ I-~
~-'~ 0
0
I
0
I I I'o ~-~ 0
I ~. 0
0
I I--.,. ~-,~ I~0 0
0
0
~) ~
I
I
I,~. 0 I
O
' !
0
I I I I ~ 1,0 ~-,. ~,.
0
I
0
I
I
I
I
/ ~
I
!
'
~ I
~-.~k.~ O
O K,,~ O
k.~. t,O I I-~.l',O-,,3 Ik) O
O
o
I l I r ~ i,~ i-~
I
! I-,.l,O h) 1.-~.k~ O
I
! I..~I...~O
O
t
0
0
I ~,0 4=-- 0
I I I ~-~ I"~ r ~ I~o o
J.,..,..,..O,..,..,~-l,O O
O O I
O
I
I k.~ I~,0 0
I
.~ I I ,~a i-,. l-U 1.0 i-,~ 0
I
i-~ i-,,~ 0
I ~,. ~.~. 0
f'O IX.) ~,~ 0
0
I
I I I r,o 4::- O~ I~,
0
0~0
l t l ~,~ ~a, i.~ i...~ 0
I ~,, I-~ 0
I %o ~ . 0
l..a. 0
0
0
I ~. 0
0
I
I i..~. i.,~. 1.0 I.~
0 ~
I I k.r,,. I ~ i-~ I-~. 0
I ~,,. ~,~ 0
I i~0 i.~ I.~ I~0 0
I',0 0
0
II
II
p~
e~
_~.,.
(m
0 -t
(m
..~
o
5"
o
o
o
o
o
o
o
o
o
o
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O O
O
0
0
O
0
0
o
o
o
o
o
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
0
0
o
o
o
o
o
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
0
0
o
o
o
o
o
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
0
0
o
o
o
o
o
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
0
0
O
O
O
0
0
O
O
O
0
0
O
O
O
0
0
O
O
O
O
O
O
O
o
o
O
O
O
O
O
O
O
I I-".O O
O k . O i-~k.~ O,, O
l.-'.k.~O % CD 00-.,,] O I b,) ~-~.bo .O- Ox-.,.],,O k.m O
Ok~
O
~-" h ~ k ~
.~-Oxk.,~ O x 4~- O
I i-" k.0 k.0 ,,.~ O,.--3 -',3',.~O
o
o
o
I-~ ~...o
l.-~t.o o
o
o
~-,.o
l-~ o
I }-~ o
o
i I I ~.,.i-.,,k.o~-,.i--,.~.,.k.~~-,.t,0 o k.~ ~,. o
l,o i..,.t~ l-,..~,o
! ~-~ ix) t,o l~0 ~.,. "~ ~-.~o
I I i-.~o k.~ I-~ I%) I-*k.~ i,,0%.~~-~i.,0%_o o
I I o k ~ F~ i,o I.-,.i-,.I-~ [x)k.~o,A)k.~ i-~ o
~-,.k.o 4:--~.~.~.,.[,ok.0 k.~ .~- ~-,..~-o
i-~ i-L C0 .~-k.o ~-.,.k.,0 ro %.m k.,0 4u---k~o
I I ~" k.~ I"ok..o ~':'~-" ~o l'ok m k.,~ O x ~[%)O
h)k~ O
O
i i-,.~....k.~.~-L ~,,.l-~ o l,o o
Ok.~ O
I O k.~ I--~O'-.k~ .~- OXk,,~ O0 O', O
O
I I-~'O
i-.'.l-,,h) t,ok.~ o o
I I I l*O I ',O O ~-~ 4:--~ C 0 0 - . , , 1 0 ! ! I
O
..~-O !
I .~- O !
I I--" ! O,,..~ I.-~O k.o--.,,]C o k ~ O % O l l l I-~. i-~. i--"h) i-~"Oxl-~ ~ - O x x O i-~ O O O
t~ O
i ~t'o~-" O
~-~'O
~.~.o
o
o
O
O
O
O
O
O
O
I--"..~'*-~ O', O I
I I ~ i-.~k.n CO--,~ O !
I-~ O k.~ O,, 4 : - O k O
O
O
I-" I%)k ~ O'.,PO CO h) O
~.',I-~k.~ .4z-~::-~--~k.~--.,,10 O,, I-~.O I I
I I-~'O
O
O
~
I-" O
O
O
O
I,O O I h-'.O
~-~ l-'.O
~-"l-'~k.o I-~ ~
~"-~O
O
O
O
O
o
O
O
O
0
0
I I I I I O O O i-~.i~.k.~ i-.,.CO O , , O O I I I 1 1 I I%) O t,O I-~ I-,,i-*-,,] ..~--i-,.O O ! I I I I h.." h) ',O O ~-'.O l-~ .~-k,.)O x - - , 3 0 O 3 0 I !
0
I ~
I I I Ix) CO l--bk~J0 i ! I I I I-~1 -I I-.~ PO - ~ 0 , , - ~ 0 0
I I-~ 0
I I-~ 0
I I-~ 0
I%) O O O ! I I I.~ I i-."i-~.Ix) ~-~.~.:.O
I-':'O
O
O
O
0
0
.n.
a,q
eb
q
179
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders
shows the difference between output of codec with and without transmission errors. The propagation is terminated about ten lines later. The horizontal error propagation ends in about 30 pels. Fig. 8(b) is the part of picture B quantizing noise which is very similar to Fig. 8(a). Fig. 8(c) shows that the quantizing noise reduced significantly, by using =~ and/3 =2. Both the leaky and nonleaky predictors are evaluated based on quantitative criteria [17]. This evaluation starts with Fig. 9. In each case, the leaky predictors have a clustered value of tre compared to the three nonleaky predictor. For test pictures C and D, leaky predictors perform better, gaining their advantage on isolated wideband transitions, i.e., reduce the picture overload. For test picture B, the highly active region is smaller than the uniform region. Because the introduction of leak becomes less efficient in a uniform region, it increases the prediction error in this region. The
improvement in ~e in the highly active region does not fully compensate the prediction errors in the uniform region. In picture A, the leaky predictors perform similar to the nonleaky predictors resulting from the loss in uniform region compensated by the gain in active region. The peak differential values are shown in Fig. 10 and, in general, the leaky predictors have the lower peak value. Statistically speaking, this result is significant because more of the larger predictions have been converted to smaller prediction errors, than smaller predictions, to larger prediction errors. In fact, the leaky predictors have performed better in edge prediction and middle range prediction than the nonleaky predictors. Non-leak -----Leak Z_~.)
--
105 100 95 9o 85
Z-3
20.0 Z-3-19.0
Nearest
~"18.0
Planar
Cr:.~ouT_l l Z-3 - Nearest -~ Neighbor \'%. Contour 2 ~= Contour I-2-D-II ----JNearest~ Neighbor ~ Planar - - A Contour 2--Contour 1 ~
17.0
o~
~ 16.0 ~15.0 Planar
[Z-D-n-- A
20INei :: \ Nearest__%
]Contour 1~' ~ 11' O~contour 2 ~ iC°nt°ur i/~ 1°'°rI~..... t J I
- -
Non
Leaky
-
Nearest
Noi g h b o ~ Planar
1-/-/ Contour .2-4l Nearest__/
Contour
Neighbor
.
. . . .
Contour 1"-~
Conto~2---~. 2_D_II~ / ~ Contour 2~_ Contour i--" 2-D-ll-Contour 2~F Nearest_~/ Nearest / Neighbor / Neighbor
14. O F
I
>,
Test Picture
leaky predictors p~=~trcfors
Fig. 9. Comparison between nonleaky predictors(a = 1,/3 = l) and leaky predictors (~ =~,/3 7 =~) ~ on the basis of the standard deviation of the prediction error, ,r~. :nonleaky predictors; . . . . . :leaky predictors.
70
65
2-D-II Planar
Contour 1__
iNearest
Neighbor- I Contour I~
~ ~o E 55
Z-3-
Neighbor--'~ Contour 2-Planar Z-3.~
-j5
%®
.~
Contour I ~
8o
1-~Contour 2-J
Contour
2-D-II ~
Neighbor-2-D-II Nearest
Contour I--~
Contour 2 ~ Nearest .... 40 Neighbor---~ Contour I--/ 35 Contour 2
/If
Neighbo{- / Planar . 9 ~ Contour 2 /
Z-3 g, 50 Nearest'-e-~ Neighbor--~ 45
Nearest
Nearest Neighbor
l--P/Contour 2-J Contour
Contour It
% 1 Nelghbor'\~[ Contour 2"-" Nearest
C
A ~est
D
Picture
Fig. 10. Comparison between nonleaky predictors (a = 1,/3 = -_,) on the basis of peak I) and leaky predictors (a =7, /3-2 differential value e0eak. In Fig. 11, the error signal entropies are compared between the leaky and nonleaky predictors. In general, the leaky predictors produce a larger entropy than the nonleaky predictors. With the Vol. 10, No. 2, March 1986
180
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive T V coders
Non-leak ---- --Leak 6.o 5.9
5.8 5.7 m
z 5.6
g ~5.5 Nearest
g s . , ,Neighbor 1 Contour 1-~ ..~ ~ 5.3 o
Nearest Neighb~r-]. Contour I~ Contour IJF Contour i-% N e a r e s t Contour 2J Nelghbo¥- -. Contour 2~ Contour 2V" N e a r e s t _ 2 | Contour ~ Neighbor | l Contour 1"]_~ Nearest___J Neighbor~l Contour 2J I Contour2-'] Nearest Neighbor--
Contour2 J
5.2
~o 5.1
o5.0 Contour I~I Contour 2 ~
Neighbo~ Contour~
g.9 4.8
A
B Test Picture
Fig. 11. Comparison between nonleaky predictors (a = 1,/3 = 1) and leaky predictors (a = ~,/3 = t) on the basis of the prediction error entropy H. exception of test frame D, predictors 1 and 2 exhibit a smaller trend. Although the leaky predictors yield larger entropy for three of the four cases, this margin is basically gained by increasing the spread of the smaller prediction errors (Fig. 12). The overall effect of lower epeak and (]re can be examined by plotting the cumulative distribution of the prediction error magnitudes. Fig. 13 shows the cumulative occurrence frequencies for the nonleaky and leaky predictors, for picture C where F(le]) is the occurrence frequency of the predictor error within the interval ( - e , e). For other test pictures, the results are similar. It can be observed from this figure that the leaky predictor has the advantage at medium and high prediction errors, but the nonleaky predictors perform better in reducing the number of smaller errors. For example, consider the cumulative occurrence frequencies for ]e I = 30. The difference in F(rel) for the leaky and nonleaky predictors is 1.58%. This means that when the leak function Signal Processing
with a =~7 and/3 =~l is used, on the average, 4053 more prediction errors per frame will have values below 30 rather than above 30. Note that the leak factors ~ and/3 play an important role. As c~ and /3 become smaller, the predictor becomes less efficient resulting in larger granular noise. If they continue to become smaller, the granular noise will be visible. A good choice for a and /3 is a - - ~7, 1 /3 = ~. For ct < 7 and fl < ½, the improved medium and high prediction errors cannot offset the increase in granular noise. In summary, the leaky predictors are more complex than the nonleaky predictors, because the former introduce the leak factor. From the test data, it can be seen that (a) the leaky predictors are more effective in reducing both large and medium prediction errors, (b) the propagation error is significantly reduced without increasing the transmission rates, and (c) the spread of the smaller prediction errors is increased (see Fig. 12).
6. C o n c l u s i o n s
Using the different predictors, both fixed and adaptive, and using the concept of providing leaks in both the predictor output and the predictor function, the propagation of channel errors is investigated by computer simulation. We may conclude the following. (1) The channel error propagation through the picture depends upon (a) the activity of the local region, (b) the magnitude of the error and whether or not the error is terminated is determined by the thresholds Hth, Vth, and SL, and (c) the type of predictors. In uniform areas, the propagation pattern is similar to that of the Z -3 predictor. In highly active areas, the error performance of the adaptive predictor is as bad as the fixed predictors and worse than the 2-D-II. Error propagation in the vertical direction is terminated if a horizontal boundary or uniform region is met. Horizontal error propagation is terminated for contour predictor if the error falls within a contour transition and is small enough so that the wrong contour is not met.
181
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders
Occurrence Frequency
(xlO 3) 20 Picture
C
Predictor •
Non-leak
o
Leak
1
10 0
00 •
O 0
O
O 0
0
0
5
0 0
0
O
• @
0
0 ~e
!
-30
o °
J
-e
-20
110
- 1, 0 Prediction
0
|
20
e
30
Error
Fig. 12. C o m p a r i s o n b e t w e e n n o n l e a k y predictors ( ~ = 1,/3 = 1) a n d leaky p r e d i c t o r s ( a = 7 , / 3 =½) on the basis of the p r e d i c t i o n error o c c u r r e n c e f r e q u e n c y f(e).
(2) By introducing a leak in the predictors, the channel error is significantly reduced and quickly attenuated or terminated. By quantitative tests, the performance is compared between leaky and nonleaky predictors. The leaky predictor has the following characteristics: (a) dramatically reduces the error propagation, (b) the mean square error is reduced, (c) the epeak is significantly decreased, and (d) in uniform regions, the quantizing noise is increased, i.e., larger granular noise. For a reasonable leak, this should not be visual: (e) in moderate areas, the quantizing noise is somewhat reduced, and (f) in highly active regions, the quantizing noise is reduced resulting in improved edgebusiness. (3) It is more important to decrease the number of large prediction errors than it is to decrease the
spread of the smaller prediction errors, which are quantized with high precision. Based on this study, the following recommendations are made. - T h o u g h the leak function can reduce a n d / o r terminate the error propagation, it is not enough to obtain the high quality picture in a noisy channel (see Fig. 8(c)). Error protection by forward error correction is therefore necessary, even though this will increase the transmission rate. - The error-correcting code should be capable of overcoming both random and burst errors. It may be worthwhile to investigate the effects of other schemes such as hybrid D-PCM [28] and DDC [30] in reducing the channel error sensitivity of the proposed contour predictors. Vol. 10, No. 2, March 1986
182
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders
lOO. ooo °
o~. it t Jt ~t It It
t
~95
i o
}85
Predictor
x a=l, [J.t
X0
i
•
.o2
t
~'-r/8,
~=J
o o¢=7/e,(~=J/2 X
,O
x
i
o
?o.
6g.
X e
60
o '
'
' 30
' 40
J 50
lOAir,plitud2Oof ±h~ Prediction Error, lel
' 60
70
Fig. 13. Cumulative occurrence frequency of the prediction error amplitude for the nonleaky and leaky predictors (picture C).
Acknowledgment The authors sincerely appreciate the reviewer's comments which have resulted in a substantial improvement of this paper.
References [1] C.K. Un and H.S. Lee, "A study of the comparative performance of adaptive delta modulation systems", IEEE Trans. Commun., Vol. COM-28, January 1980, pp. 96-101. [2] C.C. Cutler, "Differential quantization of communication signals", U.S. Patent 2605361, July 29, 1952. [3] J.B. O'Neal, "Predictive quantizing systems (DPCM) for the transmission of television signals", BSTJ, Vol. 45, May/June 1966, pp. 689-721. [4] J.P. Rossi, "Sub-Nyquist-encoded PCM NTSC color television", SMPTE J., Vol. 85, January 1976, pp. 1-6. [5] H. Kaneko and T. Ishiguro, "Digital television transmission using bandwidth compression technique", IEEE Trans. Commun., Vol. COM-28, July 1980, pp. 14-22. [6] K. Sawada and H. Kotera, "A 32 Mbit/s component separation DPCM coding system for NTSC color TV", IEEE Trans. Commun., Vol. COM-26, April 1978, pp. 458465. Signal Processing
[7] K. Sawada and H. Kotera, "A 32 Mbit/s transmission of NTSC color TV signal by composite DPCM coding", IEEE Trans. Commun., Vol. COM-26, October 1978, pp. 1432-1439. [8] R. Burkhardt and J. Wasser, "Digital television transmission with 34 Mbit/s", SMPTE 1, Vol. 89, April 1980, pp. 244-248. [9] J.E. Thompson, "Differential encoding of composite color television signals using chrominance-corrected prediction", IEEE Trans. Commun., Vol. COM-22, August 1974, pp. 1106-1113. [10] I. Dinstein, "DPCM prediction for NTSC composite signals", Comsat Tech. Review, Vol. 7, Fall 1977, pp. 429-446. [11] N.R. Cox, "An adaptive encoder for NTSC composite video", National Telecommun. Conf., New Orleans, LA, pp. C9.4-1-C9.4-5, November 29-December 3, 1981. [12] W. Zschunke, "DPCM picture coding with adaptive prediction", IEEE Trans. Commun., Vol. COM-25, November 1977, pp. 1295-1302. [13] P. Cohen and J.P. Adoul, "Adaptive differential coding of picture signals based on local contour prediction", 1976 Nat. Telecommun. Conf. Rec., Dallas, TX, 1976, pp. 6.1-16.1-5. [14] P. Camana, "Video bandwidth compression: A study in tradeoff", IEEE Spectrum, Vol. 16, June 1979, pp. 24-30. [15] J.O. Limb, C.B. Rubinstein and J.E. Thompson, "Digital coding of color video signals: A review", IEEE Trans. Commun., Vol. COM-25, November 1977, pp. 1349-1385. [16] A.K. Jain, "Image data compression: A review", Proc. IEEE, Vol. 69, March 1981, pp. 349-389. [17] N.R. Cox, "An adaptive DPCM algorithm for predicting contours in NTSC composite video signals", Conf. on Pattern Recognition and Image Processing, Dallas, TX, August 1981, pp. 240-247. [18] N.F. Maxemchuck and J.A. Stuller, "Reduction of transmission error propagation in adaptively predicted, DPCM encoded pictures", BSTJ, Vol. 58, July 1979, pp. 14131423. [19] J.E. Thompson, "Differential coding for digital transmission of PAL color television signal", IEEE Conf. Publ. 88, pp. 26-32; presented at: lnternat. Broadcasting Conf., London, England, September 1972. [20] J. Max, "Quantizing for minimum distortion", IRE Trans. Inform. Theory, Vol. IT-6, March 1960, pp. 7-12. [21] D.K. Sharma and A.N. Netravali, "Design of quantizers for DPCM coding of picture signals", IEEE Trans. Commun., Vol. COM-25, November 1977, pp. 1267-1274. [22] E.G. Bowen and J.O. Limb, "Subjective effects of substituting lines in a video-telephone signal", IEEE Trans. Commun., Vol. COM-24, October 1976, pp. 1208-1211. [23] R.W. Hamming, "Error detecting and error correcting codes", BSTJ, Vol. 29, April 1950, pp. 147-160. [24] W.W. Peterson, Error-Correcting codes, M.I.T. Press, Cambridge, MA, and Wiley, New York. [25] R.C. Bose and D.K. Roy-Chaudhuri, "On a class of errorcorrecting binary group codes", Inform. and Control, Vol. 3, September 1960, pp. 279-290. [26] A. Hocquenghem, "Codes correcteurs d'erreurs", Chiffres, Vol. 2, 1959, pp. 147-156.
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders [27] S. Lin, An Introduction to Error-Correcting Codes, PrenticeHall, Englewood Cliffs, N J, 1970. [28] M.C.W. van Buul, "Hybrid D-PCM, a combination of PCM and DPCM", IEEE Trans. Commun., Vol. COM-28, March 1978, pp. 362-368. [29] R. Steel, D.J. Goodman and C.A. McGonegal, "A difference detection and correction scheme for combating DPCM transmission errors", IEEE Trans. Commun., Vol. COM-27, January 1979, pp. 252-255. [30] H.J. Matt and J.L. Massey, "'Determining the burstcorrecting limit of cyclic coder", IEEE Trans. Inform. Theory, Vol. IT-26, May 1980, pp. 289-297.
183
[31 ] A.N. Netravali and J.O. Limb, "Picture coding: A review", Proc. IEEE, Vol. 68, November 1980, pp. 366-406. [32] V. Devarajan and K.R. Rao, "DPCM coders with adaptive prediction for NTSC composite TV signals", IEEE Trans. Commun., Vol. COM-28, July 1980, pp. 1079-1084. [33] V. Devarajan and K.R. Rao, "Channel error propagation in predictor adaptive differential pulse code modulation (DPCM) coders", Proc. SPIE, Advances in Image Transmission II, Vol. 249, San Diego, CA, July 31-August 1, 1980, pp. 28-33.
Vol. 10, No. 2, March1986
171
C H A N N E L ERROR P R O P A G A T I O N IN ADAPTIVE TV CODERS* C.J. H W A N G and K.R. RAO Department of Electrical Engineering, The University of Texas at Arlington, Arlington, TX 76019, U.S.A. Received 31 October 1984 Revised 14 February 1985 and 14 August 1985
Abstract. DPCM coders with adaptive predictors are used and compared with nonadaptive DPCM coders for processing composite color television signals. The transmission error propagation for different predictors both adaptive and fixed is investigated. Propagation of transmission noise is dependent on the type of prediction and on the location of noise, i.e., whether in a uniform region or in an active region. By introducing leak in predictor output and/or predictor function, it is shown that error propagation can be significantly reduced. The leaky predictors not only attenuate and/or terminate the channel error propagation but also improve the predictor performance based on quantitative evaluation such as peak value and mean-square-error. In order to protect the high picture quality from error propagation, an error-correcting code is needed. Zusammenfassung. Der Einsatz von DPCM-Codierverfahren mit adaptivem Prfidiktor bei der Verarbeitung zusammengesetzter Videosignale wird untersucht; diese Verfahren werden mit adaptiven DPCM-Codierverfahren verglichen. Insbesondere wird fiir verschiedene--adaptive wie nichtadaptive--Pr~idiktoren die Fehlerfortpflanzung bei 0bertragungsstSrungen untersucht. Die Fortpflanzung rauschartiger UbertragungsstSrungen h~ingt v o n d e r Art des Pr~idiktors ab, aul~erdem besteht eine Abh~ingigkeit yon der Stelle, an der das Rauschen auftritt, d.h., davon, ob das Bild an dieser Stelle wenig oder stark strukturiert ist. Wie gezeigt wird, kann die Fehlerfortpflanzung dutch die Einfiigung einer k/instlichen D~impfung in das Ausgangssignal des Pr~idiktors u n d / o d e r die zur Ubertragung anstehende Pr~idiktionsfunktion erheblich reduziert werden. Pr~idiktoren dieser Art d~impfen bzw. beenden nicht nur die Fortpflanzung von l'_'lbertragungsfehlern, sic ver~esseru dariiber hinaus auch das Verhalten des Gesamtsystems, wie sich durch quantitative Messungen mit Hilfe des Kriteriums der maximalen Abweichung oder des mittleren quadratischen Fehlers nachweisen l~il3t. Um die hochwertige Bildqualit~it vor 0bertragungsfehlern zu sch/itzen, ist allerdings ein fehlerkorrigierender Code bei der Ubertragung notwendig. R6sum6. Les codeurs MICD avec des pr6dicteurs adaptatifs sont utilisgs et compar6s avec les codeurs non adaptatifs MICD pour traiter les signaux composites de t616vision couleur. La propagation de l'erreur de transmission pour les diff6rents prgdicteurs, adaptatifs et non adaptatifs, sont investigugs. La propagation du bruit de transmission est d6pendant du type de pr6diction et de la position du bruit, c'est-b.-dire si le bruit se trouve dans une rggion uniforme ou une r6gion active. II esl: montr6 que la propagation des erreurs peuvent ~tre r6duite consid6rablement en introduisant une distribution ~t la sortie du pr6dicteur et/ou fi la fonction de prgdiction. Les pr6dicteurs distributifs non seulement attenuent et/ou arr~tent la propagation des erreurs de voie mais am61iorent aussi les performances des pr6dicteurs bas6es sur des gvaluations quantitatives telles que valeur de pic et erreur quadratique moyenne. Pour prot6ger la haute qualit6 des images de la propagation d'erreur un code correcteur d'erreur est n6cessaire. Keywords. Predictive coding, TV signals, channel error propagation.
1. Introduction Predictive techniques for digital processing of signals can be broadly classified as (i) delta modu* This paper is based on the research by Mr. C. J. Hwang for his Master Thesis as a partial requirement for the M.S. Degree from the University of Texas at Arlington, TX.
lation (DM) [1], and (ii) differential pulse code modulation (DPCM) [2,3]. The latter involves prediction (adaptive or nonadaptive) of the present sample based on previous sample(s). The error signal representing the difference between the actual sample and its predicted value is quantized, coded and transmitted over a digital link for recon-
0165-1684/86/$3.50 (~ 1986, Elsevier Science Publishers B.V. (North-Holland)
C.J, Hwang, ICIL Rao / Channel error propagation in adaptive TV coders
172
struction at the receiver end (Fig. 1). In a good prediction scheme, the range and entropy of the error signal are small compared to those of the
, Prwdtctor
'
error sensitivity of adaptive DPCM coders for composite color TV signals is now analyzed. It is hoped that this investigation can contribute to the design of channel encoder for incorporation into the codec built by Cox [11].
+
2. Data base (a) ~'rom Channel
(b) Fig. 1. Block diagram of a DPCM system: (a) transmitter, (b) receiver.
original signal. By incorporating appropriate quantization and coding schemes, the error signal can be transmitted at a reduced bit rate thereby achieving significant band-width compression. DPCM and other image coding techniques are described elsewhere [15, 16, 31]. The DPCM concept has been utilized by several researchers for transmitting broadcast quality TV signals [4-13]. For color image sequences, various DPCM techniques have been proposed, simulated and some have been implemented in hardware [11, 14-17]. In particular, Cox [11] has designed and built an encoder which implements the DPCM simulator (predictor, quantizer, buffer, etc.,) for digital processing of composite color TV signal sampled at 10.7 MHz. This encoder, however, has no error detection and correction capability. In previous papers [11, 17, 32], some adaptive methods based on contour prediction were reported. Their effectiveness in redundancy reduction was analyzed in terms of both quantitative and subjective criteria. Initial results on channel noise propagation in adaptive predictors have been reported [33]. By introducing leaks in the predictor function and/or predictor output [18], it has been shown that the transmission error propagation can be considerably reduced. Their ability to combat the channel Signal Processing
The data base is obtained from the NTSC color television signal, which has a bandwidth of 4.2 MHz. Such a signal has 30 frames per see, where each frame consists of 525 horizontal lines arranged in two interlaced fields. This signal was sampled at 10.74 MHz (three times the color subcarder frequency, f~c = 3.58 MHz) and was digitized to 8 bit PCM. Only a 512 x 512 array of every single frame, corresponding to most of the visible portion of the video signal was used for simulation. Four static pictures were used for simulation (see Fig. 2): PictureA: Lady head and shoulders (Fig. 2(a)). Picture B: Newscaster with a news slide in the background (Fig. 2(b)). Picture C: Fred and Wilma Flintstone (Fig.
2(c)). PictureD: An interviewer in a crowded background (Fig. 2(d)).
3. Predictor
Contour predictors originally developed by Zschunke [12] for monochrome images have been extended to color images [11, 17,32,33]. These adaptive predictors can track co~atours, slopes, and transitions and therefore can better predict a pel in an active region compared to the fixed predictors. The contour predictors 1 and 2 [17] (described below) together with the 'nearest neighbor' predictor [13] and fixed predictors are simulated. Pel locations are described in Fig. 3. Pels with the same notation have similar subcarrier phase.
173
C.J. Hwang, K.R. Rao / Channel errorpropagation in adaptive TV coders
Fig. 2. (a) Original picture A. (b) Original picture B. (c) Original picture C. (d) Original picture D.
Vth ( Vth
is the vertical threshold), and
Is2- $71<1s - $1ol. •S 6
S5
¢ : o # S 4 S 3 S 2 Sj
:
:
o
Fig. 3. Notation for contour prediction.
Predictor 1
(I) Using the last reconstructed sample $2, ( S 2 - S s ) is obtained. If IS2-$51 is less than some horizontal threshold Hth, a uniform region or horizontal contour is assumed and S~ is predicted as $4, i.e., $1 = $4. (II) However, if Is=-ssl> Hth, a contour is assumed. (a) $ 1 = S 6 if IS7-S,oI>SL (SL is the minimum slope requirement), sign ( $ 7 - S~o) = s i g n ( S 2 - $5), IS2- $71 <
(b) S~ -- $9 if IS~o- S~3[ > SL, sign(S~o$13) = sign(S2 - $5), IS2 - S~ol < Vth, and IS2- $1o1 < IS2- S7I. (III) If both tests (II)(a) and (II)(b) fail, it is difficult to determine a contour direction. Hence, $1 = $4. The thi'esholds chosen are Hth = 8, SL = 8, and Vth = 64. Predictor 2
Predictor 2 is identical to predictor 1 with the following exception. If both tests (II)(a) and (II)(b) fail, and if IS2 - $51/> Wth, then $1 = $2. Wth is the wideband or large threshold transition (Wth = 32). In a prototype e n c o d e r / s i m u l a t o r [11] that has been built, both predictors 1 and 2 have been implemented in real time. Vo[. 10, No, 2, March 1986
174
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders
Predictor 3
Predictor 3 is the composite equivalent of the 'nearest neighbor' predictor [13] and is defined as follows:
The contour predictors (predictors 1 and 2) have been utilized in composite color TV image coding [11, 17, 34].
rS6 if IS2-&l< &-S:o[ 4. Quantizer
and I S 2 - $71 < I S 2 - $51 ,
~ = S~ if ls2- S:ol < ls2- s71 $4
and [Sz- $1o1< [Sz - $51, otherwise.
Predictor 4
Predictor 4 is the 2-D-II composite predictor proposed by Dinstein [10], i.e., ,~,=S9+
½(S~- Slo). Predictor 5
Predictor 5 is the composite planar predictor [19]. Sl = $ 2 + $ 9 - S 1 o . Predictor 6
Predictor 6 is the simple S -3 predictor. $1 = $4.
Based on a statistical study of prediction errors, a max symmetric quantizer [20] for minimizing the mse was designed by Cox [17]. This quantizer was later adjusted subjectively based on the visibility thresholds using the DPCM simulator built by Cox [ 11]. Reflecting a compromise between the quantizer complexity and minimizing the number of bits per pel (BPP), a dual word length quantizer (Table 1) with corresponding code representation (Table 2) was adopted [11, 17]. To match this code to a constant bit rate, provision has been made for a fixed word length mode whenever the number of long words along a horizontal scan line exceed a preset limit.
Table 1 29/15 level symmetricquantizer based on visibilitythresholds [17] (only positive levels are shown) Dual-Length Input
Output
Fixed-Length Code
Input
Output
0
0
0 - 1
0
1
1
2 - 5
3
2
2
6 - II
8
3
3
12 - 19
15
4 bit
4-
6
5
20 - 28
24
7-
9
8
29 - 38
33
14
12
39 - ~o
51 - 255
io
-
20
17
21 - 28
24
29 - 38
33
39 - 50
44
51 - 63
57
64 - 77
70
15 -
78
-
93 -
Signal Processing
Mode
92
255
85 IOO
8 bit
57
Mode Code
4 bit
175
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders
Table 2 4/8 bit dual-word-length code construction [17]
but also corrected. However, there is a limit to the burst error correction of cyclic codes [30].
1111 1110 1101
5.1. Effect o f channel errors 4 bit words .......... a s s l g n e d to 15 most-probable levels
O6Ol oooo
.......... long word prefix
00001111
.......... frame sync code
ooooliiO 00001101 t oooolioo
8 bit long words ....... assigned to 14 least-probable levels
ooo6oool
00000000
.......... not a s s l K n e d
5. Channel error propagation
Error protection, for the D P C M coded video signals can be provided in many ways. One method is by predictor leak, that is, using a fraction [15] of the previous element value for prediction. By using a recent idea of prediction function leak [ 18], the transmission error propagation can be significantly reduced. This is the so-called 'leak factor' or 'leaky integrator' concept. This technique has the advantage of maintaining the transmission bit rate, but the predictor may become less efficient. A second method adds redundancy in the form of periodic PCM updates. Another technique is to add error-detection codes which enable single or multiple errors to be detected at the receiver. When an error appears, the erroneous sample is replaced with an estimate which is close to the actual value. This is called error c o n c e a l m e n t [22]. When high error rates are to be expected, this method is preferred. This is because more errors can be detected than corrected, given the amount of added redundancy. In the forward error correction scheme [23-27], error correcting codes are encoded in the transmitter and are decoded in the receiver such that single or multiple errors are not only detected,
The channel perturbations were introduced between the transmitter and the receiver for investigating the sensitivity of the predictor to transmission errors. An error propagation is effectively terminated when the receiver output is refreshed by the PCM even though this technique leads to an increase in the overall bit rate. This additional bit rate can, however, be avoided by a technique called hybrid D-PCM proposed by Van Buul [28]. In this method, by a clever combination of the PCM and DPCM, the advantages of both, i.e., the fast error recovery of the former and the bit-rate reduction of the latter, are retained. Another technique for counteracting the error propagation in a DPCM coder is the difference detection and correction ( D D C ) scheme [29]. In this method the difference criterion is adaptively set and an error detection is followed by the correction of the DPCM output sequence. The error propagation not only depends on the type of predictor, but also on the location of error. The areas of the picture chosen for error examination were categorized as uniform, moderately active, or highly active defined as follows [17]: uniform moderately active: highly active :
0.0 <~cc/pel < 0.05, 0.05 <~cc/pel < 0.4, 0 . 4 ~< cc/pel --z-1.0,
where cc/pel was computed by dividing the number of contour checks (cc) (predictors 1 and 2 describe the contours) performed within the region by the number of pels within the region. Contour check refers to # of contours for predictors 1 and 2. Exceeding the horizontal threshold Hth or the vertical threshold V~h counts as a contour check. The error patterns for predictors 4-6 are shown in Fig. 4. The Z 3 predictor propagates the error in horizontal direction until it disappears at the end of line. The planar predictor distributes error propagation in both the vertical and horizontal V o l 10, No. 2, March 1986
176 0 0 0 0 0 0 0 0
C.J. Hwang, ICR. Rao / Channel error propagation in adaptive TV coders 0 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0
o o o oo 00
O 18 18 18 18 0 0 0 0 0
0 18 18 18 18 18 0 0 0 0
0 18 18 18 18 18 18 0 0 0
0 18 18 18 18 18 18 0 0 0
0 18 18 18 18 18 18 18 0 0
0 18 18 18 18 18 18 18 o
0 0 1 2 4 18 0 0 0 0 0
0 0 0 1 2 9 18 0 0 0 0
0 0 0 1 1 4 9 0 0 0 0
0 0 0 0 I 2 4 18 0 0 0
0 0 o 00 1 2 9 18 0 o
0
(a) 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0 0 0
0 18 18 0 0 0 0 0 0 0
0 18 18 0 0 0 0 0 0 0
0 18 18 18 0 0 0 0 0 0
0 18 18 18 18 0 0 0 0 0
18
(b) 0 0 0 0 0 0 0 0 0 0 0
0 0 0 C 0 0 0 0 0 0 0
0 18 0 0 0 0 0 0 0 0 0
0 9 18 0 0 0 0 0 0 0 0
0 4 9 0 0 0 0 0 0 0 0
0 2 4 18 0 0 0 0 0 0 0
0 1 2 9 18 0 0 0 0 0 0
0 1 i t, 9 0 0 0 0 0 0
by the Hth, SL, and Vth parameters, to find a general error propagation pattern is very tedious. As the amount of luminance activity increases, the opportunity for the error propagation increases. As Hth is increased, errors will propagate further in the horizontal direction, but less in the vertical direction. In summary, the propagation of transmission errors is influenced by (a) type of predictor, i.e., fixed, leaky fixed, or adaptive predictor, (b) one-, two-, or three-dimensional, i.e., intraline, intrafield, or interframe prediction, (c) the location of the error, i.e., whether in uniform region or highly active region, and (d) in the case of the contour predictor, it depends mainly on the activity of the local region, the duration of the contours, and the magnitude of the error. The characteristic of the adaptive predictor is very similar to that of a Z -3 predictor in the uniform region and is worse than that of a fixed predictor in the highly active region. Some examples of error propagation for adaptive predictors are shown in [33].
(c) Fig. 4. Error patterns whose decoded value is 18. (a) Predictor 6, (z-3). (b) Predictor 5, planar. (c) Predictor 4, 2D-II.
directions. The 2-D-II predictor is monotonically decreasing the error propagation in the horizontal direction. However, the error extends diagonally to each successive line. In the case of adaptive predictors, a transmission error can happen both due to the wrong value for the prediction and due to selecting a wrong predictor. The error propagation is similar to the Z -3 predictor, as long as the signal slopes of the previous line remain below SL (over a 3-pel distance). Some conditions will terminate the error propagation. If the error has the same sign for the slope which lasts for the next three pels, the horizontal propagation error will end. If the error has the opposite sign, horizontal propagation error must last for three pels. In either case, if the error is small enough so that it does not affect the selection of the contour direction, then no propagation occurs. Because the contour direction is affected Signal Processing
5.2. Predictor output attenuation [18] The predictor output leak and predictor function leak can provide significant reduction in channel error propagation without decreasing the transmission rate [18]. The basis for the technique is a generalization of the notion of predictor output attenuation which includes attenuation of the adaptive prediction function.
5.3. Predictor output leak The effect of transmission errors can be reduced by attenuating the predictor output F ( . ) by a constant a, 0 < a < 1, such that g = c~F(') + ( 1 - a)r/, where S is the new predicted value of the predictor output and r/is a constant in the span of possible picture values. The prediction error has the form
S - S= S - ( ~ F ( " )+ (1-u)~I) = a ( S - F ( ' ) ) + (1 - ~ ) ( S - r/),
177
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders 0 0
0 0 -1
-108 0-108 0-114 0
-2
-35 0 -35 17 -14 69 -35 17
-21
-35 12 -26 3 -14 -21
5
0 0 0 0 0 0 3 O-lO8 0 0-108 0-108 -1 0-108 -1 0 -91 -7 O-lO8 -7 0-111 -1 0-111 -1 -5 0 -2 -5 0 -2 0 - 3 5 -21 0 - 3 5 -21 0 -35 21 0 - 3 5 21 - 1 2 25 - 2 6 -21 17 -23 - 2 1 17 - 2 6 -21 17 -23 69 - 1 4 -21 17 -14 -21 -21 17 - 1 4 -21 17 -14 5 -21 11 5 -21 17 5 -28 17 5 -28 17
0
0
3
O-lO8
0
0 0 0 19 21 2 0-108 -1 -1 21 -91 -1 0-111
0 19
0 0-108 -1 -1 -7 21 -91 -1 -1 0-111 o -2 -5 -5 0 -2 -5 0 0 - 3 5 -21 0 - 3 5 -21 o -35 21 21 0 -35 21 17 - 2 3 -21 -21 17 -23 -21 -21 17 -23 -21 17 -23 -21 17 -14 -4 51 - 2 3 -4 51 -21 17 -14 -4 51 - 2 3 -4 5 -21 51 17 5 -21 51 5 -21 -28 17 5 -21 17
Fig. 5. Difference between receiver output with and without transmission error, picture C line 10-23 error value -100, a = 1,/3 = 1, predictor 1.
where /3 is a constant, 0
where S is the input data. The error consists of both D P C M a n d . P C M information. As a approaches zero, the system changes from D P C M to PCM. The reduction in error p r o p a g a t i o n for picture C from line 10 to line 23 is shown in Fig. 6 for a = ~ 7 (compare with Fig. 5). Although an i m p r o v e m e n t is obtained with this approach, the error is still intolerable. Further reduction in a leads to visible degradation in the output caused by the quantizing noise. Hence, a is chosen to be 7 15 31 a constant ~>~ such as Tg, ~5.4. Prediction f u n c t i o n leak [18]
Since the effect of channel errors in an adaptive codec is to make the value o f the prediction function F ( . ) uncertain, the receiver loop must in fact estimate not only the input data S but also the function F ( . ). The prediction is set by S=/3F(.) +(1-/3)F(. 0
-105 7 -1 -43 10 0 0 0 0 0 0 o
Using this method, Fig. 7 (in highly active region)
),
0 0 0 0 0 -92 0 -71 -8 -4 - 5 5 -1
6 -18 -37 5 o 9 8 0 o 0 0 o 0
,S=oq3F(')+a(1-/3)F(')+(l--o~)'O.
14 18 0 0 0 o o
2 0 0 0 0 o o
0 0 o 0 0 -81 o -63 -7
-4 -48 4 5 -31 -31 4 o 8 7 3 o 0 o o o 0 o o 0 0 o
o -25
0 0
0 0 0 -70
o -55
-6
-4 -42 3 5 -27 -28 3 o 7 6 3 o 0 0 o 0 0 0 - 9 16 - 1 5
-36
0 o 0 0 o -62 0 -/s8 - 6 -4 -37 3 4 -23 -24
3
o
6
5 2 0 0 0-100
0 19
0 - 3 6 -32
0
0 -28 -36 - 1 9 -31 - 3 1 -7 -6 0 -34 -22 - 1 5 -2 - 6 - 1 3 - 4 0
-8 8
-3
0 0 0 o o -42 -3 -32 3 -20 0 3 4 2 0 -9 0 -9 0 -9 -12 0 -13 -6 -12 35
Fig. 6. Reduction of transmission error propagation using predictor output leak (a = g,7/3 = 1), picture C line 10-23, error value -100, predictor 1 (compare with Fig. 5). Vol. 10, No. 2, March 1986
o
II
II
@
z2
II
~'~
O.z
~'~
00
o
N
,O
.m
I
I
I
I
I
I
I
I
I I
I I I I I ~-~ ~ , ~..~ h.) I.-~ 0
I
I
I
!
I
I
I
I
I
I
I
I I ~ 0
0 I
I ~
I
I
0
o
O
O O
!
!
r
I I I i%) I-.~ 4~-i.~ I ~ t.~
I
! I i'~o O x O
I I O ~.-~..~o
I ! O i-~ ~-.~ O
O k ~
I
I I t~ 0',4=" Ox~'-~%~00
f
o
i-.~ o
o
! ! r.-~ o % . m t,o o to o
! I ! O I..~ O ~-~ 4 = - k ~ ~-~ ~-~ r o ~.-~
O
I I ~-~ ~
o
O O
I
!
o
I I I I O ~-~, ~.-~ O K,~ t~O O ~-~ . ~ - r o
I
~-~ 0
I ~-~ ~
0
O
OO
. ~ I'o Ix) ~-'~ ~-~ t'o
O t,O I--s O k ~ ' , . ~ O O k.o r o
I
I I I I I O i-~ O I-~ I.-~ t~O O I~O I-~ I ~
0
I I I I ' 0 1 -L I"0 ~'~ ,'4='I'0 I '0 0 ~ -'L
I
I
I ~0 ~-~ 0
0
I
0
0
0
I
0
I
~) O
I ~-~.O
' oo
0
t r~. O
I ~-,,.O
0
O
o
I
I
0
0
I
I l
I
I
I ro h-~.O
I
I ~ i'~O 0
I ~-~ 0 I.-~ 0
Ik) O
o
O
o
i
i
O
I i-~
i
!
! l.~o.4~-o
! l,o i-~ O
C,xk.o l.-*O0 h-~
I I~ ~-" h..~.I-~ ~,0 po i-~ -.,.]o
I I I h.~.i-~ -..40 k.m ~-~.O
I ~-~ 4 ~ - 0
I-.,.O
I I f r~ t-* i-.,,k.0 h-,,
! i-~ t i-...r,0 t..-,.~.~ .4~-o
I h-,.o
I ~,. 0
I I I I I'~ I--,~ I-~ ..~- I.~
I
t,O O
I ! I-.,.l-~ b O O
t.o ~-b i-~.O
O
O
0
I h..~O
l..OI,Ok.O ~-~ O i
O
I
0
I I O k , . ) . ~ , , ~ I-~
~-'~ 0
0
I
0
I I I'o ~-~ 0
I ~. 0
0
I I--.,. ~-,~ I~0 0
0
0
~) ~
I
I
I,~. 0 I
O
' !
0
I I I I ~ 1,0 ~-,. ~,.
0
I
0
I
I
I
I
/ ~
I
!
'
~ I
~-.~k.~ O
O K,,~ O
k.~. t,O I I-~.l',O-,,3 Ik) O
O
o
I l I r ~ i,~ i-~
I
! I-,.l,O h) 1.-~.k~ O
I
! I..~I...~O
O
t
0
0
I ~,0 4=-- 0
I I I ~-~ I"~ r ~ I~o o
J.,..,..,..O,..,..,~-l,O O
O O I
O
I
I k.~ I~,0 0
I
.~ I I ,~a i-,. l-U 1.0 i-,~ 0
I
i-~ i-,,~ 0
I ~,. ~.~. 0
f'O IX.) ~,~ 0
0
I
I I I r,o 4::- O~ I~,
0
0~0
l t l ~,~ ~a, i.~ i...~ 0
I ~,, I-~ 0
I %o ~ . 0
l..a. 0
0
0
I ~. 0
0
I
I i..~. i.,~. 1.0 I.~
0 ~
I I k.r,,. I ~ i-~ I-~. 0
I ~,,. ~,~ 0
I i~0 i.~ I.~ I~0 0
I',0 0
0
II
II
p~
e~
_~.,.
(m
0 -t
(m
..~
o
5"
o
o
o
o
o
o
o
o
o
o
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O O
O
0
0
O
0
0
o
o
o
o
o
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
0
0
o
o
o
o
o
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
0
0
o
o
o
o
o
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
0
0
o
o
o
o
o
o
o
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
O
0
0
O
O
O
0
0
O
O
O
0
0
O
O
O
0
0
O
O
O
O
O
O
O
o
o
O
O
O
O
O
O
O
I I-".O O
O k . O i-~k.~ O,, O
l.-'.k.~O % CD 00-.,,] O I b,) ~-~.bo .O- Ox-.,.],,O k.m O
Ok~
O
~-" h ~ k ~
.~-Oxk.,~ O x 4~- O
I i-" k.0 k.0 ,,.~ O,.--3 -',3',.~O
o
o
o
I-~ ~...o
l.-~t.o o
o
o
~-,.o
l-~ o
I }-~ o
o
i I I ~.,.i-.,,k.o~-,.i--,.~.,.k.~~-,.t,0 o k.~ ~,. o
l,o i..,.t~ l-,..~,o
! ~-~ ix) t,o l~0 ~.,. "~ ~-.~o
I I i-.~o k.~ I-~ I%) I-*k.~ i,,0%.~~-~i.,0%_o o
I I o k ~ F~ i,o I.-,.i-,.I-~ [x)k.~o,A)k.~ i-~ o
~-,.k.o 4:--~.~.~.,.[,ok.0 k.~ .~- ~-,..~-o
i-~ i-L C0 .~-k.o ~-.,.k.,0 ro %.m k.,0 4u---k~o
I I ~" k.~ I"ok..o ~':'~-" ~o l'ok m k.,~ O x ~[%)O
h)k~ O
O
i i-,.~....k.~.~-L ~,,.l-~ o l,o o
Ok.~ O
I O k.~ I--~O'-.k~ .~- OXk,,~ O0 O', O
O
I I-~'O
i-.'.l-,,h) t,ok.~ o o
I I I l*O I ',O O ~-~ 4:--~ C 0 0 - . , , 1 0 ! ! I
O
..~-O !
I .~- O !
I I--" ! O,,..~ I.-~O k.o--.,,]C o k ~ O % O l l l I-~. i-~. i--"h) i-~"Oxl-~ ~ - O x x O i-~ O O O
t~ O
i ~t'o~-" O
~-~'O
~.~.o
o
o
O
O
O
O
O
O
O
I--"..~'*-~ O', O I
I I ~ i-.~k.n CO--,~ O !
I-~ O k.~ O,, 4 : - O k O
O
O
I-" I%)k ~ O'.,PO CO h) O
~.',I-~k.~ .4z-~::-~--~k.~--.,,10 O,, I-~.O I I
I I-~'O
O
O
~
I-" O
O
O
O
I,O O I h-'.O
~-~ l-'.O
~-"l-'~k.o I-~ ~
~"-~O
O
O
O
O
o
O
O
O
0
0
I I I I I O O O i-~.i~.k.~ i-.,.CO O , , O O I I I 1 1 I I%) O t,O I-~ I-,,i-*-,,] ..~--i-,.O O ! I I I I h.." h) ',O O ~-'.O l-~ .~-k,.)O x - - , 3 0 O 3 0 I !
0
I ~
I I I Ix) CO l--bk~J0 i ! I I I I-~1 -I I-.~ PO - ~ 0 , , - ~ 0 0
I I-~ 0
I I-~ 0
I I-~ 0
I%) O O O ! I I I.~ I i-."i-~.Ix) ~-~.~.:.O
I-':'O
O
O
O
0
0
.n.
a,q
eb
q
179
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders
shows the difference between output of codec with and without transmission errors. The propagation is terminated about ten lines later. The horizontal error propagation ends in about 30 pels. Fig. 8(b) is the part of picture B quantizing noise which is very similar to Fig. 8(a). Fig. 8(c) shows that the quantizing noise reduced significantly, by using =~ and/3 =2. Both the leaky and nonleaky predictors are evaluated based on quantitative criteria [17]. This evaluation starts with Fig. 9. In each case, the leaky predictors have a clustered value of tre compared to the three nonleaky predictor. For test pictures C and D, leaky predictors perform better, gaining their advantage on isolated wideband transitions, i.e., reduce the picture overload. For test picture B, the highly active region is smaller than the uniform region. Because the introduction of leak becomes less efficient in a uniform region, it increases the prediction error in this region. The
improvement in ~e in the highly active region does not fully compensate the prediction errors in the uniform region. In picture A, the leaky predictors perform similar to the nonleaky predictors resulting from the loss in uniform region compensated by the gain in active region. The peak differential values are shown in Fig. 10 and, in general, the leaky predictors have the lower peak value. Statistically speaking, this result is significant because more of the larger predictions have been converted to smaller prediction errors, than smaller predictions, to larger prediction errors. In fact, the leaky predictors have performed better in edge prediction and middle range prediction than the nonleaky predictors. Non-leak -----Leak Z_~.)
--
105 100 95 9o 85
Z-3
20.0 Z-3-19.0
Nearest
~"18.0
Planar
Cr:.~ouT_l l Z-3 - Nearest -~ Neighbor \'%. Contour 2 ~= Contour I-2-D-II ----JNearest~ Neighbor ~ Planar - - A Contour 2--Contour 1 ~
17.0
o~
~ 16.0 ~15.0 Planar
[Z-D-n-- A
20INei :: \ Nearest__%
]Contour 1~' ~ 11' O~contour 2 ~ iC°nt°ur i/~ 1°'°rI~..... t J I
- -
Non
Leaky
-
Nearest
Noi g h b o ~ Planar
1-/-/ Contour .2-4l Nearest__/
Contour
Neighbor
.
. . . .
Contour 1"-~
Conto~2---~. 2_D_II~ / ~ Contour 2~_ Contour i--" 2-D-ll-Contour 2~F Nearest_~/ Nearest / Neighbor / Neighbor
14. O F
I
>,
Test Picture
leaky predictors p~=~trcfors
Fig. 9. Comparison between nonleaky predictors(a = 1,/3 = l) and leaky predictors (~ =~,/3 7 =~) ~ on the basis of the standard deviation of the prediction error, ,r~. :nonleaky predictors; . . . . . :leaky predictors.
70
65
2-D-II Planar
Contour 1__
iNearest
Neighbor- I Contour I~
~ ~o E 55
Z-3-
Neighbor--'~ Contour 2-Planar Z-3.~
-j5
%®
.~
Contour I ~
8o
1-~Contour 2-J
Contour
2-D-II ~
Neighbor-2-D-II Nearest
Contour I--~
Contour 2 ~ Nearest .... 40 Neighbor---~ Contour I--/ 35 Contour 2
/If
Neighbo{- / Planar . 9 ~ Contour 2 /
Z-3 g, 50 Nearest'-e-~ Neighbor--~ 45
Nearest
Nearest Neighbor
l--P/Contour 2-J Contour
Contour It
% 1 Nelghbor'\~[ Contour 2"-" Nearest
C
A ~est
D
Picture
Fig. 10. Comparison between nonleaky predictors (a = 1,/3 = -_,) on the basis of peak I) and leaky predictors (a =7, /3-2 differential value e0eak. In Fig. 11, the error signal entropies are compared between the leaky and nonleaky predictors. In general, the leaky predictors produce a larger entropy than the nonleaky predictors. With the Vol. 10, No. 2, March 1986
180
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive T V coders
Non-leak ---- --Leak 6.o 5.9
5.8 5.7 m
z 5.6
g ~5.5 Nearest
g s . , ,Neighbor 1 Contour 1-~ ..~ ~ 5.3 o
Nearest Neighb~r-]. Contour I~ Contour IJF Contour i-% N e a r e s t Contour 2J Nelghbo¥- -. Contour 2~ Contour 2V" N e a r e s t _ 2 | Contour ~ Neighbor | l Contour 1"]_~ Nearest___J Neighbor~l Contour 2J I Contour2-'] Nearest Neighbor--
Contour2 J
5.2
~o 5.1
o5.0 Contour I~I Contour 2 ~
Neighbo~ Contour~
g.9 4.8
A
B Test Picture
Fig. 11. Comparison between nonleaky predictors (a = 1,/3 = 1) and leaky predictors (a = ~,/3 = t) on the basis of the prediction error entropy H. exception of test frame D, predictors 1 and 2 exhibit a smaller trend. Although the leaky predictors yield larger entropy for three of the four cases, this margin is basically gained by increasing the spread of the smaller prediction errors (Fig. 12). The overall effect of lower epeak and (]re can be examined by plotting the cumulative distribution of the prediction error magnitudes. Fig. 13 shows the cumulative occurrence frequencies for the nonleaky and leaky predictors, for picture C where F(le]) is the occurrence frequency of the predictor error within the interval ( - e , e). For other test pictures, the results are similar. It can be observed from this figure that the leaky predictor has the advantage at medium and high prediction errors, but the nonleaky predictors perform better in reducing the number of smaller errors. For example, consider the cumulative occurrence frequencies for ]e I = 30. The difference in F(rel) for the leaky and nonleaky predictors is 1.58%. This means that when the leak function Signal Processing
with a =~7 and/3 =~l is used, on the average, 4053 more prediction errors per frame will have values below 30 rather than above 30. Note that the leak factors ~ and/3 play an important role. As c~ and /3 become smaller, the predictor becomes less efficient resulting in larger granular noise. If they continue to become smaller, the granular noise will be visible. A good choice for a and /3 is a - - ~7, 1 /3 = ~. For ct < 7 and fl < ½, the improved medium and high prediction errors cannot offset the increase in granular noise. In summary, the leaky predictors are more complex than the nonleaky predictors, because the former introduce the leak factor. From the test data, it can be seen that (a) the leaky predictors are more effective in reducing both large and medium prediction errors, (b) the propagation error is significantly reduced without increasing the transmission rates, and (c) the spread of the smaller prediction errors is increased (see Fig. 12).
6. C o n c l u s i o n s
Using the different predictors, both fixed and adaptive, and using the concept of providing leaks in both the predictor output and the predictor function, the propagation of channel errors is investigated by computer simulation. We may conclude the following. (1) The channel error propagation through the picture depends upon (a) the activity of the local region, (b) the magnitude of the error and whether or not the error is terminated is determined by the thresholds Hth, Vth, and SL, and (c) the type of predictors. In uniform areas, the propagation pattern is similar to that of the Z -3 predictor. In highly active areas, the error performance of the adaptive predictor is as bad as the fixed predictors and worse than the 2-D-II. Error propagation in the vertical direction is terminated if a horizontal boundary or uniform region is met. Horizontal error propagation is terminated for contour predictor if the error falls within a contour transition and is small enough so that the wrong contour is not met.
181
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders
Occurrence Frequency
(xlO 3) 20 Picture
C
Predictor •
Non-leak
o
Leak
1
10 0
00 •
O 0
O
O 0
0
0
5
0 0
0
O
• @
0
0 ~e
!
-30
o °
J
-e
-20
110
- 1, 0 Prediction
0
|
20
e
30
Error
Fig. 12. C o m p a r i s o n b e t w e e n n o n l e a k y predictors ( ~ = 1,/3 = 1) a n d leaky p r e d i c t o r s ( a = 7 , / 3 =½) on the basis of the p r e d i c t i o n error o c c u r r e n c e f r e q u e n c y f(e).
(2) By introducing a leak in the predictors, the channel error is significantly reduced and quickly attenuated or terminated. By quantitative tests, the performance is compared between leaky and nonleaky predictors. The leaky predictor has the following characteristics: (a) dramatically reduces the error propagation, (b) the mean square error is reduced, (c) the epeak is significantly decreased, and (d) in uniform regions, the quantizing noise is increased, i.e., larger granular noise. For a reasonable leak, this should not be visual: (e) in moderate areas, the quantizing noise is somewhat reduced, and (f) in highly active regions, the quantizing noise is reduced resulting in improved edgebusiness. (3) It is more important to decrease the number of large prediction errors than it is to decrease the
spread of the smaller prediction errors, which are quantized with high precision. Based on this study, the following recommendations are made. - T h o u g h the leak function can reduce a n d / o r terminate the error propagation, it is not enough to obtain the high quality picture in a noisy channel (see Fig. 8(c)). Error protection by forward error correction is therefore necessary, even though this will increase the transmission rate. - The error-correcting code should be capable of overcoming both random and burst errors. It may be worthwhile to investigate the effects of other schemes such as hybrid D-PCM [28] and DDC [30] in reducing the channel error sensitivity of the proposed contour predictors. Vol. 10, No. 2, March 1986
182
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders
lOO. ooo °
o~. it t Jt ~t It It
t
~95
i o
}85
Predictor
x a=l, [J.t
X0
i
•
.o2
t
~'-r/8,
~=J
o o¢=7/e,(~=J/2 X
,O
x
i
o
?o.
6g.
X e
60
o '
'
' 30
' 40
J 50
lOAir,plitud2Oof ±h~ Prediction Error, lel
' 60
70
Fig. 13. Cumulative occurrence frequency of the prediction error amplitude for the nonleaky and leaky predictors (picture C).
Acknowledgment The authors sincerely appreciate the reviewer's comments which have resulted in a substantial improvement of this paper.
References [1] C.K. Un and H.S. Lee, "A study of the comparative performance of adaptive delta modulation systems", IEEE Trans. Commun., Vol. COM-28, January 1980, pp. 96-101. [2] C.C. Cutler, "Differential quantization of communication signals", U.S. Patent 2605361, July 29, 1952. [3] J.B. O'Neal, "Predictive quantizing systems (DPCM) for the transmission of television signals", BSTJ, Vol. 45, May/June 1966, pp. 689-721. [4] J.P. Rossi, "Sub-Nyquist-encoded PCM NTSC color television", SMPTE J., Vol. 85, January 1976, pp. 1-6. [5] H. Kaneko and T. Ishiguro, "Digital television transmission using bandwidth compression technique", IEEE Trans. Commun., Vol. COM-28, July 1980, pp. 14-22. [6] K. Sawada and H. Kotera, "A 32 Mbit/s component separation DPCM coding system for NTSC color TV", IEEE Trans. Commun., Vol. COM-26, April 1978, pp. 458465. Signal Processing
[7] K. Sawada and H. Kotera, "A 32 Mbit/s transmission of NTSC color TV signal by composite DPCM coding", IEEE Trans. Commun., Vol. COM-26, October 1978, pp. 1432-1439. [8] R. Burkhardt and J. Wasser, "Digital television transmission with 34 Mbit/s", SMPTE 1, Vol. 89, April 1980, pp. 244-248. [9] J.E. Thompson, "Differential encoding of composite color television signals using chrominance-corrected prediction", IEEE Trans. Commun., Vol. COM-22, August 1974, pp. 1106-1113. [10] I. Dinstein, "DPCM prediction for NTSC composite signals", Comsat Tech. Review, Vol. 7, Fall 1977, pp. 429-446. [11] N.R. Cox, "An adaptive encoder for NTSC composite video", National Telecommun. Conf., New Orleans, LA, pp. C9.4-1-C9.4-5, November 29-December 3, 1981. [12] W. Zschunke, "DPCM picture coding with adaptive prediction", IEEE Trans. Commun., Vol. COM-25, November 1977, pp. 1295-1302. [13] P. Cohen and J.P. Adoul, "Adaptive differential coding of picture signals based on local contour prediction", 1976 Nat. Telecommun. Conf. Rec., Dallas, TX, 1976, pp. 6.1-16.1-5. [14] P. Camana, "Video bandwidth compression: A study in tradeoff", IEEE Spectrum, Vol. 16, June 1979, pp. 24-30. [15] J.O. Limb, C.B. Rubinstein and J.E. Thompson, "Digital coding of color video signals: A review", IEEE Trans. Commun., Vol. COM-25, November 1977, pp. 1349-1385. [16] A.K. Jain, "Image data compression: A review", Proc. IEEE, Vol. 69, March 1981, pp. 349-389. [17] N.R. Cox, "An adaptive DPCM algorithm for predicting contours in NTSC composite video signals", Conf. on Pattern Recognition and Image Processing, Dallas, TX, August 1981, pp. 240-247. [18] N.F. Maxemchuck and J.A. Stuller, "Reduction of transmission error propagation in adaptively predicted, DPCM encoded pictures", BSTJ, Vol. 58, July 1979, pp. 14131423. [19] J.E. Thompson, "Differential coding for digital transmission of PAL color television signal", IEEE Conf. Publ. 88, pp. 26-32; presented at: lnternat. Broadcasting Conf., London, England, September 1972. [20] J. Max, "Quantizing for minimum distortion", IRE Trans. Inform. Theory, Vol. IT-6, March 1960, pp. 7-12. [21] D.K. Sharma and A.N. Netravali, "Design of quantizers for DPCM coding of picture signals", IEEE Trans. Commun., Vol. COM-25, November 1977, pp. 1267-1274. [22] E.G. Bowen and J.O. Limb, "Subjective effects of substituting lines in a video-telephone signal", IEEE Trans. Commun., Vol. COM-24, October 1976, pp. 1208-1211. [23] R.W. Hamming, "Error detecting and error correcting codes", BSTJ, Vol. 29, April 1950, pp. 147-160. [24] W.W. Peterson, Error-Correcting codes, M.I.T. Press, Cambridge, MA, and Wiley, New York. [25] R.C. Bose and D.K. Roy-Chaudhuri, "On a class of errorcorrecting binary group codes", Inform. and Control, Vol. 3, September 1960, pp. 279-290. [26] A. Hocquenghem, "Codes correcteurs d'erreurs", Chiffres, Vol. 2, 1959, pp. 147-156.
C.J. Hwang, K.R. Rao / Channel error propagation in adaptive TV coders [27] S. Lin, An Introduction to Error-Correcting Codes, PrenticeHall, Englewood Cliffs, N J, 1970. [28] M.C.W. van Buul, "Hybrid D-PCM, a combination of PCM and DPCM", IEEE Trans. Commun., Vol. COM-28, March 1978, pp. 362-368. [29] R. Steel, D.J. Goodman and C.A. McGonegal, "A difference detection and correction scheme for combating DPCM transmission errors", IEEE Trans. Commun., Vol. COM-27, January 1979, pp. 252-255. [30] H.J. Matt and J.L. Massey, "'Determining the burstcorrecting limit of cyclic coder", IEEE Trans. Inform. Theory, Vol. IT-26, May 1980, pp. 289-297.
183
[31 ] A.N. Netravali and J.O. Limb, "Picture coding: A review", Proc. IEEE, Vol. 68, November 1980, pp. 366-406. [32] V. Devarajan and K.R. Rao, "DPCM coders with adaptive prediction for NTSC composite TV signals", IEEE Trans. Commun., Vol. COM-28, July 1980, pp. 1079-1084. [33] V. Devarajan and K.R. Rao, "Channel error propagation in predictor adaptive differential pulse code modulation (DPCM) coders", Proc. SPIE, Advances in Image Transmission II, Vol. 249, San Diego, CA, July 31-August 1, 1980, pp. 28-33.
Vol. 10, No. 2, March1986