- Email: [email protected]
A video coding method considering cell losses in ATM-based networks
Signal Processing:Image Communication 3 (1991) 291 300 Elsevier
291
A video coding method considering cell losses in ATM-based networks Hideyoshi T o m i n a g a , Hirohisa Jozawa, Masahisa K a w a s h i m a and Tsuyoshi H a n a m u r a School of Science and Engineering, Waseda University, 3-4-10hkubo Shinjuku-ku, Tokyo, 169, Japan
Abstract. This paper describes a video coding method which is capable of compensating for information losses due to cell discard. The proposed method separates picture elements into two groups by using quincunx subsampling. Separated picture elements are coded severally. Picture elements in one group are called higher priority pixels and their coded data are transmitted in higher priority cells, while ones in the other are called lower priority pixels and their coded data are transmitted in lower priority cells. In case of network congestion, the proposed method prevents fatal degeneration in image quality by selectively discarding lower priority cells. Furthermore we take into consideration that higher priority cells are not always free from discard. As a countermeasure against losses of higher priority cells, we examine two schemes, one of which is called intrablock prediction and the other is cyclic refreshing. Experimental results show that the proposed method effectivelysuppresses degradation in image quality compared with conventional coding methods.
Keywords. Asynchronous transfer mode, video coding. 1. Introduction A n important feature o f b r o a d b a n d - I S D N is that networks for different services, i.e. voice, image, data f i l e , . . . , are all integrated into one. A T M ( a s y n c h r o n o u s transfer mode) equally deals with any types o f data in the f o r m o f short fixed length packets called cells. Since users are provided with variable rate interface in A T M networks, we can transmit data o f high burstiness more efficiently. M o r e o v e r statistical multiplexing based on packet switching produces higher utilization o f channel capacity. But there is a d r a w b a c k in A T M networks that we have possibility o f information losses due to discard o f cells. As to video transmission in an A T M network, we do not need the rate control with buffering that is hindrance to image transmission o f constant quality. Therefore improvement o f image quality with variable bit rate (VBR) transmission is expected. But on the other hand, losses o f the coded date due to cell discard will cause large degeneration in image quality. Especially in case 0923-5965/91/$03.50 © 1991 - Elsevier Science Publishers BN.
interframe coding schemes are employed, decoding errors will effect following frames making image quality worse and worse. Hence specialized error concealment measures are inevitable. U n d e r these circumstances, several coding schemes which are capable o f recovery from inform a t i o n losses have been studied. A main principle in such coding schemes is a priority control which presumes that several logical channels with different priorities are offered. These coding schemes are referred to as layered coding. Video data are decomposed into two parts. T w o parts are given priorities from a h u m a n perceptional point o f View. One o f the two contains a lower frequency part o f video signals and is called M S P (most significant part), while the other contains a higher frequency part and is called LSP (least significant part). M S P and LSP are transmitted in a higher priority channel and a lower priority channel, in other words a guaranteed channel and an enhancement channel, respectively. Several technics to decompose video signal have been discussed. One employs the Q M F (quadrature mirror filter) and
292
H. Tominaga et aL / A video coding method
decomposes the video signal in the first stage of encoding. Another decomposes the video signal in the last stage of encoding. When LSP data are lost in a network, we can retain image quality tolerable by simply replacing lost data with zero. However, when MSP data are lost, image quality will degenerate largely. Therefore it is important to take measures against losses of MSP data. In this paper we propose a new coding scheme which employs DPCM. A specialized subsampling technique called quincunx subsampling, which has been successfully used in a field of HDTV video coding, is used to separate video data into higher priority data and lower priority data. As countermeasures against losses of higher priority data, two schemes called intra-block prediction and cyclic refreshing are examined. This paper is organized as follows. In Section 2 we briefly describe the coding method. In Section 3 we describe the algorithm in packetization and transmission. In Section 4 we describe the algorithm in recovery from data losses. In Section 5, we show some experimental results and make remarks. In Section 6 we make the conclusion.
2. The coding method
2.1. Hierarchical structure of coded information according to the loss priority Since ATM handles media with different delay and loss qualities in a unified way, the introduction of service classes sorting them using the transmission delay time, the cell loss rate, etc., as parameters is being considered. Table 1 shows examples of service classes. A system which divides coded data into two categories, higher priority data and lower priority data, depending on the influence they exert on the picture quality, is considered in this paper. The system makes use of the fact that the loss rate can be specified by the service class. In other words, coded data is transmitted by arranging it into two categories of packets: higher priority cells that exert large influence on the Signal Processing:ImageCommunication
Table 1 Examples of service classes in ATM Service class Class 1 Class 2
Class 11 Class 12
Delay quality
Loss quality
Strict Strict Loose
Strict Loose Loose
picture quality and have small cell loss rate and exert small influence on the picture quality and lower priority cells that have small loss rate. With this method, it is possible to realize selective discard of lower priority cells. This minimizes the picture quality deterioration due to cell loss when the network is congested.
2.2. Sub-sampling of quincunx Grouping of coded data into higher priority data and lower priority data by the influence it exerts on the picture quality is required to use the service classes mentioned in Section 2.1. In this paper, the pixels are separated in two groups, G1 and G2, by quincunx sampling, and prediction coding is applied to each group. Since G1 and G2 generate different coded data, it is possible to transmit them as independent packets, with one group as higher priority data and the other group as lower priority data. The criteria to classify these groups G1 and G2 as higher priority and lower priority data is explained in Section 2.4 along with the properties of the error propagation accompanying cell loss.
2.3. Interframe / &traframe alternative interpolative and extrapolative prediction Interframe/intraframe alternative prediction is the most effective prediction coding algorithm for compression efficiency. When this prediction method is used as an algorithm for this video packet coding, completely independent prediction coding of G1 and G2 (with one group not referring to the pictures of the other group at all) is possible. However, this causes the intraframe prediction efficiency to decrease due to quincunx sub-
H. Tominaga et al. / A video coding method Previous frame,, x
0 0 0 0 @ 0 0 o • 0 @ O @ 0 @ 0 • o ® o o o e 0 @ ; o ; " o ;... '~'~o t~'~ .." • 0 • 0 - ~ * ~ 0 • c~¥A~ ...0 0 @ 0 • 0 • ;tiP® d " ~ # @ @ 0 0 @ o o o o
I
y C u rrent
frame @ ~ ..-"~"0
,~ Interpolation @ 0 @ 0 @ 0 0 0 0 @ 0 @ 0 •
293 0 0 0 0 0 @@ 0~ 0 0 1 @ 0 @ 0 • 0 @ 0 @ 0.."@ 0 @ 0
Previous
frame o
Current • frame 0
oo
0 " @ o 0 •
• 0 @ 0
e
o
e..6
•
0 " o •
® C~~ @ 0
o
e
®
@ OlO 0 e ~ ° @l • o @ 0 @o I 0 • 0 @ 0 •I
/ O 0 @ 0 @ 0 • O @ 0 Unreconstructible pixel by cell loss O :G1 pixies O :G2 pixels r~,,~:lnterframe prediction lip :lnterframe interpolative prediction tap :lnterframe exterpolative prediction
Fig. I. Group separation of coding pixels and prediction method for each group.
sampling, and the overall coding efficiency increases as a result. Yashima's 'intraframe/interframe alternative extrapolative and interpolative prediction coding' [6] is employed for this study to minimize the loss of coding efficiency due to subsampling. The prediction method with the smaller error out of interframe prediction (fMF) and intraframe extrapolative prediction (fEp) is alternatively selected for each individual pixel to coding the G1 pixels as shown in Fig. 1. Since coding of G2 pixels is carried out with a delay of 1 or more line times compared to G1 pixels, either interframe prediction (fMv) or intraframe interpolative prediction (j]p) is alternatively selected for each individual pixel.
2.4. Order of priority of coded data As an understanding of the properties of error propagation, accompanying cell loss is required to determine the order of priority of the coded data of G1 and G2. The characteristics of error propagation in interframe/intraframe alternative extrapolative and interpolative prediction coding are shown below. (1) Since either interframe prediction or intraframe extrapolative prediction is used to encode G1 pixels, error caused by cell loss and bit error in a G1 pixel propagates to the lower side of the screen and to the successive frames (Fig. 2).
Fig. 2. Characteristics of error propagation on interframe/ intraframe adaptive DPCM using extrapolative and interpolative prediction
(2) Since the G1 pixel is referred to when coding the G2 pixel (during intraframe interpolative prediction), error in the G1 pixel propagates through to the G2 pixel (Fig. 2). (3) Cell loss and bit error in the G2 picture has the possibility of influencing successive frames, but they do not propagate to other pixels located in the frame. (4) Since G2 pixels are not referred to in G1 pictures coding, G2 pixels errors do not propagate to the G1 pixels. (5) Interframe error propagation stops when interframe prediction is selected, both in G1 and G2. The influence of the reconstruction error of the G2 pixels on the reproduced frame is negligible (because of the characteristics in (3) and (4)). On the other hand, the reconstruction error of the G1 pixels exerts conspicuous influence on the reproduced picture (because of the characteristics in (1) and (2)). It is recommended to treat G1 data as higher priority data and to transmit it through a channel with a low loss rate to minimize loss. In this study, G1 data was arranged in higher priority cells with small loss rate (Class 11) while G2 data was arranged in lower priority cells with large loss rate (Class 12), as shown in Table 2, to minimize picture deterioration caused by cell loss. When the network is congested, it is possible to select and discard G2 data lower priority cells but not to discard G1 data. Moreover, the picture quality can be improved by a large extent, even when there is cell loss, because interpolation of G2 pixels lost due to Vol. 3, No. 4, September1991
294
H. Tominaga et al. / A video coding method
Table 2 Arrangement of coded data in packets Pixel group
Higher priority/Lower priority
Service class
G1 G2
Higher priority information Lower priority data
Class 11 (large loss rate) Class 12 (small loss rate)
.•
Intraframe
,"~"*'~, ~ Predi....ctionlnterf ..~ SoOs'rn. Qui.....no ~i .....
Higher
[CoderI I
G +=~ro~c [ Intef irc~me /l~
I cATMel
priority
AnTMnel
Fig. 3. Block diagram of coding method.
cell loss can be done using the correctly-reproduced G1 pixels. Figure 3 shows the block diagram of this coding system.
3. Packetization
3.1. Cell composition method Figure 4 shows the cell composition method used in this paper. In this system, the cell information field consists of the starting coding pixel address data (synchronous information), the VLC (variable length code) for prediction error and stuffing bits, or dummy bits when the interior of the information field cannot be filled with effective data.
3.2. Arrangement of coded data & packets Since synchronous information is added to each cell, data is divided into cells so that each cell conSignalProcessing:Image Communication
tains as much pixel data as possible to provide that one pixel's data is not divided in two cells. It must be remembered, however, that data from G1 and G2 cannot be included in the same cell, and the separation of code data in packets is carried out simultaneously for G1 and G2. The separation into higher priority/lower priority data is carried out by referring to the cell header service class identifier. Higher/lower rlorlty header
Coordinates of start plxel (x,y)
I
Field of Information Cell ( fixed length )
Fig. 4. Cell structure.
Stuffing bit "~
I I
H, Tominaga et al. / A video coding method
3.3. Synchronous information Since the order of cell arrival is basically guaranteed in the ATM network [6], cell loss detection at the receiving side is possible if the cells are given sequence numbers. Since prediction error data undergo variable-length coding in this coding system, the number of coded pixels contained in each cell is variable. It is not possible to know the quality of missing data from the sequence number alone, and once a cell loss takes place, normal reconstruction of the codes becomes impractical due to the asynchronous operation of the coder and decoder. Besides the variable-length coding data address, the top pixel information data is added to each cell in order to restore the decoding synchronism after cell loss.
4. Measures against cell discards
4.1. Picture quality compensation method when cell loss is detected If cell loss is detected at the receiving side, all prediction errors of the part that was lost due to cell loss are decoded as '0'. Thus, if data of a G2 pixel is lost due to cell loss, and when intraframe interpolative prediction is used, the mean value of the surrounding G1 pixels (interpolation value) luminance is selected. On the other hand, when interframe interpolative prediction is selected, interpolation using the G2 pixel of the previous frame is automatically executed. With this method, deterioration in picture quality is kept so small that even if a lower priority cell is lost it is not optically perceptible.
4.2. Method for picture quafity compensation when higher priority cells are lost In this method, extreme deteriorations in the picture quality accompanying cell loss are suppressed through selective discarding of lower priority cells when the network is congested. It must be remembered, however, that since in actual ATM net-
295
works, when the network is abnormally congested, the class 11 cell transmission cannot be guaranteed, it is impracticable to have a zero loss rate of higher priority cells with this priority control. Such being the case, intrablock independent prediction and cyclic insertion of intraframe coded picture (cyclic refreshing) is introduced as a means of compensating the picture quality when higher priority cells are lost.
4.2.1. Intrablock independent prediction Decoding error propagates both in the intraframe and interframe directions as shown in Fig. 5 when higher priority cells containing G1 data is lost. The picture frame is divided into m x n subblocks, and prediction coding is carried out independently for each block to restrict the propagation of the error within a frame to a limited area. This measure is aimed at preventing decoding errors occurring in a given block from propagating to other blocks by carrying out the coding without referring to pixels outside the block in question. In carrying out interframe/intraframe adaptive extrapolative and interpolative prediction coding in units of sub-blocks, pixels outside the block are referred to only when pixels at the block boundary are coded (decoded). In this case, the propagation of decoding error can be suppressed by either using interframe prediction or regarding the luminance of the reference pixels outside the block as '0' in
Currentframe [ U;r::l//structi s , bloe spart
/
t~Nextframe ~
r
r
o
r
inthepropagation frame
~:rror~pagation in later frames
Fig. 5. Error propagation on losses of higher priority cells (GI data). Vol. 3, No. 4, September 1991
296
H. Tominaga et aL / A video coding method
the intraframe prediction. However, since in interframe/intraframe alternative extrapolative and interpolative prediction coding, the decoded pixels are used not only as a reference luminance value for decoding the subsequent pixels but also for judging the prediction function [6], the decoding error in a given pixel brings about judgement errors in the prediction function handling subsequent pixels. In other words, the prediction function must be fixed at the block boundary in order to suppress the propagation of judgment error of the prediction function in question. To cope with this problem, interframe prediction was forced to be selected in this study when coding pixels at the block boundary. The coding efficiency decreases at the block boundary pixels since intraframe correlation cannot be utilized at the block boundary. Conversely, the picture quality conspicuously is influenced because intraframe error propagation due to cell loss is restricted to the block in question. In experiments, the block size is defined as 8 pixels x 8 lines.
retransmitted data and refreshed data, these methods have the disadvantages of force motion and other problems. This paper explores cyclic refreshing, which is a simple and relatively efficient means for stopping error propagation in the interframe direction. From the standpoint of picture quality it is recommended to send PCM coded data, but it was decided to transmit intraframe coded data to minimize the increase in the volume of codes transmitted as a consequence of the refreshing. In other words, intraframe coded frame is forced to be transmitted at every scene change or once every N frames. The intraframe coded frame is obtained through forced selection of intraframe prediction when coding the G1 pixels and the G2 pixels. In order to realize symbiosis with the intrablock independent prediction mentioned before, it is necessary to fix the prediction system at the block boundary, and intraframe prediction is carried out by assuming luminance '0' for reference pixels located outside the block.
5. Simulation
results
4.2.2. Cyclic refreshing Since this coding system is basically interframe coding, once an error breaks out as a consequence of cell loss it cannot be corrected easily. The following methods are available for correcting decoding error in intraframe coding: (1) demand retransmission, (2) demand refreshing, (3) cyclic refreshing, (4) etc. Although cyclic refreshing is inferior to the other two methods in terms of error correction speed and reliability, the composition of the protocol and the hardware is simple, and it is effective for broadcasting and one-way video transmission. On the other hand, demand retransmission and demand refreshing are more reliable and allow quick error correction, but the protocol and the hardware are complicated. They also require larger numbers of packets and cause more transmission delay. Since the pixel must be frozen until the arrival of the Signal Processing: Image Communication
5.1. Simulation conditions Computational simulation was carried out under the conditions mentioned in Table 3 to confirm the effectiveness of this video packet coding system. The Cronkite test picture is a sequence with relatively few changes corresponding to a TV meeting, and Beach and Flowers is a sequence with rapid changes mostly consisting of panning. Moreover, since the quantization characteristic of the prediction eror is simple, linear quantization is used. In this case, 63 quantization levels are adopted to satisfy the 40 dB minimum reproduced picture quality requirement even when there is no cell loss. The coding systems used for the sake of comparison in this simulation test are: - Method A: Interframe DPCM; - Method B: Interframe/intraframe alternative prediction coding;
H. Tominaga et al. / A video coding method
297
Table 3
2 × 10 3. As can be seen in Fig. 6 the simulation
Simulation conditions
results, the picture quality deteriorates with time in system A and system B, giving evidence that these systems are prone to be adversely influenced by cell losses. Moreover, there is sudden decrease in the SNR after the scene change on the 17th frame. The scene change cannot be followed because cell loss makes normal decoding of the picture impossible. On the other hand, in this coding system (method C-I) there is practically no deterioration in the SNR even after cell loss, the loss is restricted to lower priority cells. With this method, the SNR is far better compared to method B. Picture quality virtually equivalent to that occurring with no cell loss is possible when a zero loss rate of higher priority cells is guaranteed by priority control. This coding system results in a very high picture quality improvement. However, when cells are lost without distinction between higher priority cells and lower priority cells, picture quality deterioration is very conspicuous. In this case the SNR becomes worse than that of method B. This indicates that although the proposed method has considerable advantages, conspicuous picture quality deterioration takes place when there is an unexpected loss of higher priority cells. (This can happen when the network is congested, when the difference in delay time between higher priority cells and lower priority cells exceeds a given limit, etc.) It must be
Input picture
Non-interlace 256 pixels × 256 lines × 8 bit/pixel × 30 Hz Cronkite : 16 frames Beach and Flowers: 31 frames Total 47 frames
Block size Quantization Entropy coding Cell size
8 pixels x 8 lines 63 levels, linear quantization (mid-tread) H a f m a n coding 32 byte fixed length (information field)
- Method C-I: Proposed system, interframe/intraframe alternative extrapolative interpolative prediction coding; - Method C-II: Proposed system + intrablock independent prediction ; - Method C-III: Proposed system + intrablock independent prediction + cyclic refreshing.
5.2. Influence of cell loss on the picture quality Picture quality evaluation in terms of the SNR was carried out for 3 systems: the A system, the B system, and the C-I system to examine the picture quality improvement effect by this coding system on cell loss. As a general rule, studies referring to the A T M assume cell loss rates of the order of 10 - 9 , but a rather high loss rate of 10 3 is assumed in this study to evaluate the effectiveness of the proposed system. The cell loss is assumed to be random. In this study simulation is carried out not only for the case in which only lower priority cells are lost but also for the case that no service class is specified, in other words, higher priority cells and lower priority cells are lost under the same conditions. The ratio of higher priority cells (GI pixel data) and lower priority cells (G2 pixel data) is approximately 1:1. Assuming a lower priority cell loss rate of 10 -3 with higher priority cell loss rate equal to zero, overall cell loss rate would be half. In order to maintain overall cell loss rate 10- 3, the lower priority cell rate is assumed to be
S N R (dB)
45.00 ee,e*
40.00
\
35.00
~eee~
_
_
. - .
"MethodA
DQ
~; ~h'&l"~:r ~f&;-o-f%w-e-6--' --
30.00 --
"[vlet-h~xl-ff-[(To~s-of ~o~h~-
'
25.00
20.00
t~a ~an~aao
15.00
--
10.00
-
~lo.~a
~ o o a
-Cell loss r a t e = le-3 S. . . . .
hange, x::
5.00 0,00
I0.00
20.00
30.00
40.00
50.00
F:ame number(1/30sec) Fig. 6. Quality degradation on cell losses. Vol. 3, No. 4, September 1991
H. Tominaga et al. / A video coding method
298
remembered that this simulation was carried out under unfavourable conditions in which no priority control of higher priority/lower priority cells was carried out at all, and that in actual cases lower priority cells are selectively lost due to the priority control of the cells. Even when method B is supplemented with the cell loss compensation methods such as those in Section 4.2 (intrablock independent prediction, cyclic refreshing) are added to method B, in most cases method C-I is better for picture quality except for extremely unfavourable conditions such as this simulation. Moreover, as will be shown in Section 5.5, method C-I is superior to method B for code volume. It appears that this is the best high-efficiency video coding method for ATM networks, Even in this system remedial measures such as intrablock independent prediction, cyclic refreshing, etc., must be used in order to cope with unexpected higher priority cell losses. 5.3. Picture quality improvement effect of remedial measures to cope with higher priority cell loss To examine the effect of intrablock independent prediction on picture quality the SNR was evaluated. The methods are: method C-I without any remedial measure to cope with higher priority cell loss, method C-II which uses the intrablock independent prediction, and method C-III which uses both intrablock independent prediction and cyclic refreshing. Cells were lost at random (10-3 cell loss rate) and it is assumed that they are lost under the same conditions for higher priority cells and nonpriority cells. Moreover, refreshing is assumed to be carried out once every 10 frames ( N = 10) starting from the initial frame and the scene change frame. Figure 7 shows the results. Since a cell loss's influence is limited to its block because of the independent coding of the various frames in block units, the picture quality is improved by 10-17 dB comparing with the case without intrablock independent prediction (Fig. 7). Moreover, since the picture quality is added, the Signal P r o c e s s i n g :
Image Communication
SNR (dB) -
45.00
-
R'efresh f r a m ei - -
40.00 -35.00
8u~ Iq.l.m~l~e.lkm ,DBOOODnO01
On.ooOo
30.00 -"Method C-I 25.00 20,00 -15.00 - -
Cell loss r a t e = le-3
10.00
Scene change
,/
5.00 - 0.00
10.00
20.130
I
30.00
_
40.00
50.00
Frame number (1/30sec)
Fig. 7. Effectof cell loss compensationfor higherprioritycells. SNR is kept at approximately 40 dB even with some fluctuations. As can be seen, intrablock independent prediction and cyclic refreshing give a noticeable picture quality improvement, and very effectively for compensate for higher priority cell loss. 5.4. Change in the code amount due to higher priority cell loss compensation As mentioned before, intrablock independent prediction and cyclic refreshing are effective remedies for suppressing picture quality deterioration due to higher priority cell loss. It must be remembered, however, that since the correlation between pixels cannot be utilized at the block boundary in intrablock independent prediction, the coding efficiency decreases. Moreover, at the moment of the refreshing in cyclic refreshing, correlation with the previous frame cannot be utilized and causes a decrease in the coding efficiency. The number of cells generated for the various frames is compared for three methods, method B, method C-I and method C-II to examine the increase in the code volume due to intrablock independent prediction. Figure 8 shows the results. Compared to the case without independent prediction (method C-I) the number of cells generated with intrablock independent prediction (method
299
H. Tominaga et al. / A video coding method
C-II) increases by approximately 30 cells (8 kbit) per frame under ordinary conditions and by approximately 300 cells (75 kbit) under the peak conditions of a scene change. Even with that increase, method C-II yields higher coding efficiency than interframe/intraframe alternative prediction. When the benefits are considered, the increase in code volume due to intrablock independent prediction is within acceptable limits. Next, the number of cells generated in each frame by method C-II and method C-III were compared in order to examine the variation in the code volume due to cyclic refreshing. Figure 9 shows the results. As can be seen in Fig. 9, the increase in the number of cells due to cyclic refreshing is of the order of 20 cells (5 kbit) per frame compared to intrablock independent prediction. This means an increase of the order of 50 cells (12 kbit) compared to the case without higher priority cell loss compensation (method C-I). Considering the picture quality improvement effect, this is an acceptable increase. Moreover, the refreshed screen generates less cells compared to the alternative prediction method during the scene change on the 17th frame. This is presumed to occur because the prediction method's optimum selection is not carried out during the scene change because the alternative Number of cell x 103
I
1.10 1.00 ---
~V{e'{l~odC-I
~et-h-&J- C:- f["
0.90
I
Length of cell = 32 Byte(fixed)
0.80 -0.70 0.60 -0.50 -- ~ 0.40
--
~m' m
"
Refresh frame
/I
0.30 - 0.00
10.00
20.00
I 30.00
40.00 50.00 Frame number (1]30sec)
Fig. 8. Generated cell number on adding block-independent prediction.
Number of cell x 103 1.10 -1.00 Method C-II
0.90 0.80
~et'[{~i C - I l l Refresh frame
0.70 -0.60 0.50
. ~
Length of cell = 3 2 Byte(fixed)
0.40 Scene change
0.30
/I 0.00
10.00
20.00
30.00
40.00 50.00 Frame number (1/30sec)
Fig. 9. Generated cell number on adding cyclic refresh. prediction method does not transmit the prediction selection information. 5.5. The S N R mean value and worst value comparison
The picture quality was compared in terms of the mean SNR and the worst SNR for cases with 3 cell loss rates, 10 -4, 10 -3 and 10 -2, to evaluate the effectiveness of this method using remedial measures compensate for higher priority cell loss. Figures 10 and 11 show the obtained results. Four cases: method A, method B, this coding method with loss of only lower priority cells, and this coding method with equally likely loss of cells higher priority and lower priority cells, are compared here. It should be noted that the apparent cell loss rate is doubled as mentioned in Section 5.2 when only lower priority cells are lost, to even the overall average cell loss rate compared to the case of equally likely cell loss higher priority and lower priority cell loss. As can be seen from Figs. 10 and 11, both the mean value and the worst value of the SNR of method A are low irrespective of the cell loss rate, on the other hand the SNR of method B decreases rapidly with the increase of the cell loss rate. When only lower priority cells are lost in this coding method (method C-III), the SNR is kept at approximately 40 dB irrespective of the cell loss Vol. 3, No. 4, September 1991
300
H. Tominaga et al. / A video coding method S N R (dB) 45.00 40.00
-
'
~....
35.00
-
: 21
-
. . . . . ""
"..% 30.00 --
am'k'eff+J wt A
"% ......."-.,
25.00 20.00 - -
"f,;fgiffi~'g ~ t-fro)"C'-f[i "([~s's'o( fffw~ ",.,.. '~,12t-h~l-(~fIl-([osso~ l~oih- -
15.00 - -
"......... • -.......
hQ~
10.00
..........
•
",..,,.,
5.00-le-04
3
le-03
3
le-02 Cell loss rate
Fig. 10. Mean SNR.
rate, and moreover the decrease in SNR is small compared to other methods even when higher priority cells are lost. This coding method combined with cope with higher priority cell loss compensation is a high efficiency video packet coding method with resistance to cell loss in ATM networks.
networks. In this method, picture elements are separated into two groups, which are called G1 pixels and G2 pixels. The data for G1 pixels are transmitted in higher priority cells, while the data for G2 pixels are transmitted in lower priority cells. In case of network congestion, lower priority cells are selectively discarded. The receiver can retain image quality by simply replacing lost data with zero. Furthermore we examined cyclic refreshing and intrablock prediction as measures against discards of higher priority cells. Simulation results show that we can expect 10-17 dB increase in average SNR by employing intrablock prediction, 6-10 dB increase by inserting intraframe-coded frame periodically. In experiments, the average SNR of the decoded image remains approximately 40 dB. We note that increases in total code amount by employing measures against discards of higher priority cells are tolerable considering their effect. So we can conclude that this method is practical as video coding schemes in ATM networks.
References 6. Conclusion In this paper, we examined interframe/intraframe alternative prediction coding using quincunx subsampling as a video coding method for ATM SNR (riB) 45.00 - - - -
~';,.,......,.,,
40.00-
35.00 -
"Method A
".,.. ".....~
30.00 -
'll;¢lethod B
M~$-o~-~c'[Ja-(Lossof k ........ '~vl'c?h~l-(~-fiI-(loss of b
"'"'"..,....,,....
25.00
".......
20.00 -
m
15.00 10.00 -
m
5.00 le-04
3
le-03
3
1 e-02 Cell loss rate
Fig. 11. Worst SNR. Signal Processing: Image Communication
[1] R. Kishimoto, N. Norihiro and K. Irie, "Packet loss compensation for HDTV signals in an A T M network", IEICE Tech. Rep., IE88-88, December 1988 (in Japanese). [2] Maglaris et al., "Performance model of statistical multiplexing in packet video communications", 1EEE Trans. Commun., Vol. 36, No. 7, July 1988. [3] M. Nomura, T. Fujii and N. Ohta, "Basic characteristics of variable rate video coding in ATM environment", IEEE JSAC, Vol. 7, No. 5, June 1989. [4] M. Ohta, H. Suzuki and T. Ohmachi, "Error recovery procedures for packetized inter frame video coding", Proc. 2nd Internat. Workshop on Pocket Video, B6, September 1988. [5] Verbiest et al., "The impact of the ATM concept on video coding, IEEE JSAC, Vol. 6, No. 9, December 1988. 16] Y. Yashima and K. Sawada, "A highly efficient coding method for HDTV signals", Proc. International Conference on Communication, Seattle, USA, June 1987, paper 5.6.
291
A video coding method considering cell losses in ATM-based networks Hideyoshi T o m i n a g a , Hirohisa Jozawa, Masahisa K a w a s h i m a and Tsuyoshi H a n a m u r a School of Science and Engineering, Waseda University, 3-4-10hkubo Shinjuku-ku, Tokyo, 169, Japan
Abstract. This paper describes a video coding method which is capable of compensating for information losses due to cell discard. The proposed method separates picture elements into two groups by using quincunx subsampling. Separated picture elements are coded severally. Picture elements in one group are called higher priority pixels and their coded data are transmitted in higher priority cells, while ones in the other are called lower priority pixels and their coded data are transmitted in lower priority cells. In case of network congestion, the proposed method prevents fatal degeneration in image quality by selectively discarding lower priority cells. Furthermore we take into consideration that higher priority cells are not always free from discard. As a countermeasure against losses of higher priority cells, we examine two schemes, one of which is called intrablock prediction and the other is cyclic refreshing. Experimental results show that the proposed method effectivelysuppresses degradation in image quality compared with conventional coding methods.
Keywords. Asynchronous transfer mode, video coding. 1. Introduction A n important feature o f b r o a d b a n d - I S D N is that networks for different services, i.e. voice, image, data f i l e , . . . , are all integrated into one. A T M ( a s y n c h r o n o u s transfer mode) equally deals with any types o f data in the f o r m o f short fixed length packets called cells. Since users are provided with variable rate interface in A T M networks, we can transmit data o f high burstiness more efficiently. M o r e o v e r statistical multiplexing based on packet switching produces higher utilization o f channel capacity. But there is a d r a w b a c k in A T M networks that we have possibility o f information losses due to discard o f cells. As to video transmission in an A T M network, we do not need the rate control with buffering that is hindrance to image transmission o f constant quality. Therefore improvement o f image quality with variable bit rate (VBR) transmission is expected. But on the other hand, losses o f the coded date due to cell discard will cause large degeneration in image quality. Especially in case 0923-5965/91/$03.50 © 1991 - Elsevier Science Publishers BN.
interframe coding schemes are employed, decoding errors will effect following frames making image quality worse and worse. Hence specialized error concealment measures are inevitable. U n d e r these circumstances, several coding schemes which are capable o f recovery from inform a t i o n losses have been studied. A main principle in such coding schemes is a priority control which presumes that several logical channels with different priorities are offered. These coding schemes are referred to as layered coding. Video data are decomposed into two parts. T w o parts are given priorities from a h u m a n perceptional point o f View. One o f the two contains a lower frequency part o f video signals and is called M S P (most significant part), while the other contains a higher frequency part and is called LSP (least significant part). M S P and LSP are transmitted in a higher priority channel and a lower priority channel, in other words a guaranteed channel and an enhancement channel, respectively. Several technics to decompose video signal have been discussed. One employs the Q M F (quadrature mirror filter) and
292
H. Tominaga et aL / A video coding method
decomposes the video signal in the first stage of encoding. Another decomposes the video signal in the last stage of encoding. When LSP data are lost in a network, we can retain image quality tolerable by simply replacing lost data with zero. However, when MSP data are lost, image quality will degenerate largely. Therefore it is important to take measures against losses of MSP data. In this paper we propose a new coding scheme which employs DPCM. A specialized subsampling technique called quincunx subsampling, which has been successfully used in a field of HDTV video coding, is used to separate video data into higher priority data and lower priority data. As countermeasures against losses of higher priority data, two schemes called intra-block prediction and cyclic refreshing are examined. This paper is organized as follows. In Section 2 we briefly describe the coding method. In Section 3 we describe the algorithm in packetization and transmission. In Section 4 we describe the algorithm in recovery from data losses. In Section 5, we show some experimental results and make remarks. In Section 6 we make the conclusion.
2. The coding method
2.1. Hierarchical structure of coded information according to the loss priority Since ATM handles media with different delay and loss qualities in a unified way, the introduction of service classes sorting them using the transmission delay time, the cell loss rate, etc., as parameters is being considered. Table 1 shows examples of service classes. A system which divides coded data into two categories, higher priority data and lower priority data, depending on the influence they exert on the picture quality, is considered in this paper. The system makes use of the fact that the loss rate can be specified by the service class. In other words, coded data is transmitted by arranging it into two categories of packets: higher priority cells that exert large influence on the Signal Processing:ImageCommunication
Table 1 Examples of service classes in ATM Service class Class 1 Class 2
Class 11 Class 12
Delay quality
Loss quality
Strict Strict Loose
Strict Loose Loose
picture quality and have small cell loss rate and exert small influence on the picture quality and lower priority cells that have small loss rate. With this method, it is possible to realize selective discard of lower priority cells. This minimizes the picture quality deterioration due to cell loss when the network is congested.
2.2. Sub-sampling of quincunx Grouping of coded data into higher priority data and lower priority data by the influence it exerts on the picture quality is required to use the service classes mentioned in Section 2.1. In this paper, the pixels are separated in two groups, G1 and G2, by quincunx sampling, and prediction coding is applied to each group. Since G1 and G2 generate different coded data, it is possible to transmit them as independent packets, with one group as higher priority data and the other group as lower priority data. The criteria to classify these groups G1 and G2 as higher priority and lower priority data is explained in Section 2.4 along with the properties of the error propagation accompanying cell loss.
2.3. Interframe / &traframe alternative interpolative and extrapolative prediction Interframe/intraframe alternative prediction is the most effective prediction coding algorithm for compression efficiency. When this prediction method is used as an algorithm for this video packet coding, completely independent prediction coding of G1 and G2 (with one group not referring to the pictures of the other group at all) is possible. However, this causes the intraframe prediction efficiency to decrease due to quincunx sub-
H. Tominaga et al. / A video coding method Previous frame,, x
0 0 0 0 @ 0 0 o • 0 @ O @ 0 @ 0 • o ® o o o e 0 @ ; o ; " o ;... '~'~o t~'~ .." • 0 • 0 - ~ * ~ 0 • c~¥A~ ...0 0 @ 0 • 0 • ;tiP® d " ~ # @ @ 0 0 @ o o o o
I
y C u rrent
frame @ ~ ..-"~"0
,~ Interpolation @ 0 @ 0 @ 0 0 0 0 @ 0 @ 0 •
293 0 0 0 0 0 @@ 0~ 0 0 1 @ 0 @ 0 • 0 @ 0 @ 0.."@ 0 @ 0
Previous
frame o
Current • frame 0
oo
0 " @ o 0 •
• 0 @ 0
e
o
e..6
•
0 " o •
® C~~ @ 0
o
e
®
@ OlO 0 e ~ ° @l • o @ 0 @o I 0 • 0 @ 0 •I
/ O 0 @ 0 @ 0 • O @ 0 Unreconstructible pixel by cell loss O :G1 pixies O :G2 pixels r~,,~:lnterframe prediction lip :lnterframe interpolative prediction tap :lnterframe exterpolative prediction
Fig. I. Group separation of coding pixels and prediction method for each group.
sampling, and the overall coding efficiency increases as a result. Yashima's 'intraframe/interframe alternative extrapolative and interpolative prediction coding' [6] is employed for this study to minimize the loss of coding efficiency due to subsampling. The prediction method with the smaller error out of interframe prediction (fMF) and intraframe extrapolative prediction (fEp) is alternatively selected for each individual pixel to coding the G1 pixels as shown in Fig. 1. Since coding of G2 pixels is carried out with a delay of 1 or more line times compared to G1 pixels, either interframe prediction (fMv) or intraframe interpolative prediction (j]p) is alternatively selected for each individual pixel.
2.4. Order of priority of coded data As an understanding of the properties of error propagation, accompanying cell loss is required to determine the order of priority of the coded data of G1 and G2. The characteristics of error propagation in interframe/intraframe alternative extrapolative and interpolative prediction coding are shown below. (1) Since either interframe prediction or intraframe extrapolative prediction is used to encode G1 pixels, error caused by cell loss and bit error in a G1 pixel propagates to the lower side of the screen and to the successive frames (Fig. 2).
Fig. 2. Characteristics of error propagation on interframe/ intraframe adaptive DPCM using extrapolative and interpolative prediction
(2) Since the G1 pixel is referred to when coding the G2 pixel (during intraframe interpolative prediction), error in the G1 pixel propagates through to the G2 pixel (Fig. 2). (3) Cell loss and bit error in the G2 picture has the possibility of influencing successive frames, but they do not propagate to other pixels located in the frame. (4) Since G2 pixels are not referred to in G1 pictures coding, G2 pixels errors do not propagate to the G1 pixels. (5) Interframe error propagation stops when interframe prediction is selected, both in G1 and G2. The influence of the reconstruction error of the G2 pixels on the reproduced frame is negligible (because of the characteristics in (3) and (4)). On the other hand, the reconstruction error of the G1 pixels exerts conspicuous influence on the reproduced picture (because of the characteristics in (1) and (2)). It is recommended to treat G1 data as higher priority data and to transmit it through a channel with a low loss rate to minimize loss. In this study, G1 data was arranged in higher priority cells with small loss rate (Class 11) while G2 data was arranged in lower priority cells with large loss rate (Class 12), as shown in Table 2, to minimize picture deterioration caused by cell loss. When the network is congested, it is possible to select and discard G2 data lower priority cells but not to discard G1 data. Moreover, the picture quality can be improved by a large extent, even when there is cell loss, because interpolation of G2 pixels lost due to Vol. 3, No. 4, September1991
294
H. Tominaga et al. / A video coding method
Table 2 Arrangement of coded data in packets Pixel group
Higher priority/Lower priority
Service class
G1 G2
Higher priority information Lower priority data
Class 11 (large loss rate) Class 12 (small loss rate)
.•
Intraframe
,"~"*'~, ~ Predi....ctionlnterf ..~ SoOs'rn. Qui.....no ~i .....
Higher
[CoderI I
G +=~ro~c [ Intef irc~me /l~
I cATMel
priority
AnTMnel
Fig. 3. Block diagram of coding method.
cell loss can be done using the correctly-reproduced G1 pixels. Figure 3 shows the block diagram of this coding system.
3. Packetization
3.1. Cell composition method Figure 4 shows the cell composition method used in this paper. In this system, the cell information field consists of the starting coding pixel address data (synchronous information), the VLC (variable length code) for prediction error and stuffing bits, or dummy bits when the interior of the information field cannot be filled with effective data.
3.2. Arrangement of coded data & packets Since synchronous information is added to each cell, data is divided into cells so that each cell conSignalProcessing:Image Communication
tains as much pixel data as possible to provide that one pixel's data is not divided in two cells. It must be remembered, however, that data from G1 and G2 cannot be included in the same cell, and the separation of code data in packets is carried out simultaneously for G1 and G2. The separation into higher priority/lower priority data is carried out by referring to the cell header service class identifier. Higher/lower rlorlty header
Coordinates of start plxel (x,y)
I
Field of Information Cell ( fixed length )
Fig. 4. Cell structure.
Stuffing bit "~
I I
H, Tominaga et al. / A video coding method
3.3. Synchronous information Since the order of cell arrival is basically guaranteed in the ATM network [6], cell loss detection at the receiving side is possible if the cells are given sequence numbers. Since prediction error data undergo variable-length coding in this coding system, the number of coded pixels contained in each cell is variable. It is not possible to know the quality of missing data from the sequence number alone, and once a cell loss takes place, normal reconstruction of the codes becomes impractical due to the asynchronous operation of the coder and decoder. Besides the variable-length coding data address, the top pixel information data is added to each cell in order to restore the decoding synchronism after cell loss.
4. Measures against cell discards
4.1. Picture quality compensation method when cell loss is detected If cell loss is detected at the receiving side, all prediction errors of the part that was lost due to cell loss are decoded as '0'. Thus, if data of a G2 pixel is lost due to cell loss, and when intraframe interpolative prediction is used, the mean value of the surrounding G1 pixels (interpolation value) luminance is selected. On the other hand, when interframe interpolative prediction is selected, interpolation using the G2 pixel of the previous frame is automatically executed. With this method, deterioration in picture quality is kept so small that even if a lower priority cell is lost it is not optically perceptible.
4.2. Method for picture quafity compensation when higher priority cells are lost In this method, extreme deteriorations in the picture quality accompanying cell loss are suppressed through selective discarding of lower priority cells when the network is congested. It must be remembered, however, that since in actual ATM net-
295
works, when the network is abnormally congested, the class 11 cell transmission cannot be guaranteed, it is impracticable to have a zero loss rate of higher priority cells with this priority control. Such being the case, intrablock independent prediction and cyclic insertion of intraframe coded picture (cyclic refreshing) is introduced as a means of compensating the picture quality when higher priority cells are lost.
4.2.1. Intrablock independent prediction Decoding error propagates both in the intraframe and interframe directions as shown in Fig. 5 when higher priority cells containing G1 data is lost. The picture frame is divided into m x n subblocks, and prediction coding is carried out independently for each block to restrict the propagation of the error within a frame to a limited area. This measure is aimed at preventing decoding errors occurring in a given block from propagating to other blocks by carrying out the coding without referring to pixels outside the block in question. In carrying out interframe/intraframe adaptive extrapolative and interpolative prediction coding in units of sub-blocks, pixels outside the block are referred to only when pixels at the block boundary are coded (decoded). In this case, the propagation of decoding error can be suppressed by either using interframe prediction or regarding the luminance of the reference pixels outside the block as '0' in
Currentframe [ U;r::l//structi s , bloe spart
/
t~Nextframe ~
r
r
o
r
inthepropagation frame
~:rror~pagation in later frames
Fig. 5. Error propagation on losses of higher priority cells (GI data). Vol. 3, No. 4, September 1991
296
H. Tominaga et aL / A video coding method
the intraframe prediction. However, since in interframe/intraframe alternative extrapolative and interpolative prediction coding, the decoded pixels are used not only as a reference luminance value for decoding the subsequent pixels but also for judging the prediction function [6], the decoding error in a given pixel brings about judgement errors in the prediction function handling subsequent pixels. In other words, the prediction function must be fixed at the block boundary in order to suppress the propagation of judgment error of the prediction function in question. To cope with this problem, interframe prediction was forced to be selected in this study when coding pixels at the block boundary. The coding efficiency decreases at the block boundary pixels since intraframe correlation cannot be utilized at the block boundary. Conversely, the picture quality conspicuously is influenced because intraframe error propagation due to cell loss is restricted to the block in question. In experiments, the block size is defined as 8 pixels x 8 lines.
retransmitted data and refreshed data, these methods have the disadvantages of force motion and other problems. This paper explores cyclic refreshing, which is a simple and relatively efficient means for stopping error propagation in the interframe direction. From the standpoint of picture quality it is recommended to send PCM coded data, but it was decided to transmit intraframe coded data to minimize the increase in the volume of codes transmitted as a consequence of the refreshing. In other words, intraframe coded frame is forced to be transmitted at every scene change or once every N frames. The intraframe coded frame is obtained through forced selection of intraframe prediction when coding the G1 pixels and the G2 pixels. In order to realize symbiosis with the intrablock independent prediction mentioned before, it is necessary to fix the prediction system at the block boundary, and intraframe prediction is carried out by assuming luminance '0' for reference pixels located outside the block.
5. Simulation
results
4.2.2. Cyclic refreshing Since this coding system is basically interframe coding, once an error breaks out as a consequence of cell loss it cannot be corrected easily. The following methods are available for correcting decoding error in intraframe coding: (1) demand retransmission, (2) demand refreshing, (3) cyclic refreshing, (4) etc. Although cyclic refreshing is inferior to the other two methods in terms of error correction speed and reliability, the composition of the protocol and the hardware is simple, and it is effective for broadcasting and one-way video transmission. On the other hand, demand retransmission and demand refreshing are more reliable and allow quick error correction, but the protocol and the hardware are complicated. They also require larger numbers of packets and cause more transmission delay. Since the pixel must be frozen until the arrival of the Signal Processing: Image Communication
5.1. Simulation conditions Computational simulation was carried out under the conditions mentioned in Table 3 to confirm the effectiveness of this video packet coding system. The Cronkite test picture is a sequence with relatively few changes corresponding to a TV meeting, and Beach and Flowers is a sequence with rapid changes mostly consisting of panning. Moreover, since the quantization characteristic of the prediction eror is simple, linear quantization is used. In this case, 63 quantization levels are adopted to satisfy the 40 dB minimum reproduced picture quality requirement even when there is no cell loss. The coding systems used for the sake of comparison in this simulation test are: - Method A: Interframe DPCM; - Method B: Interframe/intraframe alternative prediction coding;
H. Tominaga et al. / A video coding method
297
Table 3
2 × 10 3. As can be seen in Fig. 6 the simulation
Simulation conditions
results, the picture quality deteriorates with time in system A and system B, giving evidence that these systems are prone to be adversely influenced by cell losses. Moreover, there is sudden decrease in the SNR after the scene change on the 17th frame. The scene change cannot be followed because cell loss makes normal decoding of the picture impossible. On the other hand, in this coding system (method C-I) there is practically no deterioration in the SNR even after cell loss, the loss is restricted to lower priority cells. With this method, the SNR is far better compared to method B. Picture quality virtually equivalent to that occurring with no cell loss is possible when a zero loss rate of higher priority cells is guaranteed by priority control. This coding system results in a very high picture quality improvement. However, when cells are lost without distinction between higher priority cells and lower priority cells, picture quality deterioration is very conspicuous. In this case the SNR becomes worse than that of method B. This indicates that although the proposed method has considerable advantages, conspicuous picture quality deterioration takes place when there is an unexpected loss of higher priority cells. (This can happen when the network is congested, when the difference in delay time between higher priority cells and lower priority cells exceeds a given limit, etc.) It must be
Input picture
Non-interlace 256 pixels × 256 lines × 8 bit/pixel × 30 Hz Cronkite : 16 frames Beach and Flowers: 31 frames Total 47 frames
Block size Quantization Entropy coding Cell size
8 pixels x 8 lines 63 levels, linear quantization (mid-tread) H a f m a n coding 32 byte fixed length (information field)
- Method C-I: Proposed system, interframe/intraframe alternative extrapolative interpolative prediction coding; - Method C-II: Proposed system + intrablock independent prediction ; - Method C-III: Proposed system + intrablock independent prediction + cyclic refreshing.
5.2. Influence of cell loss on the picture quality Picture quality evaluation in terms of the SNR was carried out for 3 systems: the A system, the B system, and the C-I system to examine the picture quality improvement effect by this coding system on cell loss. As a general rule, studies referring to the A T M assume cell loss rates of the order of 10 - 9 , but a rather high loss rate of 10 3 is assumed in this study to evaluate the effectiveness of the proposed system. The cell loss is assumed to be random. In this study simulation is carried out not only for the case in which only lower priority cells are lost but also for the case that no service class is specified, in other words, higher priority cells and lower priority cells are lost under the same conditions. The ratio of higher priority cells (GI pixel data) and lower priority cells (G2 pixel data) is approximately 1:1. Assuming a lower priority cell loss rate of 10 -3 with higher priority cell loss rate equal to zero, overall cell loss rate would be half. In order to maintain overall cell loss rate 10- 3, the lower priority cell rate is assumed to be
S N R (dB)
45.00 ee,e*
40.00
\
35.00
~eee~
_
_
. - .
"MethodA
DQ
~; ~h'&l"~:r ~f&;-o-f%w-e-6--' --
30.00 --
"[vlet-h~xl-ff-[(To~s-of ~o~h~-
'
25.00
20.00
t~a ~an~aao
15.00
--
10.00
-
~lo.~a
~ o o a
-Cell loss r a t e = le-3 S. . . . .
hange, x::
5.00 0,00
I0.00
20.00
30.00
40.00
50.00
F:ame number(1/30sec) Fig. 6. Quality degradation on cell losses. Vol. 3, No. 4, September 1991
H. Tominaga et al. / A video coding method
298
remembered that this simulation was carried out under unfavourable conditions in which no priority control of higher priority/lower priority cells was carried out at all, and that in actual cases lower priority cells are selectively lost due to the priority control of the cells. Even when method B is supplemented with the cell loss compensation methods such as those in Section 4.2 (intrablock independent prediction, cyclic refreshing) are added to method B, in most cases method C-I is better for picture quality except for extremely unfavourable conditions such as this simulation. Moreover, as will be shown in Section 5.5, method C-I is superior to method B for code volume. It appears that this is the best high-efficiency video coding method for ATM networks, Even in this system remedial measures such as intrablock independent prediction, cyclic refreshing, etc., must be used in order to cope with unexpected higher priority cell losses. 5.3. Picture quality improvement effect of remedial measures to cope with higher priority cell loss To examine the effect of intrablock independent prediction on picture quality the SNR was evaluated. The methods are: method C-I without any remedial measure to cope with higher priority cell loss, method C-II which uses the intrablock independent prediction, and method C-III which uses both intrablock independent prediction and cyclic refreshing. Cells were lost at random (10-3 cell loss rate) and it is assumed that they are lost under the same conditions for higher priority cells and nonpriority cells. Moreover, refreshing is assumed to be carried out once every 10 frames ( N = 10) starting from the initial frame and the scene change frame. Figure 7 shows the results. Since a cell loss's influence is limited to its block because of the independent coding of the various frames in block units, the picture quality is improved by 10-17 dB comparing with the case without intrablock independent prediction (Fig. 7). Moreover, since the picture quality is added, the Signal P r o c e s s i n g :
Image Communication
SNR (dB) -
45.00
-
R'efresh f r a m ei - -
40.00 -35.00
8u~ Iq.l.m~l~e.lkm ,DBOOODnO01
On.ooOo
30.00 -"Method C-I 25.00 20,00 -15.00 - -
Cell loss r a t e = le-3
10.00
Scene change
,/
5.00 - 0.00
10.00
20.130
I
30.00
_
40.00
50.00
Frame number (1/30sec)
Fig. 7. Effectof cell loss compensationfor higherprioritycells. SNR is kept at approximately 40 dB even with some fluctuations. As can be seen, intrablock independent prediction and cyclic refreshing give a noticeable picture quality improvement, and very effectively for compensate for higher priority cell loss. 5.4. Change in the code amount due to higher priority cell loss compensation As mentioned before, intrablock independent prediction and cyclic refreshing are effective remedies for suppressing picture quality deterioration due to higher priority cell loss. It must be remembered, however, that since the correlation between pixels cannot be utilized at the block boundary in intrablock independent prediction, the coding efficiency decreases. Moreover, at the moment of the refreshing in cyclic refreshing, correlation with the previous frame cannot be utilized and causes a decrease in the coding efficiency. The number of cells generated for the various frames is compared for three methods, method B, method C-I and method C-II to examine the increase in the code volume due to intrablock independent prediction. Figure 8 shows the results. Compared to the case without independent prediction (method C-I) the number of cells generated with intrablock independent prediction (method
299
H. Tominaga et al. / A video coding method
C-II) increases by approximately 30 cells (8 kbit) per frame under ordinary conditions and by approximately 300 cells (75 kbit) under the peak conditions of a scene change. Even with that increase, method C-II yields higher coding efficiency than interframe/intraframe alternative prediction. When the benefits are considered, the increase in code volume due to intrablock independent prediction is within acceptable limits. Next, the number of cells generated in each frame by method C-II and method C-III were compared in order to examine the variation in the code volume due to cyclic refreshing. Figure 9 shows the results. As can be seen in Fig. 9, the increase in the number of cells due to cyclic refreshing is of the order of 20 cells (5 kbit) per frame compared to intrablock independent prediction. This means an increase of the order of 50 cells (12 kbit) compared to the case without higher priority cell loss compensation (method C-I). Considering the picture quality improvement effect, this is an acceptable increase. Moreover, the refreshed screen generates less cells compared to the alternative prediction method during the scene change on the 17th frame. This is presumed to occur because the prediction method's optimum selection is not carried out during the scene change because the alternative Number of cell x 103
I
1.10 1.00 ---
~V{e'{l~odC-I
~et-h-&J- C:- f["
0.90
I
Length of cell = 32 Byte(fixed)
0.80 -0.70 0.60 -0.50 -- ~ 0.40
--
~m' m
"
Refresh frame
/I
0.30 - 0.00
10.00
20.00
I 30.00
40.00 50.00 Frame number (1]30sec)
Fig. 8. Generated cell number on adding block-independent prediction.
Number of cell x 103 1.10 -1.00 Method C-II
0.90 0.80
~et'[{~i C - I l l Refresh frame
0.70 -0.60 0.50
. ~
Length of cell = 3 2 Byte(fixed)
0.40 Scene change
0.30
/I 0.00
10.00
20.00
30.00
40.00 50.00 Frame number (1/30sec)
Fig. 9. Generated cell number on adding cyclic refresh. prediction method does not transmit the prediction selection information. 5.5. The S N R mean value and worst value comparison
The picture quality was compared in terms of the mean SNR and the worst SNR for cases with 3 cell loss rates, 10 -4, 10 -3 and 10 -2, to evaluate the effectiveness of this method using remedial measures compensate for higher priority cell loss. Figures 10 and 11 show the obtained results. Four cases: method A, method B, this coding method with loss of only lower priority cells, and this coding method with equally likely loss of cells higher priority and lower priority cells, are compared here. It should be noted that the apparent cell loss rate is doubled as mentioned in Section 5.2 when only lower priority cells are lost, to even the overall average cell loss rate compared to the case of equally likely cell loss higher priority and lower priority cell loss. As can be seen from Figs. 10 and 11, both the mean value and the worst value of the SNR of method A are low irrespective of the cell loss rate, on the other hand the SNR of method B decreases rapidly with the increase of the cell loss rate. When only lower priority cells are lost in this coding method (method C-III), the SNR is kept at approximately 40 dB irrespective of the cell loss Vol. 3, No. 4, September 1991
300
H. Tominaga et al. / A video coding method S N R (dB) 45.00 40.00
-
'
~....
35.00
-
: 21
-
. . . . . ""
"..% 30.00 --
am'k'eff+J wt A
"% ......."-.,
25.00 20.00 - -
"f,;fgiffi~'g ~ t-fro)"C'-f[i "([~s's'o( fffw~ ",.,.. '~,12t-h~l-(~fIl-([osso~ l~oih- -
15.00 - -
"......... • -.......
hQ~
10.00
..........
•
",..,,.,
5.00-le-04
3
le-03
3
le-02 Cell loss rate
Fig. 10. Mean SNR.
rate, and moreover the decrease in SNR is small compared to other methods even when higher priority cells are lost. This coding method combined with cope with higher priority cell loss compensation is a high efficiency video packet coding method with resistance to cell loss in ATM networks.
networks. In this method, picture elements are separated into two groups, which are called G1 pixels and G2 pixels. The data for G1 pixels are transmitted in higher priority cells, while the data for G2 pixels are transmitted in lower priority cells. In case of network congestion, lower priority cells are selectively discarded. The receiver can retain image quality by simply replacing lost data with zero. Furthermore we examined cyclic refreshing and intrablock prediction as measures against discards of higher priority cells. Simulation results show that we can expect 10-17 dB increase in average SNR by employing intrablock prediction, 6-10 dB increase by inserting intraframe-coded frame periodically. In experiments, the average SNR of the decoded image remains approximately 40 dB. We note that increases in total code amount by employing measures against discards of higher priority cells are tolerable considering their effect. So we can conclude that this method is practical as video coding schemes in ATM networks.
References 6. Conclusion In this paper, we examined interframe/intraframe alternative prediction coding using quincunx subsampling as a video coding method for ATM SNR (riB) 45.00 - - - -
~';,.,......,.,,
40.00-
35.00 -
"Method A
".,.. ".....~
30.00 -
'll;¢lethod B
M~$-o~-~c'[Ja-(Lossof k ........ '~vl'c?h~l-(~-fiI-(loss of b
"'"'"..,....,,....
25.00
".......
20.00 -
m
15.00 10.00 -
m
5.00 le-04
3
le-03
3
1 e-02 Cell loss rate
Fig. 11. Worst SNR. Signal Processing: Image Communication
[1] R. Kishimoto, N. Norihiro and K. Irie, "Packet loss compensation for HDTV signals in an A T M network", IEICE Tech. Rep., IE88-88, December 1988 (in Japanese). [2] Maglaris et al., "Performance model of statistical multiplexing in packet video communications", 1EEE Trans. Commun., Vol. 36, No. 7, July 1988. [3] M. Nomura, T. Fujii and N. Ohta, "Basic characteristics of variable rate video coding in ATM environment", IEEE JSAC, Vol. 7, No. 5, June 1989. [4] M. Ohta, H. Suzuki and T. Ohmachi, "Error recovery procedures for packetized inter frame video coding", Proc. 2nd Internat. Workshop on Pocket Video, B6, September 1988. [5] Verbiest et al., "The impact of the ATM concept on video coding, IEEE JSAC, Vol. 6, No. 9, December 1988. 16] Y. Yashima and K. Sawada, "A highly efficient coding method for HDTV signals", Proc. International Conference on Communication, Seattle, USA, June 1987, paper 5.6.