- Email: [email protected]
Forward-error correction with decision feedback
INFOR~/[ATION AND CONTROL 21, 117--133 (1972)
Forward-Error Correction with Decision Feedback* GEORGE I. DAVlDA Department of Electrical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI 53201 AND SUDI-IAKAR M .
REDDY
Department of Electrical Engineering, University of Iowa, Iowa City, Iowa 52240
A n e w coding s c h e m e is presented. In this s c h e m e a given t - e r r o r - d e t e c t i n g code is u s e d for t r a n s m i s s i o n over a noisy channel. W h e n t or less errors are detected, a single r e q u e s t is m a d e via a feedback c h a n n e l for t r a n s m i s s i o n of additional r e d u n d a n c y to enable t h e decoder to correct t or less errors. It is s h o w n to be s u p e r i o r to forward error correction in rate a n d probability o f error. F o r low c h a n n e l errors t h e r e s u l t i n g rate is h i g h e r t h a n t h a t of retransm i s s i o n w i t h no significant difference in t h e probability of error.
I. INTRODUCTION
T w o basic schemes which can be used for error control are forward-error correction and retransmission (Peterson, 1961). In forward-error correction, sufficient redundancy is sent with the information to allow the decoder to correct certain error patterns. In retransmission, an error detecting code is used to request, via a feedback channel, retransmission of an erroneous block (Benice and Frey, 1964; Park, 1969). It has been shown that for lowerror probabilities, the rate for retransmission is higher than that of forwarderror correction (Benice and Frey, 1964). We consider a combined strategy.
* T h e research reported in this p a p e r was s u p p o r t e d in part b y t h e National Science F o u n d a t i o n g r a n t s G K - 2 9 9 0 a n d G K - 1 0 0 2 5 . T h i s p a p e r is part of a dissertation s u b m i t t e d to t h e D e p a r t m e n t of Electrical E n g i n e e r i n g , U n i v e r s i t y of Iowa, Iowa City, in partial fulfillment of t h e r e q u i r e m e n t s of t h e P h . D . degree. Copyright © 1972 by Academic Press, Inc. All rights of reproduction in any form reserved.
117
118
DAVIDA AND REDDY II. NEW CODING SCHEME
Let (n, k) be a code 9f block length n, k information symbols per block and r = n - k redundant symbols per block, which can correct t or less errors. Suppose that it is possible to puncture this code, that is, remove some of the redundant symbols, and obtain an (n,, k) code, with r 8 ~ ns - - k redundant symbols, which can only detect t or less errors, as shown in Fig. 1, where r ' = r - - r s is the number of redundant symbols which are removed. These r ' symbols are initially stored at the transmitter. T h e ( n s , k ) code is then used for transmission over a noisy channel, and when t or less errors are detected, a single request is made, via a noiseless feedback channel, for transmission of the remaining redundant symbols.
rI
I_.,
1l. s
E
-1d
i ~.
FIG. 1. Puncturing a code.
L
J
FIG. 2. Typical .received sequence. T h e feedback channel need not be delayless as in some feedback communication schemes (Berlekamp, 1968). A typical received sequence is shown in Fig. 2, where (ki, r~i) is the i-th block received and rl' is the set of remaining redundant symbols, had (kl, rsl) block been in error, and z is the delay in the two channels. This delay will determine the number of storage elements at the decoder,
119
ERROR CORRECTION WITH FEEDBACK THEOREM 1.
The expected value of the rate R for the proposed scheme it
~(R) = k/(n~ + P # ) ,
(1)
where PD is the probability of detecting an error. Pro@
Let M = the number of words to be transmitted, n = the length of the full code,
then E(number of blocks of length n) = M P 9 , E(number of blocks of length ns) = M(1 -- P9), E(total number of symbols transmitted) = MnP9 -t- Mns(1 -- PD), number of information symbols = Mk. Hence
E(R) = Mk/[MnPD + Mn~(1 -- PD)] = k/[nPo + n~(1 -- PD)]. Substituting ns -]- r' for n, we get
E(R) = a/(n~ + P y ) .
Q.E.D.
Consider the Binary Symmetric Channel. Assume that bounded distance decoding is used, that is, only e or less errors are corrected. We then have the following theorem: THEOREM 2. The probability of correctly decoding a block of k bits for the proposed scheme is higher than that of forward-error correction by exacly APe, where
i=e+l
Proof.
Let
P~I = probability of correctly decoding k bits for the proposed scheme, P~2 = probability of correctly decoding k bits for forward error correction.
120
DAVIDA AND REDDY
Then
i=0 •
_
e--i
yt
.
.
+ °~"~-~ ~~ v ~-~ [,~' + ~ ' ~ < ...+ (~ ~' ~)~ v'-~+q + .2
i=O
A p e = q,~ + n s p q ~ _ , q / + n s p q , _ l ( r , p q ~ , _ l ) + n~(n~ - 2
1)p,q,_~(q~,)
+ ...
-- [q° + "Pq°-" ~ n(n 2-- 1) p~q._~ + ...] = qn, _ q , @ n~pqn~+r'-I __ npqn-1
+ [,/+
n~Jn~ .-- k)] p~¢~+~,_,, 2
n(n -- 1) p,¢_, + ... 2 "
Combining terms, we get A P , = q"" - - q" - - r'pq "-a
r'(r' - 2
1) p2q,-~
+ ...
A P e = qn"[1 - - q~' - - r'pq ~'-1 + ""]
i=O
,f"
Q.E.D. /=e+l
ERROR CORRECTION W I T H FEEDBACK
121
T h e above result can be seen by observing that whenever the n~ bits are received error-free, certain uncorrectablc error patterns are avoided, as shown in Fig. 3.
O0 . . . .
O0
I I0 I0 ---II /
115
r FIG. 3. Avoided error patterns.
EXAMPLE 1. Let us look at the shortened H a m m i n g (6, 3) single-errorcorrecting binary code, whose parity check matrix H is H=
1101 0100
.
F r o m the top row we note that the first check bit forms a check on all the information positions. Thus, if we were to send the 3 information symbols and the first check bit, the decoder can detect all single errors. When a single error occurs the remaining two parity bits can be requested and, if received error-free, the single error can be corrected. T h e expected value of the rate for this code is E(R) = 3/(4 + 2PD), where P , = P[1 error] q- P[3 errors], i.e.,
PD = 4pq3 + 4P~q. Thus
E(R) = 3/4(1 + 2pq ~ + 2p3q). Also
APe :
~ ( 2~ i 6 i i=2 i ) p q -- = p2qa. I I I . PUNCTURING
All the H a m m i n g single-error-correcting codes can first be shortened and then punctured to yield a single-error-detecting code. T h e parity check
122
DAVIDA AND
REDDY
matrix for binary codes, with n = 2 m -- 1, consists of all possible nonzero m-tuples. "i 1 1 0
"" ...
1
1 !
.
0 0 ... 1 0....
1 0
0
.,.
H=
m.
,.. ...
0
1
""
1 0
0
0
l<
...
m
1 >
For convenience we assume that the last m positions are the redundant positions. If we drop all the columns in the information positions, which begin with zero, the first check bit will form a parity check on the remaining information positions and we thus obtain a single-error-detecting code with one parity bit whose parity check matrix H ' is H'=[1
1 ,C
1
1 ~8
...
1; -->
where ns = 2 m-1. Initially, only one parity check is sent, and when a parity failure is detected, the remaining (m - - 1 ) checks are sent and then correction is carried out. Other codes that can be punctured include the maximal shift register codes (Solomon and Stiffler, 1965), uniform convolutionai codes (Robinson, 1969), Reed-Solomon codes (Reed and Solomon, 1960), and a class of extended BCH codes (Davida, 1970). The Golay (23, 12) code is next shown to be puncturable. This, however, does not appear to be a general result. Consider theparitycheekmatrixH:
H=
110110000000000" "11010001 111 01000111001000000000 101 10100011100100000000 110 11010001100010000000 111 01101000100001000000 111 10110100000000100000 101 11011010000000010000 100 11101101000000001000 100 01110110100000000100 110 00111011000000000010 101 00011101100000000001 1
23
123
ERROR CORRECTION WITH FEEDBACK
Dropping column 7, we get the shortened (22, 11) whose parity check matrix H ' is
H'=
-110100! 110110000000000 111010011100100000000~ 1011010011'10010000000 110110000110001000000 111011100010000100000 111101010100000010000 101110101000000001000 100111110100000000100 100011011010000000010 110001101100000000001 101000110110000000000
.
Rows 1, 2, 3, 5 and 8 Of H' ~ o m the extended H a m m i n g (16, 11) binary code which can detect 3 or less errors. T h e parity check matrix H ~ %r the (16, 11) code is
H"=
[i0000000i] 11010011100100 01101001110010 11011100010001 00111110100000
.
Thus, we can initially store 6 parity che& bits and transmit with a rate of 11/16 instead of 11/22. When 3 or less errors are d a e a e d we can request the remaining symbols and proceed to correction.
IV.
ALTERNATE SCHEMES
T h e above scheme naturally leads one to consider the following variations, of which B is much easier to implement since it avoids the necessity of puncturing codes, which may not be readily accomplished: A. Let an (n, h) code which can correct t or fewer errors be punctured to an (n~, h) code which can correct e errors and detect t errors (e < t). T h e expected value of the rate for this scheme will be higher than the above mentioned one in cases where e or less errors are quite likely resulting in a high request rate for additional information. T h e remaining redundancy will be requested only when the number of errors exceeds e. 643/eI/2-3
124
DAVIDA AND REDDY
Then
E(R) = k/(ns q- r'PD).
(3)
B. Let (n 1 , k) be a code which can correct e errors and detect t errors, and (n2, k) be a different code which can correct t errors. Initially, the (nl, k) code is used. When more than e errors occur, the redundant symbols, computed according to the (ne, k) code, are requested and the redundant symbols at the (n I , k) code are discarded. For this case,
E(R) = k/In 1 + eD(n2 -- k)].
(4)
EXAMPLE 2. Consider the (5, 4) single-error-detecting binary code with one parity bit whose parity check matrix H~ is H 1-
[1 1 1 1 1],
and the H a m m i n g (7, 4) single-error-correcting binary code whose parity check matrix H e is H~=
0110~
.
10100 Initially, the (5, 4) code is used, and when a single error is detected, a request is made to send the parity symbols for the (7, 4) code and the parity bit from the (5, 4) code is discarded. T h e n
E(R) = 4/(5 q- 3PD), P g = (~) Pq4 + (53)P3q2 + (55)p5 : 5pq~ + lOpZq~ + pS; therefore
E(R) = 4/(5 + 15pq4 + 30pSqz + 3p5). Using two codes, one for detection and the other for correction is more readily accomplished since the two codes are not necessarily related. This leaves one with greater flexibility in choosing the two codes.
V. COMPARISON WITH RETRANSMISSION Let us consider the simple retransmission scheme where an (n, k) code is used for error detection over a noisy channel. When an error is detected,
125
ERROR CORRECTION W I T H FEEDBACK
a request is made via a noiseless feedback channel for the repetition of that erroneous block. We will consider here the scheme where the number of repetitions is unlimited. Consider the Binary Symmetric Channel.
The expected value of the rate for the simple retransmission
THEOREM 3. scheme is
E(R) ~- (1 -- eD)h/(n~ + 1), where PI) is the probability of detecting an error. Proof.
Let M ~ the number of words transmitted;
then E(number of blocks repeated once)
= MPD,
E(number of blocks repeated twice)
~ MPD 2,
E(number of blocks repeated N times) = MPD N, E(total number of bits transmitted)
= M(ns @ 1) @ M(n~ q- 1) PD
+ M(n~ + 1) PD ~ "1
= M(n~ + l) I - - P D 1
= M(n~ q- 1) 1 M • E(R)
=
M(n~ +
1)[1/(1
-- PD)] =
k(1 - -
-
-
PD
P~)/(n~ + 1).
Q.E.D.
The probability of error for the simple retransmission scheme is
THEOREM 4.
P~ = (1 -- PD -- q~)[1/(1 -- PD)]"
P~oof. P, ~ P(undetected error) + PD[P(undetected error)]
q- (PD)n[p(undetected error)] q- ---, V~ = (I -- PD - - q~)(1 + PD q- PD 2 -k "") 1 =
-
Po -
¢o
( 1 -
P o )
Q.E.D.
126
DaVIOa AND
REDDY
The expected values of the rate for the shortened and punctured Hamming codes for both the proposed scheme and retransmission are shown in Figs. 4-6 for values of n s of 4, 128, and 512. The word error probabilities are shown at selected points. The corresponding word error probabilities are shown in Figs. 7-9. In the case of the proposed scheme, when an error is detected, the remaining r ' parity bits are requested (only once), whereas in retransmission the whole code-word is repeated. There is no significant difference in the probability of error between the two schemes. Significant difference does exist in the rate, where the proposed scheme.is superior. When two codes are used, that is, one for detection and the other for correction, retransmission has a lower probability or error, as shown in Figs. 10-12. But this is achieved at a significant reduction in the rate, as in Figs. 13-15, where again the word error probabilities are shown at selected points.
1,0 O
i 0
~0
X
,80 -:
ILl
X
(4~3):
r'=
2
~'-.0
.60
.40
~D
-- -- -- PROPOSED
eJCHEME
x
o)
p-
.20
RETR/~.NSMISSION
,1,
,I,
,I.
~o-7
:6s
~o
-5
, i . . I . , -4
,o
-3
io
I, -2
io
, I . . i - |
Io
I
P
FIG. 4. Comparison of the proposed scheme (single-error correction) with retransmlssion (single-error detection). The word-error probabilities are shown at selected points.
127
ERROR CORRECTION WITH FEEDBACK N IO X 1,0
U2 8112T}, r = 7 ,
o-.~ -o--
(128,127)
%
.80
W
-6--.o
.60
I-<~ n.-
o~
..;
.40
- - - -- PROPOPOSED SCHEME
.~0
RETRANSMISSION
, I ,
,~7
.
I
~;6
.
.I
65
.
. I . . I
,;4
.
,;3
.
I,
,~
.
I.
I~1
.I
,
p
FIG. 5. Comparison of the proposed scheme (single-error correction) with retransmission (single-error detection). T h e word-error probabilities are shown at selected points.
V I . CONCLUSIONS
As expected, the above scheme performs better than forward-error correction. It was seen that the probability of error for the proposed scheme is slightly higher than the simple retransmission, but that it has significantly higher rate. Since the number of requests for additional redundant bits is limited to one, the overflow problem has been considerably reduced. It is expected that for equal rate or equal probability of error the proposed scheme will perform better than the retransmission scheme in the probability of error and the rate, respectively.
L~IZ,6II~I,
hO
~',l)
.-.
o
{512~
.00
-
..:
..:
-;
0
I.l..J , 6 0 I-l0 .4.0
--
- PROPOSED
.ZO
SCHEME
R ETRANSMISSION
FIO. 6. Comparison of the proposed scheme (single-error correction) with retransmission (single-error detection). The word-error probabilities are shown at selected points. |,0 m
CE 0 BE re" LLi LL.
0 >.. I-._l CO CO
0 0:: 0..
- - -- -- P R O P O S E D
DE 0
/
• I
.
•
I.
~
•
I
.
•
I
•
•
SCHEME
RETRANSMISSiON
i.
I
•
•
I
,
•
I
FIc. 7. Comparison of the proposed scheme (single-error correction) with retransmission (single-error detection).
1.0
cr O r~ r~ L~
'T
LL 0 >t---
-n <~ m O n c~ rY O
[~"
i
I
,
td 7
.
I
,
~d~
,
I
,
,o-5
.
I
,
jo-4
,
I
i
,d ~
,
I
,
,d ~
.
I
,
jo"~
I
I
I.
P Fie. 8. Error probability for the proposed scheme (single-error correction) and retransmission (single-error detection).
1.0
O no a:
"9
, 63
Lfl
I,L 0 >I---
,6 ~ m <~ m O rr ~-
169
l'Y
O
.I,
j6 r
, I,
io "~
.I
j6 5
...... I ,
,~
,I,
i~ 3
. ~ . . I . ~
~6 2
16 j
I.
P
FIG. 9. Error probability for the proposed scheme (single-error correction) and retransmission (single-error detection).
hO
/ ....
PROPOSED SCHEME
/ t
n. 0
I
RETRANSMISSIO N
I l
O~ W 0 >_
I
t I
r'~
I I
iI
I
!
'1.
Jd ~
t
0 r~. n
Ir
i/ (2~11)
.J m m
/I
(I~,H)
l~ c
I
I
L~'~
.
I
I
0
I I I .I
. ~.
,d r
,.I{.
,o-~
o
Jl,
,d 5
.I,
td ~
. i . . I , . ,
,d J
,d ~
10-I
~
p
FIc. 10. Comparison of the proposed scheme (3-error correction) w i t h retransmission (3-error detection). 1,0 -----PROPOSED
rr O tic
tr
SCHEME
//
' RETRANSMISSION
/
/
l0 3
W ta. O >_
,
10~
I-_I m
I
cn 0 n~ [I.
(128j12o) I /
|
I
I I / /
n" 0 -i$ I0
•
L6r
•
.
i~
•
|
~5
.
•
•
•
•
16+ ,63 ~62
.
i
i~i
i
|
I.
FIo. 11. Comparison of the proposed scheme (3-error correction) with retransmission (3-error detection).
1.0 .
rr 0
I
,
i//
-3 I0
(612~5 02). /
rr" LLI LL 0 >,.. I'._1
I
/
-CD Io
(33
-s
IO (33
0
C3 (3:: 0
It)
----
-PROPOSED
5CHENE
RE T R A N S M ~ S S ~ O R .15 IO
,
I
t67
,
.
I
.
166
.
I
,
165
.
I
,
.
I ff~
I
.
,
I d3
1
,
i62
.
I
,
,I
i6~
I.
p
FIC. 12. Comparison of the proposed scheme (3-error correction) with retransmission (3-error detection).
I.O
b'O
.8
(I 6,il),(26,11)
.~.
t _O
.4 LIJ
!'or .3
t67
io-e
1if5
164
,-~ I0
I0-2
I I0-~
I
P
FIG. 13. Comparison of the proposed scheme (3-error correction) with retransmission (3-error detection). T h e word-error probabilities are shown at selected points,
IO
(12811g0) f,144~12 O)
hO
-
x ~
'o
.. : .8 i
.6 LLI l--
n.* .4
.2
'" RETR& NS MISS|ON
,
I
.
Id 7
,
I
,
16 e
,
I
,
~6s
,
I
.
164
~, ,
I
,
~6~
.
I
"m"
,~,..-J --
16 2
Io"~
0 "~I
i.
P
FIG. 14. Comparison of the proposed scheme (3-error correction) with retransmission (3-error detection). T h e word-error probabilities are shown at selected points. N
1.0 _
,,~.=:<
IO
(512~502) 153Z~502)
X N
.,~'6ix .,.
,l.,,..,.
i~7
I(~6 ib5
i~4
,;
,,..~;:
~
16~
i~I
|~Z
~,
I
p
FIG. 15. Comparison of the proposed scheme (3-error correction) with retransmission (3-error detection). T h e word-error probabilities are shown at selected points.
ERROR CORRECTION WITH FEEDBACK
133
ACKNOWLEDGMENT The authors would like to thank Professor John P. Robinson for the numerous discussions and helpful suggestions. They would also like to thank the reviewers for correcting errors in an earlier version of the manuscript. RECEIVED: M a y
6,
1971
REFERENCES 1. BENICE, R. J. AND FREY, A. H. (1964), An analysis of retransmission systems, IEEE Trans. Commun. Technology CTI2, 135-145. 2. BERLEKAMP, E. R. (1968), Block coding for the binary symmetric channel with noiseless delayless feedback, in "Proceeding of the Symposium on Error Correction Codes at the University of Wisconsin," Wiley, New York. 3. BEt~LEI~AMP,E. R. (1968), "Algebraic Coding Theory," McGraw-Hill, New York. 4. DAVmA, G. I. (1970), Multiple extension of the minimum distance of BCH codes, in Proceedings of the Eighth Annual Allerton Conference on Circuit and System Theory. 5. FANO, R. M. (1963), "Transmission of Information," M1T Press, Cambridge, MA. 6. GraVEN, P. E. (1961), Feedback communication systems, in "Lectures on Communication Systems Theory" (E. J. Bagdady, Ed.), McGraw-Hill, New York. 7. MASSEY, J. L. (1963), "Threshold Decoding," M I T Press, Cambridge, MA. 8. PARK, J. H. (I969), Information rates and errors using decision feedback, IEEE Trans. Commun. Technology CT17, 20-24. 9. PETERSON, W. W. (1961), "Error Correcting Codes," M I T Press, Cambridge, MA. 10. RBED, I. S. AND SOLOMON, G. (1960), Polynomial codes over certain finite fields, J. Soc. Ind. Appl. Math. 8, 300-304. 1l. ROBINSON, J. P. (1969), Punctured uniform codes, IEEE Trans. Information Theory IT15, 149-152. 12. SCHALKWlJK, J. P. M. AND KAILATH, T. (1966), A coding scheme for additive noise channels with feedback: 1. No bandwidth constant, IEEE Trans. Information Theory IT12, 172-182. 13. SCHALKWlJK, J. P. M. (1966), A coding scheme for additive noise channels with feedback: II. Bandlimited signals, IEEE Trans. Information Theory IT12, 183-189. 14. SOLO~ON, G. AND STIFF~R, J. J. (1964), Algebraically punctured cyclic codes, Information and Control 8, 170-179.
Forward-Error Correction with Decision Feedback* GEORGE I. DAVlDA Department of Electrical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI 53201 AND SUDI-IAKAR M .
REDDY
Department of Electrical Engineering, University of Iowa, Iowa City, Iowa 52240
A n e w coding s c h e m e is presented. In this s c h e m e a given t - e r r o r - d e t e c t i n g code is u s e d for t r a n s m i s s i o n over a noisy channel. W h e n t or less errors are detected, a single r e q u e s t is m a d e via a feedback c h a n n e l for t r a n s m i s s i o n of additional r e d u n d a n c y to enable t h e decoder to correct t or less errors. It is s h o w n to be s u p e r i o r to forward error correction in rate a n d probability o f error. F o r low c h a n n e l errors t h e r e s u l t i n g rate is h i g h e r t h a n t h a t of retransm i s s i o n w i t h no significant difference in t h e probability of error.
I. INTRODUCTION
T w o basic schemes which can be used for error control are forward-error correction and retransmission (Peterson, 1961). In forward-error correction, sufficient redundancy is sent with the information to allow the decoder to correct certain error patterns. In retransmission, an error detecting code is used to request, via a feedback channel, retransmission of an erroneous block (Benice and Frey, 1964; Park, 1969). It has been shown that for lowerror probabilities, the rate for retransmission is higher than that of forwarderror correction (Benice and Frey, 1964). We consider a combined strategy.
* T h e research reported in this p a p e r was s u p p o r t e d in part b y t h e National Science F o u n d a t i o n g r a n t s G K - 2 9 9 0 a n d G K - 1 0 0 2 5 . T h i s p a p e r is part of a dissertation s u b m i t t e d to t h e D e p a r t m e n t of Electrical E n g i n e e r i n g , U n i v e r s i t y of Iowa, Iowa City, in partial fulfillment of t h e r e q u i r e m e n t s of t h e P h . D . degree. Copyright © 1972 by Academic Press, Inc. All rights of reproduction in any form reserved.
117
118
DAVIDA AND REDDY II. NEW CODING SCHEME
Let (n, k) be a code 9f block length n, k information symbols per block and r = n - k redundant symbols per block, which can correct t or less errors. Suppose that it is possible to puncture this code, that is, remove some of the redundant symbols, and obtain an (n,, k) code, with r 8 ~ ns - - k redundant symbols, which can only detect t or less errors, as shown in Fig. 1, where r ' = r - - r s is the number of redundant symbols which are removed. These r ' symbols are initially stored at the transmitter. T h e ( n s , k ) code is then used for transmission over a noisy channel, and when t or less errors are detected, a single request is made, via a noiseless feedback channel, for transmission of the remaining redundant symbols.
rI
I_.,
1l. s
E
-1d
i ~.
FIG. 1. Puncturing a code.
L
J
FIG. 2. Typical .received sequence. T h e feedback channel need not be delayless as in some feedback communication schemes (Berlekamp, 1968). A typical received sequence is shown in Fig. 2, where (ki, r~i) is the i-th block received and rl' is the set of remaining redundant symbols, had (kl, rsl) block been in error, and z is the delay in the two channels. This delay will determine the number of storage elements at the decoder,
119
ERROR CORRECTION WITH FEEDBACK THEOREM 1.
The expected value of the rate R for the proposed scheme it
~(R) = k/(n~ + P # ) ,
(1)
where PD is the probability of detecting an error. Pro@
Let M = the number of words to be transmitted, n = the length of the full code,
then E(number of blocks of length n) = M P 9 , E(number of blocks of length ns) = M(1 -- P9), E(total number of symbols transmitted) = MnP9 -t- Mns(1 -- PD), number of information symbols = Mk. Hence
E(R) = Mk/[MnPD + Mn~(1 -- PD)] = k/[nPo + n~(1 -- PD)]. Substituting ns -]- r' for n, we get
E(R) = a/(n~ + P y ) .
Q.E.D.
Consider the Binary Symmetric Channel. Assume that bounded distance decoding is used, that is, only e or less errors are corrected. We then have the following theorem: THEOREM 2. The probability of correctly decoding a block of k bits for the proposed scheme is higher than that of forward-error correction by exacly APe, where
i=e+l
Proof.
Let
P~I = probability of correctly decoding k bits for the proposed scheme, P~2 = probability of correctly decoding k bits for forward error correction.
120
DAVIDA AND REDDY
Then
i=0 •
_
e--i
yt
.
.
+ °~"~-~ ~~ v ~-~ [,~' + ~ ' ~ < ...+ (~ ~' ~)~ v'-~+q + .2
i=O
A p e = q,~ + n s p q ~ _ , q / + n s p q , _ l ( r , p q ~ , _ l ) + n~(n~ - 2
1)p,q,_~(q~,)
+ ...
-- [q° + "Pq°-" ~ n(n 2-- 1) p~q._~ + ...] = qn, _ q , @ n~pqn~+r'-I __ npqn-1
+ [,/+
n~Jn~ .-- k)] p~¢~+~,_,, 2
n(n -- 1) p,¢_, + ... 2 "
Combining terms, we get A P , = q"" - - q" - - r'pq "-a
r'(r' - 2
1) p2q,-~
+ ...
A P e = qn"[1 - - q~' - - r'pq ~'-1 + ""]
i=O
,f"
Q.E.D. /=e+l
ERROR CORRECTION W I T H FEEDBACK
121
T h e above result can be seen by observing that whenever the n~ bits are received error-free, certain uncorrectablc error patterns are avoided, as shown in Fig. 3.
O0 . . . .
O0
I I0 I0 ---II /
115
r FIG. 3. Avoided error patterns.
EXAMPLE 1. Let us look at the shortened H a m m i n g (6, 3) single-errorcorrecting binary code, whose parity check matrix H is H=
1101 0100
.
F r o m the top row we note that the first check bit forms a check on all the information positions. Thus, if we were to send the 3 information symbols and the first check bit, the decoder can detect all single errors. When a single error occurs the remaining two parity bits can be requested and, if received error-free, the single error can be corrected. T h e expected value of the rate for this code is E(R) = 3/(4 + 2PD), where P , = P[1 error] q- P[3 errors], i.e.,
PD = 4pq3 + 4P~q. Thus
E(R) = 3/4(1 + 2pq ~ + 2p3q). Also
APe :
~ ( 2~ i 6 i i=2 i ) p q -- = p2qa. I I I . PUNCTURING
All the H a m m i n g single-error-correcting codes can first be shortened and then punctured to yield a single-error-detecting code. T h e parity check
122
DAVIDA AND
REDDY
matrix for binary codes, with n = 2 m -- 1, consists of all possible nonzero m-tuples. "i 1 1 0
"" ...
1
1 !
.
0 0 ... 1 0....
1 0
0
.,.
H=
m.
,.. ...
0
1
""
1 0
0
0
l<
...
m
1 >
For convenience we assume that the last m positions are the redundant positions. If we drop all the columns in the information positions, which begin with zero, the first check bit will form a parity check on the remaining information positions and we thus obtain a single-error-detecting code with one parity bit whose parity check matrix H ' is H'=[1
1 ,C
1
1 ~8
...
1; -->
where ns = 2 m-1. Initially, only one parity check is sent, and when a parity failure is detected, the remaining (m - - 1 ) checks are sent and then correction is carried out. Other codes that can be punctured include the maximal shift register codes (Solomon and Stiffler, 1965), uniform convolutionai codes (Robinson, 1969), Reed-Solomon codes (Reed and Solomon, 1960), and a class of extended BCH codes (Davida, 1970). The Golay (23, 12) code is next shown to be puncturable. This, however, does not appear to be a general result. Consider theparitycheekmatrixH:
H=
110110000000000" "11010001 111 01000111001000000000 101 10100011100100000000 110 11010001100010000000 111 01101000100001000000 111 10110100000000100000 101 11011010000000010000 100 11101101000000001000 100 01110110100000000100 110 00111011000000000010 101 00011101100000000001 1
23
123
ERROR CORRECTION WITH FEEDBACK
Dropping column 7, we get the shortened (22, 11) whose parity check matrix H ' is
H'=
-110100! 110110000000000 111010011100100000000~ 1011010011'10010000000 110110000110001000000 111011100010000100000 111101010100000010000 101110101000000001000 100111110100000000100 100011011010000000010 110001101100000000001 101000110110000000000
.
Rows 1, 2, 3, 5 and 8 Of H' ~ o m the extended H a m m i n g (16, 11) binary code which can detect 3 or less errors. T h e parity check matrix H ~ %r the (16, 11) code is
H"=
[i0000000i] 11010011100100 01101001110010 11011100010001 00111110100000
.
Thus, we can initially store 6 parity che& bits and transmit with a rate of 11/16 instead of 11/22. When 3 or less errors are d a e a e d we can request the remaining symbols and proceed to correction.
IV.
ALTERNATE SCHEMES
T h e above scheme naturally leads one to consider the following variations, of which B is much easier to implement since it avoids the necessity of puncturing codes, which may not be readily accomplished: A. Let an (n, h) code which can correct t or fewer errors be punctured to an (n~, h) code which can correct e errors and detect t errors (e < t). T h e expected value of the rate for this scheme will be higher than the above mentioned one in cases where e or less errors are quite likely resulting in a high request rate for additional information. T h e remaining redundancy will be requested only when the number of errors exceeds e. 643/eI/2-3
124
DAVIDA AND REDDY
Then
E(R) = k/(ns q- r'PD).
(3)
B. Let (n 1 , k) be a code which can correct e errors and detect t errors, and (n2, k) be a different code which can correct t errors. Initially, the (nl, k) code is used. When more than e errors occur, the redundant symbols, computed according to the (ne, k) code, are requested and the redundant symbols at the (n I , k) code are discarded. For this case,
E(R) = k/In 1 + eD(n2 -- k)].
(4)
EXAMPLE 2. Consider the (5, 4) single-error-detecting binary code with one parity bit whose parity check matrix H~ is H 1-
[1 1 1 1 1],
and the H a m m i n g (7, 4) single-error-correcting binary code whose parity check matrix H e is H~=
0110~
.
10100 Initially, the (5, 4) code is used, and when a single error is detected, a request is made to send the parity symbols for the (7, 4) code and the parity bit from the (5, 4) code is discarded. T h e n
E(R) = 4/(5 q- 3PD), P g = (~) Pq4 + (53)P3q2 + (55)p5 : 5pq~ + lOpZq~ + pS; therefore
E(R) = 4/(5 + 15pq4 + 30pSqz + 3p5). Using two codes, one for detection and the other for correction is more readily accomplished since the two codes are not necessarily related. This leaves one with greater flexibility in choosing the two codes.
V. COMPARISON WITH RETRANSMISSION Let us consider the simple retransmission scheme where an (n, k) code is used for error detection over a noisy channel. When an error is detected,
125
ERROR CORRECTION W I T H FEEDBACK
a request is made via a noiseless feedback channel for the repetition of that erroneous block. We will consider here the scheme where the number of repetitions is unlimited. Consider the Binary Symmetric Channel.
The expected value of the rate for the simple retransmission
THEOREM 3. scheme is
E(R) ~- (1 -- eD)h/(n~ + 1), where PI) is the probability of detecting an error. Proof.
Let M ~ the number of words transmitted;
then E(number of blocks repeated once)
= MPD,
E(number of blocks repeated twice)
~ MPD 2,
E(number of blocks repeated N times) = MPD N, E(total number of bits transmitted)
= M(ns @ 1) @ M(n~ q- 1) PD
+ M(n~ + 1) PD ~ "1
= M(n~ + l) I - - P D 1
= M(n~ q- 1) 1 M • E(R)
=
M(n~ +
1)[1/(1
-- PD)] =
k(1 - -
-
-
PD
P~)/(n~ + 1).
Q.E.D.
The probability of error for the simple retransmission scheme is
THEOREM 4.
P~ = (1 -- PD -- q~)[1/(1 -- PD)]"
P~oof. P, ~ P(undetected error) + PD[P(undetected error)]
q- (PD)n[p(undetected error)] q- ---, V~ = (I -- PD - - q~)(1 + PD q- PD 2 -k "") 1 =
-
Po -
¢o
( 1 -
P o )
Q.E.D.
126
DaVIOa AND
REDDY
The expected values of the rate for the shortened and punctured Hamming codes for both the proposed scheme and retransmission are shown in Figs. 4-6 for values of n s of 4, 128, and 512. The word error probabilities are shown at selected points. The corresponding word error probabilities are shown in Figs. 7-9. In the case of the proposed scheme, when an error is detected, the remaining r ' parity bits are requested (only once), whereas in retransmission the whole code-word is repeated. There is no significant difference in the probability of error between the two schemes. Significant difference does exist in the rate, where the proposed scheme.is superior. When two codes are used, that is, one for detection and the other for correction, retransmission has a lower probability or error, as shown in Figs. 10-12. But this is achieved at a significant reduction in the rate, as in Figs. 13-15, where again the word error probabilities are shown at selected points.
1,0 O
i 0
~0
X
,80 -:
ILl
X
(4~3):
r'=
2
~'-.0
.60
.40
~D
-- -- -- PROPOSED
eJCHEME
x
o)
p-
.20
RETR/~.NSMISSION
,1,
,I,
,I.
~o-7
:6s
~o
-5
, i . . I . , -4
,o
-3
io
I, -2
io
, I . . i - |
Io
I
P
FIG. 4. Comparison of the proposed scheme (single-error correction) with retransmlssion (single-error detection). The word-error probabilities are shown at selected points.
127
ERROR CORRECTION WITH FEEDBACK N IO X 1,0
U2 8112T}, r = 7 ,
o-.~ -o--
(128,127)
%
.80
W
-6--.o
.60
I-<~ n.-
o~
..;
.40
- - - -- PROPOPOSED SCHEME
.~0
RETRANSMISSION
, I ,
,~7
.
I
~;6
.
.I
65
.
. I . . I
,;4
.
,;3
.
I,
,~
.
I.
I~1
.I
,
p
FIG. 5. Comparison of the proposed scheme (single-error correction) with retransmission (single-error detection). T h e word-error probabilities are shown at selected points.
V I . CONCLUSIONS
As expected, the above scheme performs better than forward-error correction. It was seen that the probability of error for the proposed scheme is slightly higher than the simple retransmission, but that it has significantly higher rate. Since the number of requests for additional redundant bits is limited to one, the overflow problem has been considerably reduced. It is expected that for equal rate or equal probability of error the proposed scheme will perform better than the retransmission scheme in the probability of error and the rate, respectively.
L~IZ,6II~I,
hO
~',l)
.-.
o
{512~
.00
-
..:
..:
-;
0
I.l..J , 6 0 I-l0 .4.0
--
- PROPOSED
.ZO
SCHEME
R ETRANSMISSION
FIO. 6. Comparison of the proposed scheme (single-error correction) with retransmission (single-error detection). The word-error probabilities are shown at selected points. |,0 m
CE 0 BE re" LLi LL.
0 >.. I-._l CO CO
0 0:: 0..
- - -- -- P R O P O S E D
DE 0
/
• I
.
•
I.
~
•
I
.
•
I
•
•
SCHEME
RETRANSMISSiON
i.
I
•
•
I
,
•
I
FIc. 7. Comparison of the proposed scheme (single-error correction) with retransmission (single-error detection).
1.0
cr O r~ r~ L~
'T
LL 0 >t---
-n <~ m O n c~ rY O
[~"
i
I
,
td 7
.
I
,
~d~
,
I
,
,o-5
.
I
,
jo-4
,
I
i
,d ~
,
I
,
,d ~
.
I
,
jo"~
I
I
I.
P Fie. 8. Error probability for the proposed scheme (single-error correction) and retransmission (single-error detection).
1.0
O no a:
"9
, 63
Lfl
I,L 0 >I---
,6 ~ m <~ m O rr ~-
169
l'Y
O
.I,
j6 r
, I,
io "~
.I
j6 5
...... I ,
,~
,I,
i~ 3
. ~ . . I . ~
~6 2
16 j
I.
P
FIG. 9. Error probability for the proposed scheme (single-error correction) and retransmission (single-error detection).
hO
/ ....
PROPOSED SCHEME
/ t
n. 0
I
RETRANSMISSIO N
I l
O~ W 0 >_
I
t I
r'~
I I
iI
I
!
'1.
Jd ~
t
0 r~. n
Ir
i/ (2~11)
.J m m
/I
(I~,H)
l~ c
I
I
L~'~
.
I
I
0
I I I .I
. ~.
,d r
,.I{.
,o-~
o
Jl,
,d 5
.I,
td ~
. i . . I , . ,
,d J
,d ~
10-I
~
p
FIc. 10. Comparison of the proposed scheme (3-error correction) w i t h retransmission (3-error detection). 1,0 -----PROPOSED
rr O tic
tr
SCHEME
//
' RETRANSMISSION
/
/
l0 3
W ta. O >_
,
10~
I-_I m
I
cn 0 n~ [I.
(128j12o) I /
|
I
I I / /
n" 0 -i$ I0
•
L6r
•
.
i~
•
|
~5
.
•
•
•
•
16+ ,63 ~62
.
i
i~i
i
|
I.
FIo. 11. Comparison of the proposed scheme (3-error correction) with retransmission (3-error detection).
1.0 .
rr 0
I
,
i//
-3 I0
(612~5 02). /
rr" LLI LL 0 >,.. I'._1
I
/
-CD Io
(33
-s
IO (33
0
C3 (3:: 0
It)
----
-PROPOSED
5CHENE
RE T R A N S M ~ S S ~ O R .15 IO
,
I
t67
,
.
I
.
166
.
I
,
165
.
I
,
.
I ff~
I
.
,
I d3
1
,
i62
.
I
,
,I
i6~
I.
p
FIC. 12. Comparison of the proposed scheme (3-error correction) with retransmission (3-error detection).
I.O
b'O
.8
(I 6,il),(26,11)
.~.
t _O
.4 LIJ
!'or .3
t67
io-e
1if5
164
,-~ I0
I0-2
I I0-~
I
P
FIG. 13. Comparison of the proposed scheme (3-error correction) with retransmission (3-error detection). T h e word-error probabilities are shown at selected points,
IO
(12811g0) f,144~12 O)
hO
-
x ~
'o
.. : .8 i
.6 LLI l--
n.* .4
.2
'" RETR& NS MISS|ON
,
I
.
Id 7
,
I
,
16 e
,
I
,
~6s
,
I
.
164
~, ,
I
,
~6~
.
I
"m"
,~,..-J --
16 2
Io"~
0 "~I
i.
P
FIG. 14. Comparison of the proposed scheme (3-error correction) with retransmission (3-error detection). T h e word-error probabilities are shown at selected points. N
1.0 _
,,~.=:<
IO
(512~502) 153Z~502)
X N
.,~'6ix .,.
,l.,,..,.
i~7
I(~6 ib5
i~4
,;
,,..~;:
~
16~
i~I
|~Z
~,
I
p
FIG. 15. Comparison of the proposed scheme (3-error correction) with retransmission (3-error detection). T h e word-error probabilities are shown at selected points.
ERROR CORRECTION WITH FEEDBACK
133
ACKNOWLEDGMENT The authors would like to thank Professor John P. Robinson for the numerous discussions and helpful suggestions. They would also like to thank the reviewers for correcting errors in an earlier version of the manuscript. RECEIVED: M a y
6,
1971
REFERENCES 1. BENICE, R. J. AND FREY, A. H. (1964), An analysis of retransmission systems, IEEE Trans. Commun. Technology CTI2, 135-145. 2. BERLEKAMP, E. R. (1968), Block coding for the binary symmetric channel with noiseless delayless feedback, in "Proceeding of the Symposium on Error Correction Codes at the University of Wisconsin," Wiley, New York. 3. BEt~LEI~AMP,E. R. (1968), "Algebraic Coding Theory," McGraw-Hill, New York. 4. DAVmA, G. I. (1970), Multiple extension of the minimum distance of BCH codes, in Proceedings of the Eighth Annual Allerton Conference on Circuit and System Theory. 5. FANO, R. M. (1963), "Transmission of Information," M1T Press, Cambridge, MA. 6. GraVEN, P. E. (1961), Feedback communication systems, in "Lectures on Communication Systems Theory" (E. J. Bagdady, Ed.), McGraw-Hill, New York. 7. MASSEY, J. L. (1963), "Threshold Decoding," M I T Press, Cambridge, MA. 8. PARK, J. H. (I969), Information rates and errors using decision feedback, IEEE Trans. Commun. Technology CT17, 20-24. 9. PETERSON, W. W. (1961), "Error Correcting Codes," M I T Press, Cambridge, MA. 10. RBED, I. S. AND SOLOMON, G. (1960), Polynomial codes over certain finite fields, J. Soc. Ind. Appl. Math. 8, 300-304. 1l. ROBINSON, J. P. (1969), Punctured uniform codes, IEEE Trans. Information Theory IT15, 149-152. 12. SCHALKWlJK, J. P. M. AND KAILATH, T. (1966), A coding scheme for additive noise channels with feedback: 1. No bandwidth constant, IEEE Trans. Information Theory IT12, 172-182. 13. SCHALKWlJK, J. P. M. (1966), A coding scheme for additive noise channels with feedback: II. Bandlimited signals, IEEE Trans. Information Theory IT12, 183-189. 14. SOLO~ON, G. AND STIFF~R, J. J. (1964), Algebraically punctured cyclic codes, Information and Control 8, 170-179.