A morse-code recognition system with LMS and matching algorithms for persons with disabilities

btmmtbmal

Journal of

Medical lnformatics ELSEVIER

International

Journal

of

Medical Informatics 44 (1997) 193-202

Original

article

A morse-code recognition system with LMS and matching algorithms for persons with disabilities Ching-Hsiang Department

of Electrical

Engineering,

Shih, Ching-Hsing National

Cheng-Kung

Luo *

University,

Received 23 July 1996; accepted 6 March

Tainan

70101,

Taiwan,

ROC

1997

Abstract

Single-switch communication is an effective auxiliary method for persons with disabilities. However, it is not easy to recognize the Morse codes typed by them. In our earlier proposed Morse code auto-recognition method, using the Least-Mean-Square (LMS) adaptive algorithm, it was demonstrated that the system could successfully recognize the Morse-coded messages at unstable typing speeds. However, the speed variation had to be limited to a range between 0.67 and two times the present speed. In the case of beginners or those with heavy disabilities, this rule can not always be complied with, producing a low recognition rate of 20%. To address this limitation, this paper offers an advanced recognition method which combines the Least-Mean-Square algorithm with a character-by-character matching technique. The recognition rate for this method from simulated and real data from various sources is as high as 75% or more on average. This practical application of the single-switch method means a step forward toward alternative communication for disabled persons. 0 1997 Elsevier Science Ireland Ltd. Keywords:

Morse code; Adaptive

signal processing;

Least-Mean-Square

algorithm;

Matching

* Corresponding author. Tel: + 886 6 2757575, ext. 62375; fax: + 886 6 2345482: e-mail: [email protected]

them should be designed or developed to improve their communication [ 1- 61. Among the many possible methods, Morse code communication could be the most effective, especially in cases of difficulty in typing with a keyboard (for instance, when fingers have been lost in an accident). Morse code is a simple and useful method for transmitting signals. All characters and digits are coded as a series of key pressing

1386-5056:97/$17.00

reserved.

1. Introduction

Because of physical restrictions, persons with disabilities have difficulty in effectively communicating with other people or environmental facilities. Therefore, communication methods and the auxiliary equipment for

9 1997 Elsevier

PIf S I386-5056(97)00020-2

Science

Ireland

Ltd.

All rights

194

C.-H.

Shih,

C.-H.

Luo /International

e,(x,)24(xsX %(X,)> M*,Xe,(x,+,) ,... Morse code digital stream

’

Journal

of’ Medical

Character Separating

Infirmaticv

3

44 (1997)

Matching Decision

193-202

+

A $ pJ

Space average

Fig. 1. Block diagram of the improved algorithm. It contains three sections: character separating, matching decision, and adaptive signal processing. The space between characters is first identified in the character separation section. Then the codes in the separated Morse character are regarded as a set of messages to be translated into its corresponding alphanumeric character in the matching decision section. To catch up with the typing speed, the average space in a character feeds into the adaptive processing (using the LMS algorithm) section, and the character separation section identifies the space between characters based on the processing output.

states: ‘dash’ (‘--‘) and ‘dot’ (‘. ‘). For example, ‘A’ is coded as ‘. -‘; ‘B’ is ‘ - . . . ‘. The standard length ratio of ‘dot’ to ‘dash’ is one to three. If the switch is unpressed, one standard ‘dot’ length represents a space between ‘dot’ and ‘dash’; three standard ‘dot’ lengths represent a space between characters; and seven standard ‘dot’ lengths represent a space between words. However, automatic recognition of the Morse code typed by the disabled is not easy due to their difficulty in maintaining a stable typing rate [1,2]. In 1996, Luo and Shih [l] proposed a system that could recognize varying typing speeds using an adaptive technique, the Least-Mean Square (LMS) algorithm. Their proposed system could adjust its characteristics to successfully recognize a message under unstable typing conditions, but the speed change between characters had to be limited within the range (that is, the typing wh < nk + 1 < 2nk [l] speed from one character to another could not be raised by a factor of 1.6 or slowed down to less than half that of the preceding

character). To satisfy this limitation, the disabled person had to be well-trained or the system would not successfully recognize his Morse code message. However, this limitation could not always be complied with by the beginner or those with very serious disabilities; therefore, the proposed method could not be effectively used by them. To address this limitation, an advanced method which combines the Least-Mean-Square algorithm and the matching technique is proposed here. 2. Recognition method The Morse code element may be a ‘dot’ or ‘dash’, depending on the pressed time length; it also may be a space between a ‘dot’ and ‘dash’, between characters or words, depending on the released time length. To distinguish these five conditions (i.e. dot, dash and three types of spaces), a decision rule based on the LMS algorithm output was established in the previous version of this paper [l]. The system adjusted its recognition char-

C.-H.

Shih,

C.-H.

Luo /International

Journal

acteristics only according to each Morse code ‘dot’. Therefore, the speed change between characters had to be limited to the range (2/3)h < 4+ I < 2~ [l]. In this paper, we present a new method for recognizing Morse code. It processes the data character by character, as opposed to the Morse code element ‘dot’ processed in the previous version. The improved algorithm of the automatic recognition is shown in Fig. 1. There are three major sections in the paper: one dealing with character separation, one with matching decision, and one with adaptive signal processing. The space between characters is first identified in the character separation section so that the codes of an unknown character can be regarded as a set of messages to be translated into its corresponding character in the matching decision section. The

\

q(xJ

/ 4

No-+

Save j=

e,(x,) j+l

44 (1997)

193-202

195

the duration ofjth Morse code element of the input character xi; as a key is pressed. It could be ‘dot’ or ‘dash’. the duration ofjth space of the input character xi; as a key is released. It could be one of three spaces: the space between Morse code elements, the space between characters, or the space between words.

According to the definition above, a Morse code character xi can be represented as xi = (el(xi), bl(x;).*.ej(xi), bj(xi),... bn- l(x;)e,(xi>). 1
Yes

A character is separated, go to next character separation

Fig. 2. The flow chart of character

Informatics

matching decision [7] calculates the Euclidian distance between the codes of the unknown character x; and the codes set in the Morse code table. The character in the table which has the minimum Euclidian distance is picked up as the unknown character. To catch up with the typing speed, the average of the space length in a character feeds into the adaptive processing (using LMS algorithm) section, and the character separation section identifies the space between characters based on the processing output. The following are the definitions of symbols used for describing the algorithm in detail.

bj(xi)

j=l

I

of Medical

ai separation.

the average of spaces bj(xj) (j = ln- 1) in xi is the ith input date of the LMS algorithm.

196

C.-H.

I e,(x,) : -

Shih, C.-H.

ez(x,) --I-

e,(x,)

Luo /International

! el(xg+l)

Journal

eAx,+, 1

of’ Medical

Informafk

e3(x,+,)

44 (1997)

e,(x,+,)

193-202

:

4(x,+,

4(x,)

1

Fig. 3. An example of typing speed variation outside of the range (2/3)n, < nk+ , < 2~. x, (’ - .) represents the character ‘R’, while x, + , (. - - -) is ‘J’ in which the lengths of each element (e,, (xi+ ,), e,(x,+ ,), e3(xi+ ,), e,(x,+ ])) are twice as long as in x,. ai-] i

ai = 1

n-1 1 i=

I’

bj(X;)

I

for y1= 1 for y1= (2, 3,4, 5)

n-1

S; the ith standard ‘space’ length as predicted by the LMS algorithm. Si = y-] =y;z-’ (1) and yi is the ith LMS output. 2.1. Character

separation

The purpose of character separation is to identify the space between characters so that the Morse code elements of a character xi can be isolated. Suppose that a stream of input signals in characters xi ~ ,, xi, xi+ ,,. . . are &-A b,(xi- 11, e&c), hh),... e,(xJ, b,(x,), eI(xi+ ,), b,(x;+ ,) ,.... The character separation section must identify the spaces between characters (i.e. b,(x;- i) and b,(x;)) in order to isolate e,(x,),. . . e,(x,) of a character x,. The flow chart of character separation shown in Fig. 2 is described below: 1. Reset the index j = 1 2. Check whether bj(x;) 2 2S, or not. If yes, jump to step (4). 3. Let j =j+ 1 and save
4. Check whether b,(x;) > 5S, or not. If not, jump to step (6). 5. This means the end of a word. So, print a ‘space’ on the screen.

6. The key-pressed Morse elements in xi character are ej(xi) for j = 1 - yt, 7. go to step (1) to separate the next character, if any. As for the first character xl, it cannot be isolated immediately because the initiai value S, is lacking. Therefore, to recognize the first character, the system first collects ten spaces, picks out the four minimal values for them, calculates the average value for these four minimums, and assigns the calculated average value as the initial standard space length Si. Once the key-pressed Morse elements in a character are identified, they are sent to the matching decision section for character recognition.

2.2. Matching

decision

A set of Morse code elements in an unknown character is compared to the standard sets listed in the Morse Code Table (Table 1) so that this unknown character can be recognized; this procedure is called matching [7]. The matching method used here calculates the Euclidian distances between the unknown character xi and the known characters in the Morse Code Table. The character in the table that has the shortest distance is chosen as the unknown character. *’ In Table 1, there are 36 sets of Morse code elements representing 26 English characters and 10 digits. The set in the mth column of

~!W< E (1) I (1.1) 9 (1.1.1) H ~l.l.l.lJ 5 ~1.1.1.1.1)

N (3.1) R (1.3.1) F f1.1.3.1J 3 (1.1.1.3.3)

A

3

(I.71 II ll.l,3) ” (1.1.1.3) 4 ~1.1.1.1.3)

T (3)

2 -

M (3.3) W (13.3) I. /1.3.1.1J 2 (1.1.3,3.31

4

K (3.1.3) J (1.3.3.3) 6 (3.1.1.1.1)

6

7

8

9 ---

IO

II

I?

G 0 (3.3.1) (3-3.3) B x c Y Z Q (3.1.1.1) (3.1.1.3J f3.1.3.1) (3.1,3.3) (3,3,1,1) (3.3.1.3J 8 9 0 7 O.?.l.l.l) (3.3.3.1.1) (3.3.3.1.1) (1.1.3.1.3) -___ drg~s m the Morse Code Table Row n (n = I -5) INS the characters which have II Morse code elements, and columtl w, describes the a character has two Morse elements (n = 2). it could he one of four characters. 1. A. N. M. l-he set in the mth column of ntb row 1s I, 1) represents the character ‘II’ which has three Morse elements (’ -). The data of each set xn the table IS obtamed from the standard 1s I-3

D (3.1.1) P (1.3.3.1) I (1.3,3.3.3)

<

-__---. -~ There are 36 SW 01 Morse code elements representmg 26 En&h char;rcters and ten number of all pooalble characters which have n Morse elements. For example, when denoted iis I,,,,, representmg a correspondmg character or dqt. For exam~k ,,? = , I, Morse code defimtion Consequently. the length rat,” 01 ‘doi to ‘dash’ \n a character

5

4

3

2

I

I

198

C.-H.

Shih,

C.-H.

Luo jlnternational

Journal

the nth row is denoted as t,, representing a corresponding character or digit. For example, tx2 = (1 ,1,3) represents the character ‘U’ which has three Morse elements (. . - ). By calculating the degree of match between the unknown character xi and each of the 36 sets in the Morse Code Table, the character xi can be determined. The procedure for the matching decision is as follows: 1. Find out Min(ej(xj)), the minimum value of eJx,), j = 1 - n, in xi 2. Normalize ej(x,) with the Min(ej(xi)), ej(Xi)

=

ej(xi) _ Mln(e,W)

forj=

1 --n

(1)

3. Find out all Euclidian distances d, by calculating the roots of the sum of the square distances between xi and the subset t,,,,,

for

m =

1 -f(n).

Informatics

44 (1997)

193-202

previous version [l]. It is also used here except that the system input (a,) is the average of the spaces bj(xi) instead of
(2)

where f(n) = [2,4,8,10,12]. 4. Find Min(&,), the minimum value of The character dim, for m = 1 -f(n). which has the best matching degree (i.e., the shortest Euclidian distance (Min(d+,,))) is picked as the unknown character. Besides the character recognition, the average of spaces b,(x;) in x, is also selected as the input (ai) of the LMS algorithm. This is completely different from the previous version [l] in which e,(x,) was selected as the input of the LMS algorithm. 2.3. Aduptive

of’ Medical

processing

The excellent performance of an LMS algorithm with nine weights was verified in the

In previous research [I], four testing protocols with a 1:3 ratio of ‘dot’ to ‘dash’ within a character generated in a random manner and a human typing pattern from practice were applied to test the performance of the system. From the conducted results, it was determined that the system could successfully recognize Morse code messages under an unstable typing condition as long as the speed variation was limited to the range (2/3)n, < nk+

I <2nk

[Il.

In this paper, to see whether the system performance satisfied our requirements, an extensive testing protocol on simulated and real data from various sources was used. The experiments were conducted on an IBM PC486. First, the same protocols as in the previous paper were tested to verify a better performance than the previous.

C.-H.

Shih, C.-H.

Luo ~International

.

I

of Medical

Journal

I...

1.1,

A

I..

A

Informatics

.

A

A

44 (1997)

..I.

199

191-202

I. A

A

: :

251,785,563,1265.442.662.257,3OZ,l9O,l94,Zl7,lO9l,403.657.197,469,516,5O5,2l3ms,.

-the ‘0 M B

600

%a g 3

400

s

200

E” i=

average of spaces in x, ..- adaptive-processing Output

800

0 0

10

20

30

40

Typing character

50

60

70

80

number

Fig. 4. The system performance in character separation. These Morse-coded messages were typed by a person who had lost both hands in an accident. Two stumps instead of hands were put in place to type the Morse codes. (a) Part of the typing message and its corresponding space lengths are provided. The dotted arrow indicates the spaces between characters or words that are decided by the character separating section. (b) The adaptive-processing output S, (dashed line) can closely follow the average of spaces in x,.

3.1. E,xperiments on the same protocols the previous paper

used in

In this testing, all recognition results were completely correct even under the following conditions: Fast suddenly up-down speed change outside the (2/3)n, < n, + , < 2~ limitation, where the previous system could not produce a correct recognition. For example, as shown in Fig. 3, xi(. - .) represents the character ‘R’, while xi + ,(. - - -) is ‘J’ in which the lengths of each element kdxj+ A e2Cyi+I>, e3(xi+ ly e4(xj+ d are not only longer than in xi but in which the speed change xi to xi+, is also out of the range (2/3)n, < nk+, < 2n, (i.e. e,(x;+ ,) > 2e,(xj)). As a result, a ‘dot’ (e,(x,+ ,)) in the

previous system could be mistaken for a ‘dash’. The present system processes the Morse code message character by character, instead of processing Morse code element ‘dot’ as in the previous version. Therefore, the character ‘J’ can be recognized correctly due to the matching technique. In the next section, experiments on real data from various sources are shown; all of the testing patterns are outside the limit of (2/3)nk < nk + , < 2nk. 3.2. Experiments sources

on real data from

various

Fig. 4 shows the system performance in character separation. These Morse-coded

C.-H.

200

Shih,

C.-H.

Luo /International

Journal

messageswere typed by a person who had lost both hands in an accident. Two stumps instead of hands were put in place to type the Morse codes. Part of the typing message and its corresponding time length are provided in Fig. 4(a). The dotted arrow indicates the spaces between characters or words that were decided by the character separating section. Fig. 4(b) shows how the adaptive-processing output Sj (dash line) adaptively follows a, (the average of spaces in Xi); it is clearly indicated that Si can closely follow the input aj. In this experiment, the recognition rate of the previous system was as low as about 15%. In contrast, the improved system can still successfully separate Morse code characters with an average successful recognition rate as high as about 72%. Character separation error

m

m 780

m 481

578ms

(9 Matching decision error 4 m

m

230

233

m 433

223ms

@I Fig. 5. An example of results in incorrect recognition. (a) Character separation error. (b) Matching decision error.

of’ Medical

Informatics

44 (1997)

193-202

Two main errors are supposed to be responsible for the incorrect recognition by the improved version: character separation errors and matching decision errors. Character separation errors are created by the space between ‘dot’ and ‘dash’ within a character having a length longer than two standard space lengths so that it is mistaken as the space between characters. For example, the length of the first space inside a character is 780 ms (see the double-arrow in Fig. 5(a)). This space length is greater than two standard space lengths (370 ms). Hence it can be mistaken for the space between characters, instead of a space between ‘dot’ and ‘dash’ in a character. This mistake can result in a character being split into two characters. Once an incorrect character separation is generated, the matching decision of the next few inputs is also affected. The decision error is closely related to the typist’s personality. For example, the time lengths of four Morse elements in an unknown character in Fig. 5(b) are 230,233,433,223 ms. After normalization with the minimum time length (i.e. 223 ms), the length ratio among these four elements is 1.03: 1.04: 1.96: 1. Apparently, the matching decision unit recognizes it as 1:l :l:l (i.e. as the character ‘I’) instead of 1:1:3: 1 (i.e. as the character ‘F’), which is intended by the disabled. Hence, an incorrect decision is generated. To solve this problem, we can adjust the length ratio in the Morse Code Table with a format specially designed for the disabled to be more germane to their individuality. This can increase the recognition rate. Seven hundred characters of Morse code were collected from seven individuals, each of whom typed the same 100 characters. The recognition results for set 1 to set 7 are summarized in Table 2. The ‘test patterns’ columns show the individuals’ typing speed (WPM). Among the test patterns, the highest possible speed was 16 WPM (words per minute) for typing, while the lowest speed

C.-H.

Shih,

C.-H.

Luo /International

Journal

of Medical

Injiwmatics

44 (1997)

193-202

201

Table 2 The recognition results for set 1 to set 7 Test patterns Set No. No. No. No. No. No. No.

1 2 3 4 5 6 7

Experimental results in success rate Typing speed (WPM)

Previous system (“XI)

Proposed system (“%I)

10 16 15 9 13 16 12

23 18 33 11 16 20 23

70 87 90 67 73 70 69

The ‘test patterns’ columns show the typing speed (WPM). The ‘experimental results in success rate’ columns are the success rate for the previous system and the proposed system.

was 9 WPM (note that a word has about five characters on average). After processing these seven tests of irregular Morse code, a 75% successrate on average was obtained for the proposed system.

4. Conclusion

Morse code communication is an easy and effective method for the disabled to communicate with other people. However, automatic recognition of the Morse code typed by the disabled is less than successful due to their difficulty in maintaining a stable typing rate. In the previous research [l] we applied the adaptive LMS algorithm to recognize Morse code messages typed by the disabled, and its performance was verified in a simulation situation. The designed system could successfully recognize the message under an unstable typing condition, but the speed change between characters had to be limited within the range In this paper, a new 12i3h < nk+ I < 2nkrecognition method which combines the LMS algorithm and a matching technique is proposed. From the results, we conclude that the improved system can better recognize irregular typing than the previous version. The

major difference between these two systems is the separation process. In the previous version, ‘dot’s and ‘dash’s in a character were identified element by element, while in this improved version ‘dots and ‘dashs in a character are regarded as a set to be identified. Even though both versions adopt the restricted Morse rule, the improved version has a significantly improved recognition capability in comparison with the previous version. The idea of processing data with a whole character imitates the human auditory recognition system. Based on our tests of experts, they always recognize the character by the combination of its elements. This system was built on a PC-486 computer because it requires a lot of computation time. To make this equipment portable, digital signal processing (DSP) chips for fast calculation may be used to shorten the computation time, and the system can be implemented with a single-chip microprocessor. In future development, the length ratios in the Morse Code Table may be made adjustable with intelligent algorithms such as a neural network, so that the system can adapt itself to human personalities; this could increase the probability of success in recognition.

202

C.-H.

Shih, C.-H.

Luo / International

Journal

References [l] C.-H. Luo, C.-H. Shih, Adaptive Morse-coded single-switch communication system for the disabled, Int. J. Biomed. Comput. 41 (1996) 99-106. [2] C.-H. Shih, C.-H. Luo, Adaptive single-switch communication system for the disabled, Proc., J. Biomed. Eng. Appl., Basic Commun. (1994) 551556. [3] L.J. Mowinski, H.L. Lauer, The adaptive firmware card expands computer access to a blind bilateral hand amputee, OCLC Micro 5 (1989) 23-24.

qf Medical

Informatics

44 (1997)

193-202

[4] B. O’Neal, Technological solutions for individuals with disabilities, Media Methods 29 (1992) 34-35. [5] L.T. Hauck, SAM: An improved input device, Johns Hopkins APL Technical Digest 13 (1992) 490-493. [6] J.J. French, F. Silverstein, A.A. Siebens, Inexpensive computer based Morse code communication system, RESNA ‘86, 1986, pp. 259-261. [7] R.C. Gonzalez, R.E. Woods, Digital Image Processing, Addison-Wesley, Reading, MA, 1992.