On-line character recognition using handwriting modelling

Pattern Recognition, Vol. 26, No. 7. pp. 993 I(X)7, 1993 Printed in Great Brilain

(~131 3203/93 S6.00+.00 Pergamon Press Ltd c~ 1993 Pattern Recognition Society

ON-LINE CHARACTER RECOGNITION USING HANDWRITING MODELLING BRUNO SIMARD,t BIRENDRA PRASADA~and R. MAHESH K. SINHA§ INRS T616communications,16 Place du Commerce, lle-des-Soeurs, Verdun, Qu6bec, Canada H3E I H6 (Received I February 1991; in revisedJorm 18 September 1992;receivedlor publication 5 January 1993) Abstract An oscillatory model of handwriting, previously used for synthesis, is modified. The modified model better suits the dynamics of handwriting. It also provides parameters which can be used for on-line handwritingrecognition.A user-dependentsystem,operating on segmentedhandwritten lowercase characters yields a recognition performance exceeding90~, for cooperative users. Handwriting characterization Oscillatory handwriting model

On-line

Characterrecognition

1. I N T R O D U C T I O N

Direct on-line data entry devices offer a natural means of providing man machine interfaces. Optical page readers with some constraints are now commercially available. However, recognition of cursive and handwritten scripts still pose problems for researchers/' 6~ An on-line recognition system can make use of information regarding speed, tracing sequence, number of strokes, etc., that are not available in off-line OCR. However, the interactive environment has problems of its own regarding learning algorithms, users' adaptability, hardware complexity, etc. Several attempts have been made in this direction in the pastJ 3-1s~ Burr ~s'l 1~ has used a correlation-based shape matching algorithm for character comparison and a word dictionary for resolution of character confusions. Tappert ~5~performs experiments using word recognition rather than isolated character recognition. Tappert's system does not break words into letters in an explicit fashion. Rather, it dynamically considers all possible segmentation points for letters within a word, while also dynamically considering all warping paths for letters within this segmentation. Several researchers/12'~9 30~ have studied the process of handwriting generation through modelling and in certain cases came up with some distinct characterizations for different character shapes. Model-based recognition articulates the generative process and hence is a powerful technique. In this paper, we have modified the oscillatory model of handwriting~2a~for characterizing shapes. A shape descriptor for handwriting is derived from the analysis model. This shape descriptor is the relative phase difference between horizontal and vertical oscillations in

+ Presentaddress: Bell-NorthernResearch,Ottawa, Canada. ++Also with Bell-Northern Research, Montreal, Canada. Author to whom all correspondence should be addressed. § Present address: Indian Institute of Technology, Kanpur, India.

Warping

Learning prototypes

the handwriting. It is observed that the relative phase difference leads to a better shape characterization as compared to the conventional (pen tip) position and velocity information. One benefit of our analysis model is that it provides a method of time normalization for the character prototypes, therefore allowing for efficient prototype comparison without dynamic programming. A continuous learning process is invoked for acquisition and modification of the prototypes. Our system is user dependent and does not handle cursive script segmentation explicitly. The system has a recognition performance exceeding 90~i for a cooperative user. In Section 2, we look at the oscillatory model of handwriting. An extension of this model to suit our application is presented in Section 3. In Section 4, we examine the details of the recognition system. Experimental results are presented in Section 5.

2. T H E O S C I L L A T O R Y M O D E L

Most people write with the joint of the wrist for movements in the horizontal direction and joints of the thumb and forefinger for movements in the vertical direction. Most left-handers reverse the roles of wrist and fingers so as not to cover fresh handwriting with their hand. Although handwriting movements are not always so easily reduced, it is an interesting fact that a person can write his own characteristic handwriting with the help of two joints, permitting movements in two more or less perpendicular directions. Van der Gon and Thuring~27~give substantial evidence of this physiological orthogonality. It is an accepted working hypothesis that the hand can be factored in two orthogonal degrees of freedom for the handwriting process.tt 2,27-20.31.32) In each of these directions, muscles can be modelled by a pair of opposing springs to represent agonist and antagonist muscles. In Fig. 1 (Hollerbach ~2s~)the g indices refer to agonist muscles and the n indices to antagonist. Note how the y springs

993

B. SIMARDet al.

994

+

~1 \ k , / # m~

xn

+

\V\mv/

k2

kl (2t

Yn

Rn, x

kg,x

where a and 0 are constants. Since the solution contains as many arbitrary constants as the degree of the differential equation, this solution must be the general solution. The vertical motion equation can be formulated in the velocity domain by differentiating equation (2) once

Xg

g,

1

)(t)=osin(./(kll(t-to,+O). \ ~l k m y /

Y9

Fig. 1. Spring-mass representation of the model (Hollerbach~28~). Description of the symbols used: rn~, mass of the hand and pen being subjected to horizontal movements; mr, mass of the fingers and pen being subjected to vertical movements; x, position of one end of the spring representing the agonist/ antagonist muscle in the horizontal movement; y, position of the one end of the spring representing the agonist/antagonist muscle in the vertical movement; k, rigidity of the spring representing the tension in the agonist/antagonist muscle in the horizontal/vertical movement; g, agonist muscle; n, antagonist muscle.

act only on the pen mass my, whereas the x springs act on the pen mass and on the y springs platform. For the horizontal movement, a constant velocity sweep is superimposed on the oscillation. Yashuhara ~81 describes further the anatomy of the hand, attributing horizontal motion towards the right to the muscle Extensor Carpi Ulnaris, and motion to the left to the muscle Abductor Pollicis Longus. Vertical motion cannot be attributed so directly to specific muscles, as there are several joints in the fingers that combine their efforts in a synergistic action that results in the vertical movement. Nevertheless, their combined action can be modelled by one pair of springs, much like a parallel/series combination of springs can be modelled by just one spring with a larger/smaller elasticity constant. An analysis of the forces exerted by the spring and the resultant acceleration yields the following equation for the movement in the vertical direction:

mr~(t ) =

kg,r(yg

-

y(t)) - k,,y(y(t) - Yn)"

The horizontal motion equation follows a similar development, with the exception that a constant velocity sweep is superimposed on the oscillation to account for the fact that we write from left to right across a page. This brings us to Hollerbach's formulation ~2~ of the model ~(t) = a sin (Wx(t - to) 4- dpx) 4- c 9(t) = b sin (wy(t - to) + (Oy).

(3)

Choosing w~,w r in the mathematical description is equivalent to choosing the elasticity constants of the springs and weight of the masses in the mechanical representation. Choosing a, b, ¢k~,0r in the mathematical description is equivalent to choosing the initial position and velocity of the pen mass in the mechanical set-up of Fig. 1. This implies that simple character shapes can be generated by the initial set of parameters as the only source of control over the model. Eden cz21showed that an elaborate control can produce complex character shapes encountered in handwriting. Eden states that the generative theory also suggests that the sequence of handwriting segment and the parameter settings that produce function segments can be determined uniquely for handwriting. However, he has stopped short of describing a process by which the parameters could be determined. Eden's handwriting experiments were all performed by having an operator vary the relevant parameters until he was satisfied with the output. It is noted that no attempt was made to program an algorithm that would generate automatically the model parameters that correspond to a given handwriting sample. To date, no record has been found of attempts at this task, leaving a lacuna to the model.

By simplifying we get mr g(t ) + (kg,r + k,,r)y(t ) = (kg,yyg + kn,yYn)

myra(t) + k l y ( t ) = k 2

3. EXTENSION O F THE OSCILLATORY M O D E L

(1)

where k 1 = kg,,v 4- kn, r and k2 = kg,yyg + k.,yy,. We now have in equation (1) a second-order linear differential equation. This is in agreement with Yashuhara's statements ~8~ that the hand be considered a second-order system. Let us j u m p directly to a solution of this differential equation, as the intermediate steps are not of interest:

Let us summarize the above outlined handwriting model and extraction of its parameters (see Appendix A)

Yc(t)=axsin(wx(t-to)+Ox)+b

x

(4)

where t o is an arbitrary constant and a,, wx, q5x, bx are functions of time whose solution for time t = to +

On-line character recognition (n - 1)At~ is found by A

s, =

X n ÷ t --

Xn

At~

w . , = 2 f , sin '

~ _~(t)J..... + (n -- ljAt~

(5.1)

[j( 4\A's.JJJ

(5.2)

function q~x is different from q5x. The parameters ax, wx, b~ can be evaluated exactly the same way as for equation (4). In Appendix B we have derived the formula for extracting 0,, which is given as argx,,

1 )AZs" 4sin 2(w~,A&/2))

b~=s,+

995

tan-

(5.3)

,[s,+l+s,-2bx_.tan(W~At~],.

k

A ts.

\

2

)J

0., = a r g x , - ( i ~ 2 Wx,)At~. qS.,,,= wxo(n - ;)At~

3,1. The siyngficance of 0.~ - 4)y +tan

t[s""+s"-2h""tan(Wx'2AtS)]A's,, (5.4)

/((Sn+l_kSn__2bx.~2(

AISn ~2~

a.,,,= ~,/ \\2coslwx, Atj2)/ +\2sintwx,,Atj2)/ ) (5.5) where .l~ denotes the sampling rate; At, the sampling interval; x, the nth sample of parameter X(t); s, the approximation of horizontal velocity at the nth sample instant; A"s, the nth order difference of s,. The motion in the ),-direction is symmetrical to that in the x-direction

);'(t) = a s. sin (w~.(t - t o) + qSy) + be,.

(61

Note that the parameter b~, is incorporated for pure symmetry, as it is not obvious that this parameter is equal to zero. In Appendix B we have illustrated the inadequacy of this model for characterizing when the writing frequency is not held constant. In order to take care of the variation in writing frequency, a new formulation for the oscillatory model is given as .'(-(t)= ax sin

)

wx(u)du + ~bx +b~. k to

('(t):aysin(t,!wx(u)du+


(7a)

(Tb)

In Appendix B we also show that the formulation of equations (4) and (7) is very similar in the sense that they are locally equivalent. The two formulations being locally equivalent, the justification and description of the model {in Section 2) still holds. Furthermore, the formulation of equations (4) and (7) become identical when wx and %. are held constant in time. For generating synthetic writing, the oscillation frequency can be held constant and therefore the previous formulation was adequate for this task. But to characterize handwriting, the new formulation is more appropriate as we cannot hold the oscillation frequency constant when we write, and the model must reflect this. Equation (7a) can be expressed as

2(t) = ax sin (w~(t - t l ) + ~ ) + b~ where t, is in the neighborhood of time t and the phase

Hollerbach (2s~has presented the role of phase change in the synthesis of character shapes. However, he states, "'The important parameter influencing letter shape is not actually phase change but the value of the horizontal velocity at the vertical velocity zero crossing". Let us examine the role of the relative phase difference in the x- and y-directions in characterizing shapes in the modified model. Figure 2 was generated by a numerical integration of the velocity components as described in equations (7) with the following parameter values: .t[~ = ,llv= 2 Hz; % = wy = 2nf~:

ax=a x . . . . 14mms ': b , = 7 m m s

'; b),=0:

4)x = n/30, 2n/30, 3n/30 . . . . . 2n (for different graphs): 4~, = 0:

to
./2/7

.X_~

~..N. C'C'x

r'-W)

./L//

..LJL ~.Y,.. C'T"~

fy')

.,4-4

.LJL

~,.~,,. C'N'~

/"g'9

./L-4

..L~

~ 6 " ~ C"¢'~

/"Y'9

.-4..,4

..~.X. ~ & ' , . C T ~

/'-g9

),.D,.. 9"4"x. ('W% X.5.. 9".k~

¢"Y%

,/'1/7

.A..A

X-k.. ~'i"X ¢-'Y'3 / ' 7 / 7

.,A..A

N..3~ t"C'x, C-Y-3 / % " 7 N.b-.. ( ' W ' x (-'U'5 ..-'7/7

.L£.

) , . . ~ C'Cx, ( " y - )

../l/7

Fig, 2, Different character shapes can be obtained just by changing the 4) parameters in the model.

B. SIMARDet al.

996

only on q6x - qby. Indeed, if qby is chosen to be equal to n the same periodic cycloid shapes are produced with q5x = 31n/30, 32g/30, 33n/30..... 3n successively. The differences would lie in a translation in space and time. The end points of the curve would be at different places along the curves because the time window t o < t < t o + 1 s was fixed, but the shape of one period of the graph remains identical. Therefore, we see that the relative phase ~bx- q5r is responsible for the shape of the oscillations, whereas the absolute phases are critically linked with the arbitrary definition of t 0. It is therefore a prime candidate to be included as a prototype descriptor of handwriting in our recognition experiment. In addition, the character prototypes include the positional information (x,, y,) of the pen which are normalized and refined through some simple pre-processing. 4. THE RECOGNITIONSYSTEM Figure 3 shows a block schematic diagram of the on-line recognition system. The data acquisition block captures relevant data from the input. The prototype generation block converts the acquired data into a prototype for comparison with prototypes stored in the dictionary. Feedback is provided to the system during the learning stage wherein new prototypes are added to the dictionary or prototypes are modified. We will examine each of these block/stages in detail. 4.1. Data acquisition Data acquisition is performed on-line with a Bit Pad One R type digitizing tablet manufactured by Summagraphics. The resolution of this peripheral is of the order of 0.1 mm or 1/200 in. and the z-direction is binarized into either an up or down position. Pen-tip position is sampled at a rate of 45 Hz. This choice is justified by the fact that frequencies encountered in handwriting movements all lie below 15 Hz (according to Vredenbregt and Koster. ~31) Yashuhara Its'32~ uses 16 Hz as the cutoff frequency. The actual value for the cutoff frequency depends on what level of energy is considered to be significant. For this reason, different values of cutoff frequency are often heard. Therefore, a sampling rate of 45 Hz is well over the nyquist rate.

Data Acquisition

=

Processing Into prototype

This is desirable considering that our handwriting signals will be of finite time length. For simplicity users are asked to write within boxes, thereby providing a simple definition of proximity. Boxes of the size 17mm by 12mm are provided. To accommodate for strokes (pen-down to next pen-up) which overlap box boundaries, a stroke is considered to be part of a box if its center is contained within the box. This allows the subjects to write loosely in boxes, thus minimizing the strain provided by this form of input. Only the first stroke has been used in this experiment. The number of strokes is used as discriminant for some character sets. 4.2. Definin9 character prototypes The recognition process attempts to match the first stroke of the character based on functions extracted by the handwriting model. This is the most complex part of the algorithm, and it is appropriate to avoid making such a comparison where it can be determined a priori that characters are substantially dissimilar. For this purpose, a set of discriminating characteristics that shall be called global features from here o n - - i s gathered to show when characters are dissimilar enough. These features are used to reduce the dimensionality of the dictionary search and primarily act as filters to the probable candidates. The six global features are: (1) number of strokes; (2) the complexity of the first stroke: number of oscillations in the first stroke: max (argy) - min (argr); (3) size of stroke 2: relative size of perimeter of subsuming boxes of strokes 2 and 1; (4) size of stroke 3: relative size of perimeter of subsuming boxes of strokes 3 and 1; (5) angle of stroke 2: angle of best fit straight line through the sample points of stroke 2; (6) angle of stroke 3: angle of best fit straight line through the sample points of stroke 3. A typical input prototype is represented as follows:

(k,f,x,y,~p,,-~x) where k is the total number of samples in the first stroke of the character; f a vector representing the six global features, x, y are vectors representing k normal-

1 Comparison of prototypes

Dictionary ~f prototypes

I

Fig. 3. Block diagram of the recognition system.

Guessed identity and c o n f i dence measure

On-line character recognition

997

Z_ Fig. 4. Example of two occurrences of the letter "e'.

ized values for the first s t r o k e of the c h a r a c t e r a n d d~y- ~b~ are c o r r e s p o n d i n g p h a s e differences.

4.3.

Let us c o n s i d e r h o w the h a n d w r i t i n g b e h a v e s with respect to this time base. W h e n arg~,(t) e q u a l s an o d d m u l t i p l e of ~z, the vertical velocity is zero a n d the vertical p o s i t i o n is at a local m a x i m u m . W h e n the time-

Prototype comparison

4.3.1. Findin,q the warpin9 closely the f u n c t i o n arg~.(t):

path.

arg~.(t) a= i wy(u)du +

Let us s t u d y m o r e

~y(t).

axis is w a r p e d to m a k e a best a l i g n m e n t of the p r o totypes, surely these e x t r e m a s in p o s i t i o n s h o u l d be aligned in time. But these are n o t the only key points: c o n s i d e r values of argy(t) equal to (2nx + 1/2)zt. T h e s e r e p r e s e n t a m a x i m u m u p w a r d velocity b e t w e e n a local

(8)

m i n i m u m a n d a local m a x i m u m . In fact, all i n t e r m e d i a t e values are of interest b e c a u s e

T h i s is a n o n - l i n e a r t r a n s f o r m a t i o n of time. N o t e

argy(t) r e p r e s e n t s a m e a s u r e of p o s i t i o n a l o n g the curve

to

t h a t it w o u l d be linear if Wy(t) were a c o n s t a n t . N e v e r theless, argy(t) is a c o n t i n u o u s , m o n o t o n i c a l l y increasing function. It h a s all the criteria to be c o n s i d e r e d a

Table 2. Processed data for second character of Fig. 4

time base itself.

Table 1. Processed data for first character of Fig. 4

n

.~,

.~,

(rad)

(rad)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

-2.13 - 1.44 -0.82 -0.31 0.09 0.35 0.47 0.47 0.37 0.18 0.05 -0.30 -0.51 0.66 --0.72 --0.66 --0.46 -0.13 0.32 0.89 1.55 2.26

-1.42 1.05 0.62 -0.14 0.35 0.80 1.18 1.47 1.64 1.67 1.57 1.34 1.00 0.58 0.11 --0.38 --0.84 1.25 -- 1.56 -- 1.76 --1.83 --1.76

-1.15 - 1.15 -1.15 -1.15 - 1.15 -1.15 - 1.15 - 1.15 - 1.15 - 1.15 -1.15 - 1.15 - 1.15 --1.15 -1.15 -- 1.15 -- 1.15 -- 1.15 1.15 -- 1.15 --1.15 --1.15

0.56 0.84 1.12 1.40 1.68 1.95 2.23 2.51 2.79 3.07 3.35 3.63 3.91 4.19 4.47 4.75 5.03 5.31 5.59 5.86 6.14 6.42

m

x,,

y,,

~bym- ~x~ (rad)

argr,, (rad)

1 2 3 4 5 6 7 8 9 10 I1 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

-2.34 - 1.81 - 1.29 -0.82 -0.40 -0.06 0.19 0.37 0.45 0.46 0.39 0.27 0.11 -0.07 -0.25 -0.40 -0.51 -0.56 -0.53 -0.43 -0.23 0.05 0.41 0.84 1.33 1.85. 2.39

1.47 - 1.39 - 1.22 -0.98 -0.68 -0.34 0.01 0.37 0.71 1.00 1.24 1.40 1.48 0.48 1.38 1.21 0.97 0.67 0.33 -0.03 -0.39 -0.72 1.01 - 1.24 - 1.40 - 1.48 1.47

-- 1.47 1.47 - 1.47 - 1.47 - 1.47 - 1.47 - t.47 - 1.47 - 1.47 1.47 - 1.47 - 1.47 - 1.47 - 1.47 - 1.47 - 1.47 1.47 1.47 - 1.47 - 1.47 - 1.47 1.47 -- 1.47 - 1.47 - 1.47 - 1.47 - 1.47

0.00 0.24 0.49 0.73 0.97 1.21 1.46 1.70 1.94 2.19 2.43 2.67 2.91 3.16 3.40 3.64 3.89 4.13 4.37 4.61 4.86 5.10 5.34 5.59 5.89 6.07 6.31

B. SIMARDet al.

998

Table 3. Warping path n(m) m

!

2 3 4

5 6 7 8 9

n(m) 1 1 1 2 2 3 4 5 6

10 7

11 8

12 9

13 9

14 10

which is independent of the size and speed of handwriting. This function is like an invariant time-base for handwriting. When the prototypes are aligned according to this time base, no further time-warping is required. Let us demonstrate this by an example. The two characters of Fig. 4 are written successively onto the digitizing tablet described earlier. The data obtained from this input device are processed into functions that describe a prototype as listed in Tables 1 and 2 alongwith the argy functions that we shall use for the timewarping process. Let us now look at how the warping path is obtained. As an example, suppose point m = 9 is chosen. Because of the monotonicity of the arg r functions, it is evident that the warping path n(m) evaluated m = 9 can be foulad simply by searching for the values ofn for which argrm[,._ 9 = 1.94. In this case, a search through the last column of Table 1 reveals that n = 6 yields the smallest difference with arg r nl, = 6 = 1.95. The process of matching argy functions is simpler and yields a monotonically increasing warping path n(m) because of the properties of the argy functions. The resulting warping path is shown in Table 3.

15 11

16 17 12 13

18 19 20 21 22 23 24 25 26 27 14 15 15 16 17 18 19 20 21 22

where w~, w 2 are weights that describe the relevance for recognition of each of the functions that describe a prototype. Values will now have to be attributed to these weights. We have seen that q~x- q~. is a good shape descriptor for discriminating characters such as "u" and " n ' . However, this function is not sufficient for discriminating characters such as "e" and 'T' where the timing between the x and y oscillations is similar, but the amplitudes are different. Therefore, each of the two terms in the sum of equation (10) is quite important for recognition. An empirical method is used to evaluate how much weight is carried by each term and to adjust w I and w 2. This way w 1 and w 2 have been assigned values of l and 0.8, respectively. 4.4. Calculatin9 a confidence measurefor the recognition Let R.be the event that a character was properly recognized, a n d / ~ be the event that a character was not properly recognized. Also let Sl be the best score obtained by the comparison process and s2 the second best. Let X be a random variable composed of (s l, (s2 - s l)

x lO0/sO. 4.3.2. Score calculationfor a prototype. The warping path established in the previous section provides an alignment of prototypes according to the time basis described by the argy functions. With this alignment of prototypes, a global comparison measure can be computed from a union of local distance measures between respectively the alignment of two prototypes. This is expressed in the following equation: 1

k

Let X -- x be the event that the random variable x takes on the particular value x = (xl, x2). Let us develop a probability-like measure of confidence that the identity of the unknown character is the same as that of the winning reference character for which the best comparison score s 1 is obtained:

P(R IX = x) -

P(R c~ X = x) P(X = x)

O = ~ = , D((x,.,y.,,Oy~ -- 0~), 0~,(m),,9,(,,,,q~y.... - 4 ~ . , j ) .

P(X = xIR)P(R) (9)

It can be seen that the process of minimization is no longer required as the warping path is known a priori. The computational gain resulting from the simplification of equation (9) is therefore a breakthrough. Let

and

P(X = x]R)P(R) + P(X = x[R)P(R) (11) Let P(RfX = x) be our measure of confidence that the unknown character was identified correctly knowing the scores obtained in the prototype comparisons. The terms on the right-hand side of equation (1 I) are estimated by statistics obtained through a small experiment involving four users. 4.5. Dictionary building

D((xm,Ym,am), ()~,, .9,, ~,))

= (wl~, w~),/((x,- ~0~ + (y,.- ~.)2) +(

WE ]min{lam--C~.l,2n--la,.--d.I}

k W l + W2//

(10)

The building of a dictionary of prototypes involves an iterative learning process illustrated in Fig. 3 by the link between the blocks "Processing into prototypes" and "Dictionary of prototypes". The dictionary is built as characters are input in a learning phase preceding the recognition phase. Because this recognition system is user dependent, each user will have to identify himself

On-line character recognition

999

I00 r

4

9O 80 F

F

2

7

~

8

7O F 6O 5O 40

20

! i

i 0 ~

o

~

~

Dictionary size

Fig. 5. Recognition of rate vs. dictionary size for each of the eight subjects.

to the recognition system, and have a separate dictionary of prototypes, with a corresponding learning phase. We need a learning process which would encompass as much information about the variability of the user's handwriting, while having a smaller number of prototypes in the dictionary. A similarity measure is used to prevent a prototype from being inserted into the dictionary.

5. E X P E R I M E N T A L

RESULTS

5.1. Character data We have limited ourselves to a character set composed of the 26 lower-case letters. To ensure that the characters input are not biased by the order they are written, the subjects were asked to write sentences of text. Subjects were allowed to use any boxes on the graphic tablet, allowing them to skip several boxes (or none) between words, as well as skipping lines. However, they were told to write separated letters. The sentences chosen are in French as all subjects were fluent in French. The text contains a total of 681 characters. The 22 sentences that were written by each subject are listed in Appendix D. The text was not meant to have an equal number of each letter of the alphabet. The number of occurrences of letters in the text does not reflect the probabilities of letters in either French or English. The sentences were chosen so as to best test the recognition system. For example: the letter "k" is easily identified because of its complexity, and so does not require extensive testing. The letters "n" and "u" have generated quite a

bit of confusion and are therefore included about as often as vowels. 5.2. Performance of the sy,stem in the learninq staqe Since the dictionary is built in an incremental fashion, we can use the dictionary size as a measure of material learnt. Let us define the dictionary size as the average number of prototypes in the dictionary per symbol in the character set. In our case, we have chosen the character set to be the lower-case alphabet. So the dictionary size will be the total number of prototypes in the dictionary divided by 26. The performance in the learning stage is then represented in Fig. 5. As this graph saturates at about 3 prototypes per symbol, a dictionary composed of 80 prototypes for our character set of 26 symbols is well within reason. Of course, some easily recognized letters have just one prototype each describing them, whereas other letters have more prototypes to reflect a greater variability, depending on the training for each user. Table 4. Steady-state performance Subject 1

2 3 4 5 6 7 8 Average

Machine errors ("~,,)

Human errors (~,)

9.9 5.5 8.2 11.2 12.4 12.6 3.2 6.2 8.6

3.0 1.8 3.9 4.1 4.4 0.9 0.2 1.1 2.4

B. SIMARDet al

1000

The recognition performance is dependent on the quality of the input, and how neat handed the subjects are. To qualify these factors, it is interesting to have the subject attempt to identify letters of their own handwriting, presented individually one at a time, out of order. The results of this are reported in the last column of Table 4. Before this last column is used to estimate the quality of the input, one must keep in mind that a subject will perform better at identifying

5.3. Steady-state performance of the system 5.3.1. Forced recognition. Let us ignore for the moment the confidence measure provided and suppose that all characters input are labeled by the system. In this subsection, the performance of the system will be examined under these conditions• The performance for each subject is summarized in the second column of Table 4.

I00 90 80 70 60 50 /

Legend

40

( a ) - -

(b) .............. (C). . . . . . . (d) . . . .

50 20

, ,""

I0 0

to

20

30

40

50

60

70

80

90

too

Acceptonce threshold (%)

Fig. 6. Rates of occurrence vs. acceptance threshold: (a) good acceptance; (b) bad rejection; (c) bad acceptance; (d) good rejection.

~00 90 8O 70 60 50 40 5O 20 ~0 0

i

0 Substitution

rete

(%)

Fig. 7. Rejection rate vs. substitution rate.

On-line character recognition

1001

I00 r 9O 8O 7O 6O 5O 4O 50 20 I0 0 0

I0

20

30

40

Acceptance

50

60

threshold

70

i

i

80

90

i

I00

(%)

Fig. 8. Percentage of errors that are rejected vs. the acceptance threshold.

his own handwriting than anybody else would do. It is worthwhile mentioning that other subjects could not identify letters in the handwriting of subject number 6 nearly as well as he did. The last column of Table 4 should therefore be considered to be a lower bound on the percentage of "objectionable" data. It is also interesting to note that Neisser and Weene ~33~found human readers achieve a recognition rate of about 97%. 5.3.2. Recognition with allowed rejection. Let us define an acceptance criterion using the confidence measure: a character will be rejected as "unrecognized" if the comparison with dictionary prototypes yields a confidence measure less than a set threshold ("acceptance threshold"). Then, an acceptance threshold of 0 corresponds to a lax acceptance criterion. An acceptance threshold of 100 corresponds to a stringent acceptance criterion. A panoply of intermediate acceptance criteria can be described by intermediate acceptance threshold values. It is interesting to study how recognition, rejection and error rates behave as a function of this acceptance threshold. There are four types of situations that one encounters;

(a) Good acceptance. Correctly labeled characters that have a confidence measure above the acceptance threshold. (b) Bad rejection. Correctly labeled characters that have a confidence measure under the acceptance threshold. Because of the acceptance criteria, these characters would be wrongly rejected and labeled "unrecognized".

(c) Bad acceptance. Incorrectly labeled characters that have a confidence measure above the acceptance threshold. These correspond to accepted characters for which a wrong identity label was assigned. (d) Good rejection. Incorrectly labeled characters that have a confidence measure under the acceptance threshold. Because of the acceptance criteria, these characters would be rightfully rejected and labeled "unrecognized". In Fig. 6, these four types of rates of occurrences are plotted as a function of the acceptance threshold. This figure depicts the performance of our system. Optimal performance is obtained when the acceptance threshold is set a value that corresponds to "bad rejection" and "bad acceptance". To evaluate compromises between the rejection rate and the substitution rate, refer to Fig. 7 which is a plot between the two. Again on this graph, we see the substitution rate (8.6%) for no rejection which corresponds to the lax acceptance criterion. Also, we see that we must reject all characters if we want to ensure a zero substitution rate (stringent acceptance criterionl. Another interestinggraph shown in Fig. 8 - is the rate of good rejections as a function of the acceptance threshold. This graph provides an interpretation for the physical significance of the acceptance process. As the acceptance criterion varies from lax (acceptance threshold = 0) to stringent (acceptance threshold = 100), more and more labeling substitutions are rejected. The relationship between the two is almost linear, illustrating well the interpretation of the acceptance process. The least linear part of the graph is for an acceptance threshold under 30%.

1002

B. SIMARDet al. 6. CONCLUSIONS AND DISCUSSIONS

The study of the oscillatory model of handwriting has led to some interesting refinements. Amongst these, a reformulation of the model was achieved that best suits the dynamics of handwriting. This reformulation provides for continuity in the phase functions in the case of a non-constant oscillatory frequency. The oscillatory model had previously been used as a way of generating synthetic handwriting, whereby one could provide simple control in the form of mathematical parameters and get the description of velocity and displacement resulting from this control. The reformulation of the model enhances it in that it also provides for a way to determine what control to apply to the model in order to generate handwriting similar to an observed sample. In effect, a method is shown to calculate the control from the handwriting, whereas previously we could only generate handwriting by supplying control over the model. This reinforces the model by demonstrating the bivalence between control and handwriting, in essence closing the loop that makes the modelling a reversible process. One of the interesting characteristics of the model is that it separates the different aspects of handwriting: size, shape, writing speed. This capability to isolate characteristics of handwriting is demonstrated in a character recognition experiment where the parameters of the oscillatory model are used to represent the characters. In this experiment, the phase functions of the model are used as a basis for the comparison of shapes. In order to compare functions of time, a timewarping technique is used. The absolute phase provided by the model, the complete argument of the sinusoid is in itself a time basis which is independent of the writing speed. That is, no dynamic programming algorithm is required to implement this time-warping technique on handwriting prototypes as the time basis can be retrieved from handwriting. After a minimal amount of training for the userdependent recognition program (on average about three copies of each character of the alphabet), results from the recognition experiment show that only 2% substitutions are made by the program when it is allowed to classify 13% of the input characters as "unrecognized". When forced to identify all characters input, the program makes on average 8.6% character substitutions. Acknowledgements This paper is based on the Master's Thesis of the first author who acknowledges the financial support of the Institut National de la Recherche Scientifique (INRS) and Fonds pour la Formation de Chercheurs et l'Aide fl la Recherche (FCAR). The work was also supported by NSERC grant number OGP0006022. We wish to thank Michael Sabourin for his help in preparing the final version of this paper. REFERENCES

1. R. Bozinovic and S. M. Srihari, Knowledge-based cursive script interpretation, Proc. 7th Conf. on Pattern Recognition, pp. 774-776, August (1984).

2. M.K. Brown and S. Ganpathy, Processing techniques for cursive script word recognition, Pattern Recognition 16, 447-458 (1983). 3. S. Mori, K. Yamamoto and M. Yasuda, Research on machine recognition of handprinted characters, IEEE Trans. Pattern Analysis Mach. Intell. PAMI-6(4), 386 405 (1984). 4. C.Y. Suen, Automatic recognition of handprinted charact e r s - t h e state of the art, Proc. IEEE 68(41, 469 487 (1980). 5. C. C. Tappert, Cursive script recognition by elastic matching, I B M J. Res. Dev. 26(6), 765 771 (1982). 6. C. C. Tappert, C. Y. Suen and T. Wakahara, The state of the art in on-line handwriting recognition, Trans. Pattern Analysis Mach. Intell. PAMI-12(8), 787-808 (1990). 7. D.J. Burr, A technique for comparing curves, Proc. IEEE Comput. Soc. Conf. on Pattern Recognition and Image Proeessing, Chicago, Illinois, pp. 271 277, August (1979). 8. D.J. Burr, Designing a handwriting reader, Proc. 5th Int. Conj'. on Pattern Reco#nition, Miami, Florida, pp. 715722, December (1980). 9. D. J. Burr, Elastic matching of line drawings, Trans. Pattern Analysis M ach. I ntell. PAM1-3(6), 708-713 ( 1981). 10. D. J. Burr, A normalizing transform for cursive script recognition, Proc. 6th Int. Conf. on Pattern Recognition, Munich, Germany, pp. 1027-1030, October (1982). 11. D.J. Burr, Designing a handwriting reader, IEEE Trans. Pattern Analysis Mach. Intell. PAMI-5(5), 554-559 (1983). 12. M. Eden, Handwriting generation and recognition, Recognizing Patterns, P. A. Kolers and M. Eden, eds, pp. 138154. MIT Press, Cambridge, Massachusetts (1968). 13. R. W. Ehric and K. J. Koehler, Experiments in the contextual recognition of cursive script, IEEE Trans. Comput. C-24(2), 182 194(1975). 14. W. W. Loy and 1. D. Landau, An on-line procedure for recognition of handprinted alphanumeric characters, IEEE Trans. Pattern Analysis Maeh. lntell. PAMI-4, 422 (1982). 15. P. Mermelstein and M. Eden, A system for automatic recognition of handwritten words, Proc. 1964 Fall Joint Computer Conf. A F I P S Conf. Proc., Vol. 25, pp. 333-342. Spartan, Washington (1964). 16. P. Mermelstem and M. Eden, Experiments on computer recognition of connected handwritten words, Inf. Control 7(2), 255-270 (1964). 17. G.T. Toussaint, The use of context in pattern recognition, Pattern Recognition 10, 189-204 (1978). 18. M. Yashuhara, Identification and decomposition of fast handwriting system, IEEE Trans. Circuits Syst. CAS30(11), 828 832 (1983). 19. J. Duvernoy, Handwriting synthesis and classification by means ofspace-variant transform and Karhumen-Loeve analysis, J. Opt. Soc. Am. 65(II), 1331-1336 (1975). 20. M. Eden and M. Halle, The characterization of cursive writing, 4th Syrup. on Inf. Theory, C. Cherry, ed., pp. 287-299. Butterworth, London (1961). 21. M. Eden, On the formalization of handwriting, Proc. Syrup. in Appl. Math., Am. Math. Soc., Providence, Rhode Island, Vol. 12, pp. 83-88 (1961). 22. M. Eden, Handwriting and pattern recognition, IRE Trans. Inf. Theory IT-8(2), 160-166 (1962). 23. M. Eden and P. Mermelstein, Mathematical models for the dynamics of handwriting generation, Proc. 16th Annual Conf. on Engineering in Medicine and Biology, Baltimore, Maryland, pp. 12-13 (1983). 24. A. G. Feldman, Change of muscle length due to shift of the equilibrium point of the muscle-load system, Biofizika 19, 534 538 (1974). 25. A. G. Feldman, Control of muscle length, Biofizika 19, 749-753 (1974). 26. J. J. D. Van der Gon, J. Thuring and J. Strackke, A handwriting simulator, Phys. Med. Biol. 6, 407-414 (1962). 27. J. J. D. Van der Gon and J. Ph. Thuring, The guiding of

On-line character recognition

28. 29. 30. 31. 32. 33.

human writing movements, Kybernetik 2(4), 145-148 (1965). J. M. Hollerbach, An oscillation theory of handwriting, Biol. Cybern. 39, 139-156 (1981). W. G. Koster and J. J. Vredenbregt, Analysis and synthesis of handwriting, Biomechanics, II, p. 77 (Medicine and Sport, Vol. 6). Hans Huber, Bern (1977). R. Plamondon and F. J. Maarse, An evaluation of motor models of handwriting, I E E E Trans. Syst. Man Cybern. 19(5), 1060 1972 (1989). J. J. Vredenbregt and W. G. Koster, Analysis and synthesis of handwriting, Philips Tech. Rev. 32, 73-78 ( 1971 ). M. Yashuhara, Experimental studies of handwriting process. Rep. Univ. Electro-Commun. 25(2), 233 254 (1975). V. Neisser and P. Weene, A note on human recognition of handwriting characters, Inf. Control 3, 191-196 (1960).

1003

"2 ' ) L 2 s i n ( w ' " ( n -

× sin ( - w " A t ' ) ]

=

2 ( w , At \ ~> ~)[a. s i n O v . . ( n - I ) A t , + 4 ) : ,

4sin t

.~(t) = ax sin (w~(t - to) + 4).<)+ b~

(A I a)

9(t) = a),sin(wy(t - to) + 4)y) + by.

(Alb)

. . " 2 ~)[s.--bx,,].

t

Solving for s,, s. = ( -

l)At,)

Xn+ 1 - - X

n

At~

1 ]A2.~. + h~. 4 sin z (wx, At./2) } "

~t.~o L \

" /+b~ lim s . = lim / A,~o A<.+oL 4(w~Atj2) 2 J "

I-

lim s . =

1 A2s. ]

lim/~,.~, . . . . . . . L

/+hx w~°IAtfJ " A2s. G Iim s . = 1 lim / - w L A " - ° L - (at:)2 [ . _ + h ....

A1Xn

A,, j = - w:..,),[mo -(at,i;J + h.,.

~ ~(t)J,-,,,+(..a,, .~(t)J . . . . . . i.-

~)At~

=

= axsin(w,(t - to) + 4)~) + b~J,=t,,+(, i)a*,

(A2)

'"n

-

1 ~!°L X(t)]

. . . . . . I.

l)A,. + bx.

1

2(01 . . . . . 0,-IIAt~ = - ...2 [ - w ~ a ~ s i n ( w ~ . ( n - 1) Wx.

Let us also approximate the nth order derivative of the horizontal velocity as an nth order difference ofs.. Note that odd numbered differences are defined as forward differences as backward. The purpose of this is to ensure that differences do not drift, i.e. the indices ofs in the formula for calculating A2s, are symmetrical about n. Let us expand Alsn=Sn+l

"

]

A2s

At~

= u~.sin(w~.(n - l)At, + 4)~.) + b~,,.

(A4)

'im r ( - '4sinS(wx,,At~/2)/

A,,~0

The parameters ax, wx, 4):obx, a r, wr, 4)r, by will be considered constant over a small time interval. Over this time interval the resulting sinusoid must fit the measured values of .~(t), ~(t). As many values of ~(t), f(t) as necessary will be considered in order to uniquely define the set of parameters. Let us start by first calculating an approximation of the velocity ~(t) from the measured positional data: At~

]

2/w,. At \

=-4sin

Consider the model given by the following equations:

a X(to+nAt,-X(to+(n-

l)At~ + 4)..)

Let us check that this difference equation is plausible by evaluating the limit at which it becomes a differential equation in the continuous time domain:

APPENDIX A. EXTRACTION OF THE MODEL PARAMETERS FROM HANDWRITING

Sn--

\F

/ w x At

= 2a~'"sin t

× Ate+ 4)OJ +b~° 2(01 ..... +l.- l)At~ = a~ sin(w~.(n - 1)At. + 4)~,.) + b~.

~(t)] .... +l. 1~At, = a. sin (w ..(t - to) + 4)~°) + b~.J ..... + (n - l)At, :t(t) = a~,. sin (w~.(t - to) + 4)~.) + b~ . The evaluation of the limit yields a known result. Therefore, our verification shows that equation (A4) is plausible. Similarly, the equivalent of equation (A4) for s. + ~ is

- lax. sin (w~nAt~ + 4)~.) + b~.

- ( a ~ sin(w~.(n

1)Jt~+~b:,.)+bx.)

= a~.[sin(w~°nAt~ + 4:,.) - sin (w~.(n - l)At~ + 4).,.)] =2a~sin(Wx~t')cos(w~,(n

s.+,=(-

;)At~ + 4)x,). (A3)

Similarly

l

c°s(w+

)

°4

"]A~s + +hx.

~-

~(AZs.+ - A 2 s . )

"

Then s.+ - s . = ( -

A ' s. _ , = s. - s. - ~ = 2a~. sin t ~

1

4 sin z (w~At~/2)]

sin2 (WxA'~) =

\

4sinZ(w~At~12)J

I

1( A2Sn+1 -- A2Sn~

1 (A3sn~

4\

4kAls.}

s.+ l

/ =

s.

Then A2s. = Als.

Als. 1

and substituting AI~ = 1/f. results in w~. = 2/'_,sin - 1

- 4

(A5)

B. SIMARD et al.

1004

And substituting equations (A8) into the trignometric identity sin 2 u + cos 2 u = 1

and from equation (A4) bx = % + ( _ 1 ~A2s " " k,4 sin 2 (WxAt~/2)/ "

(A6)

S n + 1 - - Sn

( _ s . . , + s.-2bx,,

The first-order difference of s. was earlier expanded in equation (A3). In a similar development, let us expand the first-order sum of s.

:'" = q \ \ 2 c o S ( W x

s.+ 1 + s. = G , [ s i n ( w x nAt~ + 4),.)

f

2

1

At J2)/# + --

2 sin(w..Atj2)

. (AI0)

I f we let

+ sin(wx°(n - l)At~ + 4bx,)] + 2bx,.

f w~ Ats'~ • f

"]2+(2a~,sin(w~.,Atj2))

\2ax. cos(t%Atj2)}

,~. ~= (Y.+ t -

_12)At, + O:,,,)

y.)lAt~

s i m i l a r e q u a t i o n s in t h e y - d i r e c t i o n a r e o b t a i n e d . + 2b~..

(A7)

Then, the two equations (A3) and (A7) in two u n k n o w n s ax° and ~b,. can be expressed as

2a~,cos(w:,,AtU2)

sin w~. n - 2 At~+qS~.

cos w~, n - 2

At~+¢~.

APPENDIX B. SHORTCOMINGS OF" THE OSCILLATORY H A N D W R I T I N G M O D E L A N D E X T R A C T I O N OF" PARAMETERS IN THE EXTENDED MODEL The equations representing the oscillatory handwriting model are given as

= 2 G sintw~AGi2) '

:c(t)=axsin(wx(t

Dividing equation (A8a) by (A8b)

t0) + 4bx) + b:,

(Ala)

:'(t) = a r sin (%,(t - to) + ?G) + by.

(Alb)

Let us now examine what happens when the model parameters are varied slowly. To see what happens when the parameters are varied, two sets of parameters will be applied on the model, one after the other. The following function will be used to study the vertical velocity:

.%+, + s . - 2b>. )(2a~.sin
s +, + s . - 2b:, ['wxat,'] =-" i. "tan/ " :l

As.

k

+tan

2 )

f'(t)=

0

for t < t j

arsin(2rtfyt +c~) c~rsin(2nfyt+dp)

for t l < t < t 2 for t 2 < t < t 3"

0

for t 3 < t

Let t 2 = t I + l/f,. and t 3 = t2 + 1/J~. such that ~'(t) is composed of one full cycle of each sinusoid. Also let

iVs~+ i +s.-2b~<,,tan(Wx.,At[] ] L AL% \ 2 J]'

(A9)

a,. = 13.6mms -i; ay = 14.4mms 1; ./~.= 3.6Hz; f~. = 3AHz.

v'(O +15." +10.+5."

a)

(B1)

I

1

t

I

2

I

I

3

t (sec)

-5.

-10." -15." yt(t)

(mm/~¢¢)

+15.*I0.+,5,"

b) 1

t (s~c) 2

-5.

-10." -15."

Fig. B I. Graphical representation of equation (A 1) for two different sets of parameter values.

On-line character recognition

of equation (A1), as it is locally equivalent to it with t~ in the locality of t. The phase function 4'x is different from Ox, but a x, w~, b~ can be evaluated exactly as described before in equations (Al0), (A5), (A6). We will now find how to extract the parameter qS, for the model formulation of equation (B2a). Let

We shall consider two cases, as represented in Fig. BI: (a) t t = 0.0 s; 4' = 0 rad; q~ = +

1

9

1005

n rad;

(b) t~ = 2.5s; t h = 0 r a d ; q ~ = - 8 r t r a d . 9

0~( t) A=i wx(u)du

Note that f(t) is continuous and smooth over the interval

(tl,t3)

to

lim ))(t) = lira ~ ( t ) = O m m s -I t~t 2

argx(t) A__O~(t) + ~b~(t)

t~tf

= i wx(u)du + Ox(t)"

lim p(t) = lim ~(t) = 2na~f r = 2harry = 307.6mm s -~. (~t 2

t~t 2

It is clear from Fig. BI that case (b) is just a translation n time of case Ca), yet for some reason the phase change at time t 2 is not the same in both cases. In fact, there is a difference of n rad between the two cases. We see therefore that under this formulation, the phase will not be representative of the character shape. Furthermore, the phase change at time t 2 can take on any value, this being contrary to our hypothesis that the parameters change slowly. To overcome this problem the model shall be reformulated. Let us first unveil the new formulation for the oscillatory model

) )

We shall extract ~ d t ) by first solving for Ox(t) and then for argx(t ) and subtracting them

t'

wx(u)du + thx +b:,

(B2a)

~'(t) = a~,sin

%,(u)du+~y

(B2b)

= ax sin largx(t)) + bx t + At~

®x(t + Ate) =

~ w~(u)du t

t + At.

=Ox(t)+

~ w:,(u)du

-~ Ox(t ) + w~(t)At, when evaluated at time t = to + (n - 1)At, we obtain

0 .... = 0~,, + w~,,At~

(B6)

t

Ox, = Ox(t)l ..... = ~ wx(u)du = 0 t.

Ox~ = Ox, + w:, At~ = wx,At~

for t < t 1

O , , = 0x~ + w~At~ = w~,At~ + w~At,

1 4 . 4 m m s a for t 2 < t < t 3

at(t)=

for t 3 ~ t

0 (2rt/',, w,(t): ~ "

(2hi'),

Ox =

for t < t 2 for t >_t 2

(B3)

Substituting the parameter values of equations (B3) into equations (7) gives a complete description ofj~(t) in Fig. B 1. In equations (7), the formulation of the model has eliminated the phase discontinuity in describing the movements of Fig. B1. Now that our little problem is solved, let us examine more closely the formulation of equations (7). This model formulation resembles a great deal the previous one. In fact, the formulations of equations (4) and (7) are very similar in that they are locally equivalent. We would now like to model the formulation of equations (7). We shall take advantage of the local equivalence of the two formulations to simplify this task. Let t~ be in the neighborhood of time t. We can express equation (B2a) as

tl

sin (arg~(t + - 1 A t ~ ~ ] \ \ 2 / J J ...... (,-flat.

(

I

cos arg~ t + At~ \ \ 2 J / J ...... i, lla,.

to

2a~sin(w~,At~/2) (B8b)

Dividing equation (B8a) by (B8b)

tan(argo(t+ _(

1

AtOll

s,+, + s . - 2 b x o \ 2ax° cos (Wx°Atd2)J

Sn+I

--S

n

/

.%+1 + s , - 2bx°tan fwx°At~ll

+b.~

a,%

to

= a~sin (wx(t -- q ) + ~ ) + h~.

s,+~ + s, - 2bx° 2ax. COS(WxAt~2) (B8a)

\

wx(u)du+4~

=

x ( 2ax, sin (wx~At~/'2))

wx(u)du+4)~ +b~ h)

=a~sin(~w~(u)du+

(B7)

To solve for argo(t), we can devise relationships similar to equations (A8)

by(t) - O.

.i(t) = a~ sin

wx~ At~ i=1

¢,,.(t)- o

\-S /

2 ~)] ..... ~. ,A,,

I~

(B4)

It is not surprising to find that its form now resembles that PR 26:7-B

t+A/s

= ~ w~(u)du +

13.6rams ~ for t ~ < t < t z

\

~ wx(u)du to t

t

+b r

To convince ourselves that this formulation does indeed solve the problem encountered, let us describe the movements S'(t) of Fig. BI using the new formulation of the model. The following equations represent the values that will be assigned to the model parameters: 0

)

.i(t)= axsin\~ wx(u)du + O~ + hx

to

2(tl=a~sin

\

(B5)

to

=tan

~[s~÷l+s,-2b~°tan(W~.At~

]

1006

B. SIMARDet al.

o,.#o_

4,_

~ ~ 1,

,..

,F k

Fig. CI. Handwriting samples.

,~

~,~o-

On-line character recognition

argo,, = arg~°(t)]N.+l"

-

1007

APPENDIX C. SOME HANDWRITING SAMPLES

111A,.

Figure C1 shows handwriting samples from some of the subjects.

( ( 't+}At~ ) -arg~(t/m=,,,i. 11~. arg~

APPENDIX D. TEST SENTENCES FOR THE EXPERIMENT The sentences below were used for the recognition experiment. For more information on how these sentences were used, refer to Section 6.1.2.

~.arg~(t+:At~)-(®~(t .

~argx t+

!At~-Ox(t)~]

"

1 2

+2

~)

/ d ..... i. ,,A,~

At

At~-(w~(t) ~1 / \ 2 / J ..... i.

1,A,~

=arg(,+lA,) I arg~(t + !At, \ 2 J ] ..... I.- ,~A,. =tan

' F s"+' + s " 2b:" tan (u'x"At~)] L Als. \ 2 /]"

(B9)

From equation (A 15) qS.dt) = a r g . d t ) - ( 9 ~ t evaluated at time t = t o + (n qS,,, = arg ....

I )At.:

(9,,, A's.

"tan~

2

)J

i= ),%, At,.

1. Un enfant de douze ans traversa le pont de Quebec suivi d'un wagon. 2. Je joue fi Tarzan. 3. Si vous avez faim, nous vendons de la luzerne et du beurre. 4. Ce jeune h o m m e mange de la viande bleue. 5. J' ai vu qu'elle lavait le linge sale. 6. I1 se vantait toujors. 7. Qu'auriez vous fair en voyant cela? 8. Ces deux chiens japent tr~s fort lorsqu'ils voient des chats. 9. Volulez vous venir jouer fi la balle avec moi? 10. Quelle est la somme de six et treize? 11. Un h o m m e aux yeux bleus entra dans le yacht club. 12. De vieux tonneaux remplis de bon vin rouge. 13. On peut couper quatre en seize quarts. 14. Yves fait jouer de la musique sur son walkman. 15. Je vais visiter le zoo ce weekend. 16. I1 y a des curieux qui regardent les pyjamas. 17. On ne l'a pas vu depuis douze jours. 18. Je ne veux pas y aller fi ce party du jour de Fan. 19. Barbe rousse gtait blaze. 20. 11 y a un kayak ~. vendre. 21. Un fanfaron gagne. 22. Un hiver froid est gfich6,

(BI0)

About the Author BRUNOSIMARD completed his Bachelor's degree in electrical engineering (B. lng.) from the t~cole Polytechnique (Montr6al, Canada) in 1983. He obtained his M.Sc. degree from the lnstitut National de la Recherche Scientifique-T616communications (Montr6al, Canada) in 1989, with a research focus on signal processing and character recognition. Since 1982, he has been involved in research, hardware and software product development, as well as systems engineering and planning activities for Bell-Northern Research Ltd. He is presently with the systems engineering division in Ottawa, Canada, planning advanced services and technology for telecommunications.

About the A u t h o r - - BIRENI)RA PRASADAreceived the B.Sc. and M.Sc. degrees in physics from Banaras Hindu

University, the D.I.C. diploma in electrical engineering from Imperial College, London, and the Ph.D. in electrical communications from the University of London. He is Director of Multi-Media Lab at BellNorthern Research and Professeur Invite at l'Institut National de la Recherche Scientifique (INRS)Telecommunications, Universitg du Qugbec, Montreal, Quebec, Canada. His main research interests are in the areas of digital signal processing, multi-media communications and services and h u m a n communication. His previous employment includes work at Bell Laboratories, Murray Hill, New Jersey and HolmdeL New Jersey. He was an Assistant Professor in the Department of Electrical Engineering at the Massachusetts Institute of Technology, Cambridge, Professor and the Head of the Department of Electrical Engineering at the Indian Institute of Technology, Kanpur, India. He has also acted as a consultant for industries in both India and the United States.

About the Author

R . M . K . SINHA obtained his M.Tech. in 1969 from Indian Institute of Technology, Kharagpur, and Ph.D. in 1973 from Indian Institute of Technology, Kanpur. He is currently Professor and Head of the Department of Computer Science and Engineering at Indian Institute of Technology, Kanpur. Prior to this, he has been an Invited Professor at INRS-Telecomm, Montreal, a Visiting Faculty at Asian Institute of Technology, Bangkok, and as Reader at Banaras Hindu University. His current research interests are in the areas of artificial intelligence, text recognition, document processing, natural language processing and computer vision. He has published more than 50 technical papers in these areas. He is a senior member of IEEE and a Fellow of IETE (India).