- Email: [email protected]
Classification of handprinted Kanji characters by the structured segment matching method
Pattern Recognition Letters 1 (1983) 475-479 North-Holland
July 1983
Classification of handprinted Kanji characters by the structured segment matching method Yoshiyuki YAMASHITA,
Koichi HIGUCHI,
Youichi YAMADA,
Yunosuke HAGA
Research Laboratory, OKI Electric Industry Co., Ltd., 550-5, Higashiasakawa-cho, Hachioji-shi, Tokyo 193, Japan
Received 31 January 1983 Revised 12 March 1983 Abstract: A method for handprinted Kanji character classification is proposed in this paper. After extraction of directional
line segments and partitioning of the character frame area, a feature vector that represents the distribution of strokes is generated and is matched with average vectors in a dictionary. Key words: Character recognition, Chinese characters, handprinted characters, classification, matching method, line segment.
1. Introduction In Japanese documents m a n y kinds of handprinted characters are now used, such as Kanji (Chinese characters), Hiragana, K a t a k a n a and alphanumerals. Handprinted character recognition is regarded as suitable for text entry in the Japanese office, where handwriting has never been significantly displaced by typing because o f the large character repertoire. Suen et al. (1978) reported that m a n y OCRs for handprinted characters such as K a t a k a n a and alphanumerals had been developed and were used in offices in Japan. Equipments such as Kanji keyboards have been developed and are in use for input of Kanji characters. However, they require considerable training and practice especially for a large number of character categories (more than 2000 categories are used). Mori et al. (1980) reported that Japanese engineers in the field of character recognition had been engaged in the development of recognition methods for handprinted Kanji characters. Up to this time, two kinds of recognition methods have been proposed. They are the Structural Analysis Method and the Correlation Method. F r o m the
point of view of tolerating character shape variation it has been considered that the Structural Analysis Method is ideally suitable for recognition of handprinted characters. In the recent work on handprinted character recognition some improvements are achieved by developing the Correlation Method for the recognition of handprinted characters (e.g. Yasuda et al. (1979), Sakai et al. (1973), Natio et al. (1979), U m e d a et al. (1979) and Y a m a m o t o et al. (1981)). These developments are more effective than the simple Correlation Method because they give attention to the structure of a character. A two-step strategy is regarded as suitable for handprinted Kanji character recognition, for complicated shapes and a large number of categories. The first step is classification in which several candidate categories are selected f r o m the set of all possible categories (i.e. recognition classes). The second step is discrimination by which one category name is chosen f r o m the candidates. When developing the classification method two problems must be considered. First, by what process can the shape variation peculiar to handprinted characters be tolerated? Second, by what process can the character shape and structure be reflected in
0167-8655/83/$3.00 © 1983, Elsevier Science Publishers B.V. (North-Holland)
475
Volume 1, Numbers 5,6
PATTERN RECOGNITION LETTERS
V
July 1983
2.2. Line width calculation A 2 x 2 window is aligned with each dot. Let a be the number of black dots whose three neighbors within the window are all black. Let q be the total number of black dots within the circumscribed rectangle. Line width (W) is given by
Fig. 1. Scanning directions.
W= q/(q - a). features that are used for classification? In view o f these considerations a new method (Structured Segment Matching Method) for classification of handprinted Kanji characters is investigated. After extraction of directional line segments and partitioning of the character frame area, a feature vector which represents the distribution of strokes is generated and is matched with average feature vectors in a dictionary. This method has the following advantages. First, positional variations o f strokes are significantly tolerated by partitioning the character frame area. Second, the directional line segments and feature vector are easily extracted from a pattern without employing a conventional thinning process and pattern size normalization.
2.3. Extraction o f subpatterns In this process four kinds o f subpatterns which represent structural characteristics of a pattern are extracted by scanning along the four quantized directions (Figure 1). During the scanning, runs whose run length is greater than a threshold (C x W) are separated from a pattern (C is a constant, C = 2 in the experiment mentioned below), where run and run length are defined as Run: A chain of consecutive black dots, Run length: Let r be the number of black dots in a run. For horizontal and vertical scans the run length is r. For slanted scans the run length is x/2r. In this way, subpatterns which are sufficiently long runs are extracted from an input pattern. An example of a pattern and subpatterns is shown in Figure 2.
2. Feature vector
2.4. Determination o f the boundaries o f partititons
A feature vector is extracted from a pattern by means of the following procedure.
After the determination o f pattern projections onto the horizontal axis, boundaries for a horizontal partition of the character frame area are provided by the following procedure. The procedure for a vertical partition is the same as that for a horizontal partition. (1) The gravity center gH in the range o f character left (L) to right (R) is determined.
2.1. Detection o f a character frame The circumscribed rectangle frame of a pattern is determined by detecting the most left/right column and the most t o p / b o t t o m row which contains two or more consecutive black dots.
|" J / HSP
VSP
LSP
Fig. 2. Example of a pattern and subpatterns. 476
RSP
Volume 1, Numbers 5,6
PATTERN RECOGNITION LETTERS
July 1983
V
H
L
R
L,I I,I Fig. 4. Weighted masks for blurring.
daries are determined so that each partitioned area has approximately the same number of subdivisions. Examples of frame partitioning are shown in Figure 3.
2.5. Generation of feature vector The character frame area of each subpattern is partitioned into M × N subareas by the use of the positions of gravity centers. The number of black dots in each subarea (Bm,n,k) is counted and the segment length in each subarea is calculated by the relation
SLm, n,k = Bm, n,k/W Fig. 3. Examples of frame partitioning.
where m = 1 , 2 , 3 ..... M,
(2) Gravity centers g21 and g22 are determined in the interval (L, gll) and ( g l l , R ) respectively. (3) Gravity centers g31, g32, g33 and g34 are determined in the interval (L, gEl), ( g E l , g l l ) , (g Jl, g22) and (g22, R) respectively. An adequate number of gravity center positions is obtained by continuation of the above process. The number of subdivisions is 2 P ( P is a positive number), but we may want any number of subdivisions. To cope with this, a character frame is partitioned into some large number of subdivisions which is 2 P and greater than the actually required number of subdivisions. The positions of boun-
H & 12 10 11 8 10 I 0 8 ? 14 2 4 4 5 & 0 7 12 1~ 11 2 ~ 10 8 11
n = 1 , 2 , 3 . . . . . N,
k= 1,2,3,4. By normalization of the segment length (SLm, n,k) o f each subarea by the size o f the character frame, an (M × N × 4)-dimensional feature is generated.
2.6. Blurring o f feature vector Using weighted masks, the original feature vector is blurred to decrease the effect of positional variations of line segments, An example of the masks is shown in Figure 4. An example of a blurred feature vector is shown in Figure 5.
V 2 2 ~ 1 0
2 5 4 ~ 2
3 & 1 ~ 6
L 1 0 5 4 4 I0 2 5 5 I
~ 2 0 1 0
2 9 2 5 I
~ 1 I 5 4 9 1 I I 8 7 6 9 11 12
R 3 3 ~ 2 7
8 7 2 9 ?
9 5 1 7 8
9 b 8 1 2 5 10 8 2
Fig. 5. Example of feature vector. 477
V o l u m e 1, N u m b e r s 5,6
PATTERN
RECOGNITION
3. Experiment The classification of handprinted Kanji characters was implemented on the mini-computer OKITAC 5 0 / 6 0 . The programs were written mostly in FORTRAN. Input patterns used in this experiment were optically scanned on a 128 × 128 grid. After quantization, a pattern was smoothed by aligning a 3 × 3 window with each dot. This experiment was done using the set of 881 Kanji characters (which is called basic set) defined by the Japanese Ministry of Education. The dictionary was constituted of mean feature vectors of learning samples. 40 samples per category were used as learning samples and 20 samples per category which were not learned were tested. In this experiment, Euclidean distance from dictionary vectors was used as the discriminant function. Results using three kinds of partitioning numbers are shown in Figure 6.
4. Concluding remarks On the basis of the experiment reported in this paper, it is surmised that the following reasons account for the successful results. (1) The subpatterns extracted from a pattern ....
............. .................. ~
100
•"--
X9_ _ _ ~ 9, . . . . , . ~ ~
(1)
,
~
........
-
-~"
°
98
......... "j
o~
///
o
"~
~
•
.............................................................
7x7 x_s
LETTERS
J u l y 1983
represent the structure of Kanji character pattern satisfactorily: A Kanji character pattern is regarded as a straight-line picture. (2) The partitioning of a character frame area using the gravity centers of projections provides a stable feature vector: Positional variation of strokes parallel to the integration direction is invisible in the projection. It is observed in the investigation of preliminary results that the appropriate number of subdivisions in a partition is related to an aspect of the character structure, namely the number of strokes. Details concerning this will be resolved in further research. The Structured Segment Matching Method has been proposed in this paper. The development of the method aims at introducing characteristics of character structure into the Correlation Method. Successful results obtained in the preliminary experiment confirm the effectiveness of the method for classification of handprinted Kanji characters. In other words, the primary results and work to date lead the authors to believe that a high degree of recognition rate can be obtained with a large Kanji character set and that OCRs for handprinted Kanji characters will be in use in the near future. This develops a new field of computerization and promotes office automation in Japan remarkably. However, a great deal of work is required to develop a Japanese text reader. Future research will be directed towards resolution of character confusions which are more complicated in respect of Kanji characters than of alphanumerals because of the large character repertoire and the structural complexity. Selected subjects for development are (1) Dictionary making method, (2) Candidates selection method, (3) Discrimination algorithm.
92
e-.i
Acknowledgment °
90 1
10
20
60
Rank Fig. 6. The c u m u l a t i v e r e c o g n i t i o n rate.
478
The authors wish to thank Dr. S. Nakaya and K. Tanoshima, OKI Research Laboratory, for continuous encouragement and also wish to thank a member of ETL for offering the Kanji data base.
Volume 1, Numbers 5,6
PATTERN RECOGNITION LETTERS
References
Suen, C.Y., M. Berthod and S. Mori (1978). Advances in recognition of handprinted characters. In: Proc. 4th Int. Joint Conf. Pattern Recognition, pp. 30-44. Mori, K. and I. Masuda (1980). Advances in recognition of Chinese characters. In: Proc. 5th Int. Conf. Pattern Recognition, pp. 692-702. Yasuda, M. and H. Fujisawa (1979). An improvement of correlation method for character recognition. J. IECE Japan J62-D(3) (in Japanese).
July 1983
Sakai, K. and K. Mori (1973). Clustering of Chinese characters. Trans. IECE Japan PRL73(14) (in Japanese). Naito, S. and E. Yodogawa (1979). Rough classification of handprinted Chinese characters by stroke density function. Trans. IECE Japan PRL79(3) (in Japanese). Umeda, M. and I. Masuda (1979). Classification of handprinted Chinese characters by mesh and peripheral pattern matching. Trans. IECE Japan PRL79(26 (in Japanese). Yamamoto, E., N. Fuji, T. Ito and J. Tanahashi (1981). Handwritten Kanji character recognition using the feature extracted from multiple standpoints. In: Proc. IEEE Int. Conf. Pattern Recognition and Image Processing, pp. 131-136.
479
July 1983
Classification of handprinted Kanji characters by the structured segment matching method Yoshiyuki YAMASHITA,
Koichi HIGUCHI,
Youichi YAMADA,
Yunosuke HAGA
Research Laboratory, OKI Electric Industry Co., Ltd., 550-5, Higashiasakawa-cho, Hachioji-shi, Tokyo 193, Japan
Received 31 January 1983 Revised 12 March 1983 Abstract: A method for handprinted Kanji character classification is proposed in this paper. After extraction of directional
line segments and partitioning of the character frame area, a feature vector that represents the distribution of strokes is generated and is matched with average vectors in a dictionary. Key words: Character recognition, Chinese characters, handprinted characters, classification, matching method, line segment.
1. Introduction In Japanese documents m a n y kinds of handprinted characters are now used, such as Kanji (Chinese characters), Hiragana, K a t a k a n a and alphanumerals. Handprinted character recognition is regarded as suitable for text entry in the Japanese office, where handwriting has never been significantly displaced by typing because o f the large character repertoire. Suen et al. (1978) reported that m a n y OCRs for handprinted characters such as K a t a k a n a and alphanumerals had been developed and were used in offices in Japan. Equipments such as Kanji keyboards have been developed and are in use for input of Kanji characters. However, they require considerable training and practice especially for a large number of character categories (more than 2000 categories are used). Mori et al. (1980) reported that Japanese engineers in the field of character recognition had been engaged in the development of recognition methods for handprinted Kanji characters. Up to this time, two kinds of recognition methods have been proposed. They are the Structural Analysis Method and the Correlation Method. F r o m the
point of view of tolerating character shape variation it has been considered that the Structural Analysis Method is ideally suitable for recognition of handprinted characters. In the recent work on handprinted character recognition some improvements are achieved by developing the Correlation Method for the recognition of handprinted characters (e.g. Yasuda et al. (1979), Sakai et al. (1973), Natio et al. (1979), U m e d a et al. (1979) and Y a m a m o t o et al. (1981)). These developments are more effective than the simple Correlation Method because they give attention to the structure of a character. A two-step strategy is regarded as suitable for handprinted Kanji character recognition, for complicated shapes and a large number of categories. The first step is classification in which several candidate categories are selected f r o m the set of all possible categories (i.e. recognition classes). The second step is discrimination by which one category name is chosen f r o m the candidates. When developing the classification method two problems must be considered. First, by what process can the shape variation peculiar to handprinted characters be tolerated? Second, by what process can the character shape and structure be reflected in
0167-8655/83/$3.00 © 1983, Elsevier Science Publishers B.V. (North-Holland)
475
Volume 1, Numbers 5,6
PATTERN RECOGNITION LETTERS
V
July 1983
2.2. Line width calculation A 2 x 2 window is aligned with each dot. Let a be the number of black dots whose three neighbors within the window are all black. Let q be the total number of black dots within the circumscribed rectangle. Line width (W) is given by
Fig. 1. Scanning directions.
W= q/(q - a). features that are used for classification? In view o f these considerations a new method (Structured Segment Matching Method) for classification of handprinted Kanji characters is investigated. After extraction of directional line segments and partitioning of the character frame area, a feature vector which represents the distribution of strokes is generated and is matched with average feature vectors in a dictionary. This method has the following advantages. First, positional variations o f strokes are significantly tolerated by partitioning the character frame area. Second, the directional line segments and feature vector are easily extracted from a pattern without employing a conventional thinning process and pattern size normalization.
2.3. Extraction o f subpatterns In this process four kinds o f subpatterns which represent structural characteristics of a pattern are extracted by scanning along the four quantized directions (Figure 1). During the scanning, runs whose run length is greater than a threshold (C x W) are separated from a pattern (C is a constant, C = 2 in the experiment mentioned below), where run and run length are defined as Run: A chain of consecutive black dots, Run length: Let r be the number of black dots in a run. For horizontal and vertical scans the run length is r. For slanted scans the run length is x/2r. In this way, subpatterns which are sufficiently long runs are extracted from an input pattern. An example of a pattern and subpatterns is shown in Figure 2.
2. Feature vector
2.4. Determination o f the boundaries o f partititons
A feature vector is extracted from a pattern by means of the following procedure.
After the determination o f pattern projections onto the horizontal axis, boundaries for a horizontal partition of the character frame area are provided by the following procedure. The procedure for a vertical partition is the same as that for a horizontal partition. (1) The gravity center gH in the range o f character left (L) to right (R) is determined.
2.1. Detection o f a character frame The circumscribed rectangle frame of a pattern is determined by detecting the most left/right column and the most t o p / b o t t o m row which contains two or more consecutive black dots.
|" J / HSP
VSP
LSP
Fig. 2. Example of a pattern and subpatterns. 476
RSP
Volume 1, Numbers 5,6
PATTERN RECOGNITION LETTERS
July 1983
V
H
L
R
L,I I,I Fig. 4. Weighted masks for blurring.
daries are determined so that each partitioned area has approximately the same number of subdivisions. Examples of frame partitioning are shown in Figure 3.
2.5. Generation of feature vector The character frame area of each subpattern is partitioned into M × N subareas by the use of the positions of gravity centers. The number of black dots in each subarea (Bm,n,k) is counted and the segment length in each subarea is calculated by the relation
SLm, n,k = Bm, n,k/W Fig. 3. Examples of frame partitioning.
where m = 1 , 2 , 3 ..... M,
(2) Gravity centers g21 and g22 are determined in the interval (L, gll) and ( g l l , R ) respectively. (3) Gravity centers g31, g32, g33 and g34 are determined in the interval (L, gEl), ( g E l , g l l ) , (g Jl, g22) and (g22, R) respectively. An adequate number of gravity center positions is obtained by continuation of the above process. The number of subdivisions is 2 P ( P is a positive number), but we may want any number of subdivisions. To cope with this, a character frame is partitioned into some large number of subdivisions which is 2 P and greater than the actually required number of subdivisions. The positions of boun-
H & 12 10 11 8 10 I 0 8 ? 14 2 4 4 5 & 0 7 12 1~ 11 2 ~ 10 8 11
n = 1 , 2 , 3 . . . . . N,
k= 1,2,3,4. By normalization of the segment length (SLm, n,k) o f each subarea by the size o f the character frame, an (M × N × 4)-dimensional feature is generated.
2.6. Blurring o f feature vector Using weighted masks, the original feature vector is blurred to decrease the effect of positional variations of line segments, An example of the masks is shown in Figure 4. An example of a blurred feature vector is shown in Figure 5.
V 2 2 ~ 1 0
2 5 4 ~ 2
3 & 1 ~ 6
L 1 0 5 4 4 I0 2 5 5 I
~ 2 0 1 0
2 9 2 5 I
~ 1 I 5 4 9 1 I I 8 7 6 9 11 12
R 3 3 ~ 2 7
8 7 2 9 ?
9 5 1 7 8
9 b 8 1 2 5 10 8 2
Fig. 5. Example of feature vector. 477
V o l u m e 1, N u m b e r s 5,6
PATTERN
RECOGNITION
3. Experiment The classification of handprinted Kanji characters was implemented on the mini-computer OKITAC 5 0 / 6 0 . The programs were written mostly in FORTRAN. Input patterns used in this experiment were optically scanned on a 128 × 128 grid. After quantization, a pattern was smoothed by aligning a 3 × 3 window with each dot. This experiment was done using the set of 881 Kanji characters (which is called basic set) defined by the Japanese Ministry of Education. The dictionary was constituted of mean feature vectors of learning samples. 40 samples per category were used as learning samples and 20 samples per category which were not learned were tested. In this experiment, Euclidean distance from dictionary vectors was used as the discriminant function. Results using three kinds of partitioning numbers are shown in Figure 6.
4. Concluding remarks On the basis of the experiment reported in this paper, it is surmised that the following reasons account for the successful results. (1) The subpatterns extracted from a pattern ....
............. .................. ~
100
•"--
X9_ _ _ ~ 9, . . . . , . ~ ~
(1)
,
~
........
-
-~"
°
98
......... "j
o~
///
o
"~
~
•
.............................................................
7x7 x_s
LETTERS
J u l y 1983
represent the structure of Kanji character pattern satisfactorily: A Kanji character pattern is regarded as a straight-line picture. (2) The partitioning of a character frame area using the gravity centers of projections provides a stable feature vector: Positional variation of strokes parallel to the integration direction is invisible in the projection. It is observed in the investigation of preliminary results that the appropriate number of subdivisions in a partition is related to an aspect of the character structure, namely the number of strokes. Details concerning this will be resolved in further research. The Structured Segment Matching Method has been proposed in this paper. The development of the method aims at introducing characteristics of character structure into the Correlation Method. Successful results obtained in the preliminary experiment confirm the effectiveness of the method for classification of handprinted Kanji characters. In other words, the primary results and work to date lead the authors to believe that a high degree of recognition rate can be obtained with a large Kanji character set and that OCRs for handprinted Kanji characters will be in use in the near future. This develops a new field of computerization and promotes office automation in Japan remarkably. However, a great deal of work is required to develop a Japanese text reader. Future research will be directed towards resolution of character confusions which are more complicated in respect of Kanji characters than of alphanumerals because of the large character repertoire and the structural complexity. Selected subjects for development are (1) Dictionary making method, (2) Candidates selection method, (3) Discrimination algorithm.
92
e-.i
Acknowledgment °
90 1
10
20
60
Rank Fig. 6. The c u m u l a t i v e r e c o g n i t i o n rate.
478
The authors wish to thank Dr. S. Nakaya and K. Tanoshima, OKI Research Laboratory, for continuous encouragement and also wish to thank a member of ETL for offering the Kanji data base.
Volume 1, Numbers 5,6
PATTERN RECOGNITION LETTERS
References
Suen, C.Y., M. Berthod and S. Mori (1978). Advances in recognition of handprinted characters. In: Proc. 4th Int. Joint Conf. Pattern Recognition, pp. 30-44. Mori, K. and I. Masuda (1980). Advances in recognition of Chinese characters. In: Proc. 5th Int. Conf. Pattern Recognition, pp. 692-702. Yasuda, M. and H. Fujisawa (1979). An improvement of correlation method for character recognition. J. IECE Japan J62-D(3) (in Japanese).
July 1983
Sakai, K. and K. Mori (1973). Clustering of Chinese characters. Trans. IECE Japan PRL73(14) (in Japanese). Naito, S. and E. Yodogawa (1979). Rough classification of handprinted Chinese characters by stroke density function. Trans. IECE Japan PRL79(3) (in Japanese). Umeda, M. and I. Masuda (1979). Classification of handprinted Chinese characters by mesh and peripheral pattern matching. Trans. IECE Japan PRL79(26 (in Japanese). Yamamoto, E., N. Fuji, T. Ito and J. Tanahashi (1981). Handwritten Kanji character recognition using the feature extracted from multiple standpoints. In: Proc. IEEE Int. Conf. Pattern Recognition and Image Processing, pp. 131-136.
479