Volume 51, number 4
OPTICS COMMUNICATIONS
15 September 1984
HYBRID PATTERN RECOGNITION USING THE FRAUNHOFER DIFFRACTION PATTERN Takumi MINEMOTO, Izumi TSUCHIMOTO and Satoshi IMI
Department of Instrumentation Engineering, Faculty of Engineering, Kobe University, Rokkodai, Nada-ku, Kobe, Japan Received 10 May 1984 Revised manuscript received 9 July 1984
A hybrid pattern-recognition method is proposed. The Fraunhofer diffraction pattern of an object was obtained in a coherent optical system and then it was processed by a digital image-processing system. The shift- and rotation-invariant characteristic values of the diffraction pattern was calculated and the classification of the object was carried out by using those values. The proposed method was tested in a simple experimental system by using some components of an electromagnetic relay as the sample objects to be recognized.
1. Introduction Various correlation techniques by complex filtering in a coherent optical system have been proposed after Vander Lugt [1 ]. Such pattern recognition systems are the frequency plane correlators; the majority of them are analog processors except the several hybrid systems [2,3] in which the optical correlation signals are processed digitally. In this paper, a pattern in the frequency plane (that is, the Fraunhofer diffraction pattern of an object) is used also for recognition of the object. The object pattern is converted to the Fraunhofer diffraction pattern in a coherent optical system. This process is analog processing by coherent light. The diffraction pattern is measured photometrically. The intensity distribution o f the diffraction pattern is invariant for a shift o f the object pattern and rotates together with the object pattern. Taking these characters of the diffraction pattern into consideration, shift- and rotationinvariant characteristic values are calculated digitally from the intensity distribution represented on cylindrical coordinates. Distances from the characteristic point representing an input object to the points representing the registered objects in the characteristic space are calculated and it is concluded that the input object is same as the registered object giving the minimum distance. The applicability of the proposed method is shown 0 030-4018[84/$03.00 © Elsevier Science Publishers B.V. (North-Holland Physics Publishing Division)
by a preliminary experiment for the recognition of components of an electromagnetic relay. In this experiment, incoherent images of the components are converted to coherent images by an incoherent-to-coherent image converter constructed with a Bil2SiO20 single crystal (BSO ITCC) and the intensity distribution of the diffraction pattern is measured by a solid-state TV camera. The digital processing is implemented by a small-scale image processor constructed with a microprocessor and digital memories.
2. Definition of characteristic values The intensity distribution in the Fraunhofer diffraction pattern of the object is proportional to the power spectrum of the Fourier transform of the object pattern. Then, the former has the same characteristics as the latter. We assume that g(x,y) expresses the object pattern and G(u,v), its diffraction pattern:
G(u ,o) ,x F [g(x 0')l,
(1)
where x and y are the coordinates in the input plane, u and o are the coordinates in the frequency plane, and F [ ] means the Fourier transform. It is known from the inherent characteristics of the Fourier transform [4] that (i) when g(x,y) is real the intensity distribution of the diffraction pattern is symmetrical with 221
Volume 51, number 4
OPTICS COMMUNICATIONS
///-j /"
~--~..[
/
15 September 1984
~.~maximu m
/
\ .
/.
k=~
~--"~
2N-1 2N-2 ,---
,\
';'-': . . . . . . . . : ' : ' : ' ~ , , , , , , , , , , , , I k-0 i 2 3 k0-NLI L I-o12
N ~2 N-1
/
0
-\
tSrr' 8
ll~,~_ 2N-I I
fl 2vz
( ra.d )
Fig. 1. Polar coordinates (r,0) on the frequency plane and 2N sectors.
Fig. 2. Illustration of the relation between the functions l(k) and I(1).
respect to the zero-frequency point:
reference point on the coordinate k as shown in fig. 2. The function I(1) becomes independent of the rotation o f the input pattern and has the same shape as l ( k ) . Thus, we can obtain a shift- and rotation-invariant characteristic function and, furthermore, know the orientation of the object pattern from the value of k 0. In this study, the following two values are calculated from the function I(/) as the shift- and rotation-invariant characteristic values. The first value is defined as
IG(u ,v) t2 = IG ( - u , - o ) 12,
(2)
(ii) the intensity distribution does not vary with the displacement of the input pattern in the input plane: IF[g(x,y)] 12 = IF[g(x - a , y - b)] 12,
(3)
where a and b are respectively the x- and y-components of the displacement, and (iii) the diffraction pattern rotates the same angle around the zero-frequency point as the input pattern rotates:
f
zr(k+ l )/N
l(k)=Tr(r2-r2) /IV
dOf
r2
r,
[G(r,O)12rdr,
(5)
where r 1 and r 2 are the inside and the outside radius of the sectors, respectively. From the characteristics (i) and (ii), the average intensity I ( k ) is a periodic function o f k with a period of N and is independent of the displacement of the object pattern in the input plane. But the rotation o f the input pattern has a direct effect upon I ( k ). Let the reference point on the coordinate k be in the k0th sector, which has the maximum value of I(k), and the coordinate l be the distance measured from the 222
I m = I(l = O) = l ( k = k0) ,
(7)
I 1 = I(l = N / 4 ) = I ( k = k 0 + N / 4 ) ,
(8)
12 = I(l = N / 2 ) = I ( k = k 0 + N / 2 ) ,
(9)
(4)
where a is the rotation angle. Let us consider the polar coordinates r and 0 in the frequency plane with the origin at the zero-frequency point and divide the plane into 2N small sectors as shown in fig. 1. The average intensity of the diffraction pattern I ( k ) in the kth sector is given by
zv
(6)
where
IF[g(x cosa - y sina,x sina + y cosu)] 12 = [G(u cosa - o sina, u sina + v cosa) l 2 ,
R = (/2 +/~ + I] + 12)112,
and 13 = I(l = 3N/4) = I ( k = k 0 + 3N/4).
(10)
The value R is the magnitude of a fictitious vector that has 4 rectangular components of/(/) at intervals of N/4 corresponding to an argument 0 of 45 degrees in the frequency plane• The second value is defined as c~ = cos-l ( I m / R ).
(11)
The value ~ represents the angle between the direction of the vector and the axis along which the vector has the maximum component• The determination of the exact class to which the input pattern belongs is made according to the distance in the characteristic space where the points (R,~) are distributed• The definition of the distance d in the characteristic space is given by
Volume 51, number 4
OPTICS COMMUNICATIONS
RI'-RoI2+(¢j-OOI2]1/2 d:[(
i
(12) '
where R 0 and 4~0 are the characteristic values of the input object pattern,K'j and C/the average of the characteristic values over the patterns belonging to the/'th class, and OR. and o~,. are the standard deviations of R/and ~i' respectively. The distances are calculated for all classes registered in advance. It is concluded that the input pattern belongs to the class giving the minimum values of the distances.
3.
Apparatus
A schematic diagram of the hybrid pattern-recognition system is shown in fig. 3. An object was illuminated by a short-arc mercury lamp and its silhouette was imaged on a BSO ITCC by a lens ILl (magnification: 1/8 times). The exposure of the image was monitored through a dichroic mirror DM by using a photodiode PD and controlled to be constant by adjusting automatically in time of shutter (SH1) opening. The recorded image on the BSO ITCC was read out by a collimated coherent light (633 nm from H e - N e laser).
Object
ILl
The BSO ITCC had a resolution of 15 lp/mm and an effective diameter of 7 mm. The Fraunhofer diffraction pattern of the coherent image was made on the frequency plane by a lens FTL ( f = 200 nm). The opening and/or closing of the shutters SH1 and SH2, application of high voltage to the BSO ITCC, and switching of the flashlight FL were controlled by the main controller constructed with a microprocessor 8085. The diffraction pattern on the frequency plane was filtered by using an opaque point mask (OP Mask diameter: 200/am) set at the zero-frequency point, in order to prevent a blooming effect of TV camera due to the strong zero-frequency component of the diffraction pattern. The filtered pattern was magnified and imaged on a solid-state TV camera (KP-120, HITACHI, Ltd.) by a lens IL2. The video signal from the camera was converted to a digital signal o f 8 bits and recorded real-time in digital memories in a small-scale image processor, which was constructed with a microprocessor 6809, by means of direct memory access. In the image-processing system, the diffraction pattern was divided into 128 small sectors (N = 64) on the frequency plane as explained in sec. 2. The average intensity given by eq. (5) was calculated and the other digital processes (that is, the calculations of the value
SH1 L.~ BSO ITCC ~DM f A FTL
Hg lamp P
PH ' ,'
IL2
FL
'!1
15 September 1984
~
HV
Video controller L Main ] _~controler J.
~,/D--] Digital image processor
Fig. 3. Diagram of pattern recognition system. A: analyzer, A/D: analog-to-digital converter, DM: dichroic mirror, FL: flash lamp, FTL: Fourier transform lens, HV: high-voltage source, ILl and IL2: imaging lenses, M: mirror, OP Mask: opaque point mask, P: polarizer, PD: photodiode, PH: pinhole, and SH1 and SH2: shutters. 223
Volume 51, number 4
OPTICS COMMUNICATIONS
k 0 , the characteristic values R and 4, and the distance d, and the determination of the class to which the input pattern belongs) were carried out.
4. Results and discussions Photographs of ten objects used in the experiment are shown in fig. 4. These are components o f an electromagnetic relay. The Fraunhofer diffraction patterns obtained on the TV monitor screen in the experiment are shown in fig. 5. The small dark region at the center of each diffraction pattern shows the silhouette of the opaque point mask. It is seen from fig. 5 that the diffraction patterns have point symmetry as described in sec. 2. The characteristic values R and ¢ were calculated for each object by eqs. (6)~(11). The relative positions
15 September 1984
o f the object patterns in the characteristic space is shown in fig. 6, where each point shows the average (Rj,dpj) o f 20 measurements for the object and the bars show the magnitudes of the standard deviations OR/ and o Each object is successfully kept separate from one another, except objects no. 8 and no. 9. As the experiment in classification, 100 trials of determination by means of the distance d defined by eq. (12) were carried out for each object shown in fig. 4. The classification results are listed in table 1. As anticipated above, objects nos. 1 ~ 7 and no. 10 were perfectly recognized, but in more than half trials for no. 9, it was confused with object no. 8. The trials for object no. 8 had no misjudgment in this experiment, but the wrong identification may arise with some probability if we have a larger number of trials. These misjudgment may be arise from the fact that the diffraction patterns of the objects no. 8 and no. 9 have weak intensity due -
-
m
D
7
8
9
I-
5 cm
scale
10 Fig. 4. Photographs of the objects used in the experiment. 224
.I
Volume 51, number 4
OPTICS COMMUNICATIONS
15 September 1984
Q P.
i
3
•
n
i
.
9
10
Fig, 5. The Fraunhofer diffraction patterns of the objects shown in fig. 4.
to the small scale of those objects. The classification by means of the distance d is shift- and rotation-invariant in principle as described in sec. 2. In this respect, the experiment ended with un. satisfactory results. In the experiment, the result o f classification was shift-invariant for a displacement of the input object below -+0.4 mm from the position of the registered reference object on the plane of BSO ITCC and was rotation-invariant for a rotation within -+5 degrees from the direction of the reference object. These narrow allowances for shift and rotation may originate from the following: (1) the photometric data obtained by the solid-state TV camera were inherently
150 1
3
-~1oo 10
5
4
*
* +7
5O
6 -4-
z 9
ANGLE
8
l
1
25
50
¢)
(degree)
Fig. 6, Positions of the object pattern in the characteristic space.
225
Volume 51, number 4
OPTICS COMMUNICATIONS
15 September 1984
Table 1 Classification results Class of input object
Class determined
1
2
3
4
5
6
7
8
9
10
1
100
0
0
0
0
0
0
0
0
0
2 3 4 5 6 7 8 9 10
0 0 0 0 0 0 0 0 0
100 0 0 0 0 0 0 0 0
0 100 0 0 0 0 0 0 0
0 0 100 0 0 0 0 0 0
0 0 0 100 0 0 0 0 0
0 0 0 0 100 0 0 0 0
0 0 0 0 0 100 0 0 0
0 0 0 0 0 0 100 58 0
0 0 0 0 0 0 0 42 0
0 0 0 0 0 0 0 0 100
discrete sampling data of continuous image since it has a limited photosensitive area in each pixel, (2) the TV camera had a non-uniform sensitivity, and (3) the BSO ITCC used in the experiment had a small wavefront distortion. These problems would be removed by using a combination of a rotating fan-shaped slit and a photodiode, and an incoherent-to-coherent image converter with no aberation. From the value of k 0 we can know the direction of the input object as described in sec. 2. The direction was successively determined with a resolution of 2.8 degrees in the experiment.
established in a preliminary experiment on the classification of components of an electromagnetic relay. In this study, the characteristic values were calculated from only the Fraunhofer diffraction pattern. Furthermore, if we use, for classification, other shift- and rotation-invariant characteristic values such as the area and the peripheral length calculated directly from the object pattern by simple digital processing, a larger number of objects would be recognized accurately and at the same time the position and the direction of the object would be obtained easily.
References
5. Summary A hybrid method for pattern recognition has been studied by the use of the Fraunhofer diffraction pattern of an object and simple digital image processing of the pattern. The usefulness of the method has been
226
[1 ] [2] [3] [4]
A. Vander Lugt, IEEE Trans. Inf. Theory IT-10 (1964) 139. R.J. Thompson, Proc. IEEE 65 (1977) 62. J.R. Leger and S.H. Lee, Appl. Optics 21 (1982) 274. D.C. Champeney, Fourier transforms and their physical applications (Academic Press, London and New York, 1973).