- Email: [email protected]
Coherent recognition of colored patterns
I January 1997
OPTICS COMMUNICATIONS ELSEVIER
Optics Communications
133 (1997) 77-81
Coherent recognition of colored patterns H.J. Caulfield a, I. Moreno b, J. Campos b, M.J. Yzuel b a Physics Department, Alubuma A &M b Departumento
de F&u.
University, Normal. AL 35762.
USA
Grupo de Optica. Universidad Auto’twmu de Burcelonu, 08193 Belluterra
Received 2 April 1996; revised version received
(Burcelonu). Spain
12 July 1996; accepted 2 August 1996
Abstract A properly preprocessed image of a colored scene can be used with conventional coherent correlators to recognize locate colored patterns. Color information is encoded by means of gratings with different orientation, frequency amplitudes. This codification permits to process polychromatic images in parallel with a monochromatic correlator.
1. Introduction The recognition of colored patterns by optical methods almost universally relies on separate processing of multiple color channels. Those results are then combined to give a score, a number used to determine the similarity between the input scene and the reference test [ 1,2]. This multichannel correlation process may be carried out sequentially or in parallel. In Ref. [3], multichannel color correlation in parallel is performed by using a tricolor grid and real-time spatial light modulator. As a result, the Fourier spectrum is located in different positions in the Fourier plane for the different wavelengths. A set of filters is introduced in this plane, each one matched to the target transmittance of the corresponding color channel. The correlations are superposed at the exit plane. Several architectures have been proposed to implement this idea (see for instance Refs. [4-61). These methods can be quite effective, but they lack the simplicity and convenience of more conventional coherent optical pattern recognizers designed to work on monochromatic or black-and-white scenes. The 0030~4018/97/$17.00 P/I
purpose of this paper is to recognize colored scenes using these coherent optical schemes. We assume that our coherent optical correlator 4f, joint transform - will use an SLM (spatial light modulator) as an input device and a laser beam for illumination. Thus, whatever it recognizes must be represented as a spatial pattern impressed on a laser beam. It follows immediately that our task is to encrypt the color information spatially and input that encrypted pattern into the system. We explore grating encryption, because it seems the simplest to us. This type of encryption is the theta modulation technique proposed by Armitage and Lohmann 171. With this technique they produced a color image from a black and white film in which the color object is encoded by gratings. To obtain the color image they used a white light illumination and a frequency mask that consists of opaque and transparent zones, superimposed by red, green and blue filters. Weighted mixtures of three colors have been produced with this technique [S]. Theta modulation decoders are proposed in Refs. [9,10]. Our purpose is not the recovery of the color image but the use of the
Copyright 0 I997 Elsevier Science B.V. All rights reserved
SOO30-4018(96)00525-
I
and and
78
H.J. Caulfield et al./ Optics Communicarions 133 (1997) 77-81
encoded black and white image in a monochrome single channel correlator.
2. Basic approach Two types of grating encryption occur immediately to anyone assigned this task. On one hand, we can keep the grating orientation fixed and vary some characteristic of the grating (frequency, amplitude) according to the color. Or on the other hand, while keeping the grating fixed, we can vary its orientation with the color. Obviously, mixed strategies are also possible. Since R (red), G (green) and B (blue) cameras are commercially available, we will assume hereafter RGB encryption is desired. Encryption of color is accomplished by grating orientation. The preprocessed pattern will be an addition of three gratings with different orientations, horizontal for red image, vertical for blue image and diagonal for green image. The transmission of each color image is then encoded by some characteristic of the corresponding grating, frequency, amplitude, etc. To illustrate these ideas we show two examples of encoding: (a) Transmission encoding by grating frequency. Image transmission in a single color channel can be encoded by the frequency of the corresponding grating in the preprocessed image. Zero transmission leads to zero frequency, which means constant zero background. As the transmission increases, the corresponding grating has higher frequency. The grating amplitude is kept constant. This kind of encoding makes necessary the use of high resolution SLMs to have enough different possible frequencies to be implemented. (b) Transmission encoding by grating amplitude. In this second approach, the frequency of the grating is kept constant and transmission information is encoded by means of the grating amplitude. The grating amplitude is selected to be proportional to the transmission of the corresponding color image. This latter encoding method may be suitable when low resolution SLMs are used, where frequency encoding is seriously limited by spatial resolution. As in a conventional correlator, the method is not contrast invariant. The same but brighter object will
result in a different encoded object. As the contrast varies, correlation peak will decrease. If frecuency grating encoding method is used, correlation peak may decrease quickly. If amplitude grating encoding method is used, correlation peak will not decrease so dramatically because only the amplitude on the encoded object changes, but not the shape.
3. Experimental demonstration In order to prove the performance of these encodings, we select three simple color objects. In the experiments we try to locate and recognize the color object called “target”, and distinguish it from two very similar color objects called “non-target 1” and “non-target 2”. Fig. 1 shows these three color objects and their transmission in red, green and blue channels. They consist in squared patterns with different transmission. The target and non-target 1 objects both have zero blue component. Non-target 2 object has the same red component as target object,
Fig. 1. Colored objects and red, green and blue components for the target, non-target
I
and non-target 2 objects.
HJ. Caulfield et al./ Optics Communications
Fig. 2. Preprocessed amplitude encodings.
objects
with grating
frequency
and grating
and has green component zero and blue component non-zero. These images are made up of 16 X 16 pixels. In order to get enough spatial resolution for the grating implementation, each pixel of the original image is converted into 32 X 32 pixels in the preprocessed image. resulting in a final image of 512 X 512 pix-
Fig. 3. Fourier is saturated.
transform
modulus
of the preprocessed
objects
79
133 (1997) 77-81
els. Fig. 2 shows the preprocessed objects target, non-target 1 and non-target 2 for the two types of transmission encoding, by grating frequency and by grating amplitude. Fig. 3 shows modulus of the Fourier transform of the preprocessed images with frequency and amplitude encodings where the central spots have been saturated in order to visualize the distributions. Typical diffraction orders of gratings located in directions that depend on the grating orientation are observable. Target and non-target 1 lead to similar Fourier spectra since the gratings have the same orientation in both images. Non-target 2 leads to different orientation of some diffraction orders. Correlation experiments are performed with a convergent correlator - a variant of the 4-f correlator - that uses two SLMs. The first one introduces the preprocessed image and the second one introduces an adapted filter in the Fourier plane. A good discrimination capability among the different objects and an easy implementation by means of SLM can be obtained by selecting the phase only filter. We have used two SLMs from an Epson videoprojector.
with grating
frequency
and grating
amplitude
encodings.
Central
spot
HJ. Couljield et al./Optics
80
Communications 133 (1997) 77-81
The first one, working in amplitude mode, to introduce the scene and the second one, working in phase mode, to introduce the phase only filter. Figs. 4a and 4b show computer simulated correlation peaks obtained when performing the encryption with grating frequency encoding and grating amplitude encoding respectively. Good autocorrelation peaks are obtained for the target. Some lateral peaks surrounding the autocorrelation peak appear due to the gratings periodicity. Non-target 2 result in higher cross-correlation peak since the red component is the same for this object and the target. Experimental verification obtained with optical correlator are shown in Figs. 4c and 4d. Their accordance with simulated results is quite good.
4. Discussion The primary purpose of this paper is now clearly accomplished. We have discussed the general principles whereby the new problem (recognition of full color images) can be converted to the already-solved problem (recognition of monochrome images). We have also demonstrated two versions of this system. This is a first work, not a definitive one. We have not yet analyzed the accuracy and sensitivity effects of various encryptions either in the abstract or as constrained by actual cameras and SLMs. We have not yet optimized the mask design for even one task. Of course other attributes, other than color, can be encrypted in this manner. One obvious choice is
TARGET
NON-TARGET 1
TARGET
NON-TARGET 1
NON-TARGET 1
I
ON-TARGET 2 NON-TARGET 2 Fig. 4. Correlation plane. (a) Computer simulation for grating frequency encoding. (b) Computer simulation for grating amplitude encoding. (c) Optical experiment for grating frequency encoding. (d) Optical experiment for grating amplitude encoding.
H.J. Caulfield et al./ Optics Communications
polarization. We could represent the Jones vector instant. We could even represent the spatial pattern itself on a non-point-by-point basis. For example we might define a T texture types represented by T reference patterns. We replace each point by the amplitudes of the correlations of a neighborhood centered on that point with each of the T reference textures. The applications are imagination limited.
Acknowledgements H.J. Caulfield was supported by the U.S. Army Space and Strategic Defense Command and by the U.S. Air Force Office of Scientific Research. I. Moreno, J. Campos and M.J. Yzuel were supported by the CICYT (Comisi6n Interministerial de Ciencia y Tecnologia), project TAP93-0667-C03-01 and by CIRIT (Comissi6 Interdepartamental de Recerca i Innovacio Tecnolbgica), project GRQ93-2047.
133 (1997) 77-81
81
References 111 MS. Mill&n, J. Campos, C. Ferreira and M.J. Yzuel, Optics Comm. 73 (1989) 277. l21 MS. Milltin, M.J. Yzuel, J. Campos and C. Ferreira, Appl. Optics 31 (1992) 2560. [31 C. Warde, H.J. Caulfield, F.T.S. Yu and J.E. Ludman. Optics Comm. 49 (1984) 241. [41 F.T.S. Yu, White-Light Optical Signal Processing, (Wiley, New York, 1985). [51 F.T.S. Yu and B. Javidi, Optics Comm. 56 (1986) 384. lb1F.T.S. Yu, Z. Yang and K. Pan, Appl. Optics 33 (1994) 2170. [71 J.D. Armitage and A.W. Lohmann, Appl. Optics 4 ( 1965) 399. 181 P. And& C. Ferreira, A. Pons and C. Hem&ndez, Optics Comm. 48 (1983) 103. [91 P. And&, J. Ojeda-Cast&da and J. Ibarra, Optics Comm. 60 ( 1986) 206. [lOI J. Ojeda-Castaiieda, P. And& and J. Ibarra, Optics Comm. 67 (1988) 256.
OPTICS COMMUNICATIONS ELSEVIER
Optics Communications
133 (1997) 77-81
Coherent recognition of colored patterns H.J. Caulfield a, I. Moreno b, J. Campos b, M.J. Yzuel b a Physics Department, Alubuma A &M b Departumento
de F&u.
University, Normal. AL 35762.
USA
Grupo de Optica. Universidad Auto’twmu de Burcelonu, 08193 Belluterra
Received 2 April 1996; revised version received
(Burcelonu). Spain
12 July 1996; accepted 2 August 1996
Abstract A properly preprocessed image of a colored scene can be used with conventional coherent correlators to recognize locate colored patterns. Color information is encoded by means of gratings with different orientation, frequency amplitudes. This codification permits to process polychromatic images in parallel with a monochromatic correlator.
1. Introduction The recognition of colored patterns by optical methods almost universally relies on separate processing of multiple color channels. Those results are then combined to give a score, a number used to determine the similarity between the input scene and the reference test [ 1,2]. This multichannel correlation process may be carried out sequentially or in parallel. In Ref. [3], multichannel color correlation in parallel is performed by using a tricolor grid and real-time spatial light modulator. As a result, the Fourier spectrum is located in different positions in the Fourier plane for the different wavelengths. A set of filters is introduced in this plane, each one matched to the target transmittance of the corresponding color channel. The correlations are superposed at the exit plane. Several architectures have been proposed to implement this idea (see for instance Refs. [4-61). These methods can be quite effective, but they lack the simplicity and convenience of more conventional coherent optical pattern recognizers designed to work on monochromatic or black-and-white scenes. The 0030~4018/97/$17.00 P/I
purpose of this paper is to recognize colored scenes using these coherent optical schemes. We assume that our coherent optical correlator 4f, joint transform - will use an SLM (spatial light modulator) as an input device and a laser beam for illumination. Thus, whatever it recognizes must be represented as a spatial pattern impressed on a laser beam. It follows immediately that our task is to encrypt the color information spatially and input that encrypted pattern into the system. We explore grating encryption, because it seems the simplest to us. This type of encryption is the theta modulation technique proposed by Armitage and Lohmann 171. With this technique they produced a color image from a black and white film in which the color object is encoded by gratings. To obtain the color image they used a white light illumination and a frequency mask that consists of opaque and transparent zones, superimposed by red, green and blue filters. Weighted mixtures of three colors have been produced with this technique [S]. Theta modulation decoders are proposed in Refs. [9,10]. Our purpose is not the recovery of the color image but the use of the
Copyright 0 I997 Elsevier Science B.V. All rights reserved
SOO30-4018(96)00525-
I
and and
78
H.J. Caulfield et al./ Optics Communicarions 133 (1997) 77-81
encoded black and white image in a monochrome single channel correlator.
2. Basic approach Two types of grating encryption occur immediately to anyone assigned this task. On one hand, we can keep the grating orientation fixed and vary some characteristic of the grating (frequency, amplitude) according to the color. Or on the other hand, while keeping the grating fixed, we can vary its orientation with the color. Obviously, mixed strategies are also possible. Since R (red), G (green) and B (blue) cameras are commercially available, we will assume hereafter RGB encryption is desired. Encryption of color is accomplished by grating orientation. The preprocessed pattern will be an addition of three gratings with different orientations, horizontal for red image, vertical for blue image and diagonal for green image. The transmission of each color image is then encoded by some characteristic of the corresponding grating, frequency, amplitude, etc. To illustrate these ideas we show two examples of encoding: (a) Transmission encoding by grating frequency. Image transmission in a single color channel can be encoded by the frequency of the corresponding grating in the preprocessed image. Zero transmission leads to zero frequency, which means constant zero background. As the transmission increases, the corresponding grating has higher frequency. The grating amplitude is kept constant. This kind of encoding makes necessary the use of high resolution SLMs to have enough different possible frequencies to be implemented. (b) Transmission encoding by grating amplitude. In this second approach, the frequency of the grating is kept constant and transmission information is encoded by means of the grating amplitude. The grating amplitude is selected to be proportional to the transmission of the corresponding color image. This latter encoding method may be suitable when low resolution SLMs are used, where frequency encoding is seriously limited by spatial resolution. As in a conventional correlator, the method is not contrast invariant. The same but brighter object will
result in a different encoded object. As the contrast varies, correlation peak will decrease. If frecuency grating encoding method is used, correlation peak may decrease quickly. If amplitude grating encoding method is used, correlation peak will not decrease so dramatically because only the amplitude on the encoded object changes, but not the shape.
3. Experimental demonstration In order to prove the performance of these encodings, we select three simple color objects. In the experiments we try to locate and recognize the color object called “target”, and distinguish it from two very similar color objects called “non-target 1” and “non-target 2”. Fig. 1 shows these three color objects and their transmission in red, green and blue channels. They consist in squared patterns with different transmission. The target and non-target 1 objects both have zero blue component. Non-target 2 object has the same red component as target object,
Fig. 1. Colored objects and red, green and blue components for the target, non-target
I
and non-target 2 objects.
HJ. Caulfield et al./ Optics Communications
Fig. 2. Preprocessed amplitude encodings.
objects
with grating
frequency
and grating
and has green component zero and blue component non-zero. These images are made up of 16 X 16 pixels. In order to get enough spatial resolution for the grating implementation, each pixel of the original image is converted into 32 X 32 pixels in the preprocessed image. resulting in a final image of 512 X 512 pix-
Fig. 3. Fourier is saturated.
transform
modulus
of the preprocessed
objects
79
133 (1997) 77-81
els. Fig. 2 shows the preprocessed objects target, non-target 1 and non-target 2 for the two types of transmission encoding, by grating frequency and by grating amplitude. Fig. 3 shows modulus of the Fourier transform of the preprocessed images with frequency and amplitude encodings where the central spots have been saturated in order to visualize the distributions. Typical diffraction orders of gratings located in directions that depend on the grating orientation are observable. Target and non-target 1 lead to similar Fourier spectra since the gratings have the same orientation in both images. Non-target 2 leads to different orientation of some diffraction orders. Correlation experiments are performed with a convergent correlator - a variant of the 4-f correlator - that uses two SLMs. The first one introduces the preprocessed image and the second one introduces an adapted filter in the Fourier plane. A good discrimination capability among the different objects and an easy implementation by means of SLM can be obtained by selecting the phase only filter. We have used two SLMs from an Epson videoprojector.
with grating
frequency
and grating
amplitude
encodings.
Central
spot
HJ. Couljield et al./Optics
80
Communications 133 (1997) 77-81
The first one, working in amplitude mode, to introduce the scene and the second one, working in phase mode, to introduce the phase only filter. Figs. 4a and 4b show computer simulated correlation peaks obtained when performing the encryption with grating frequency encoding and grating amplitude encoding respectively. Good autocorrelation peaks are obtained for the target. Some lateral peaks surrounding the autocorrelation peak appear due to the gratings periodicity. Non-target 2 result in higher cross-correlation peak since the red component is the same for this object and the target. Experimental verification obtained with optical correlator are shown in Figs. 4c and 4d. Their accordance with simulated results is quite good.
4. Discussion The primary purpose of this paper is now clearly accomplished. We have discussed the general principles whereby the new problem (recognition of full color images) can be converted to the already-solved problem (recognition of monochrome images). We have also demonstrated two versions of this system. This is a first work, not a definitive one. We have not yet analyzed the accuracy and sensitivity effects of various encryptions either in the abstract or as constrained by actual cameras and SLMs. We have not yet optimized the mask design for even one task. Of course other attributes, other than color, can be encrypted in this manner. One obvious choice is
TARGET
NON-TARGET 1
TARGET
NON-TARGET 1
NON-TARGET 1
I
ON-TARGET 2 NON-TARGET 2 Fig. 4. Correlation plane. (a) Computer simulation for grating frequency encoding. (b) Computer simulation for grating amplitude encoding. (c) Optical experiment for grating frequency encoding. (d) Optical experiment for grating amplitude encoding.
H.J. Caulfield et al./ Optics Communications
polarization. We could represent the Jones vector instant. We could even represent the spatial pattern itself on a non-point-by-point basis. For example we might define a T texture types represented by T reference patterns. We replace each point by the amplitudes of the correlations of a neighborhood centered on that point with each of the T reference textures. The applications are imagination limited.
Acknowledgements H.J. Caulfield was supported by the U.S. Army Space and Strategic Defense Command and by the U.S. Air Force Office of Scientific Research. I. Moreno, J. Campos and M.J. Yzuel were supported by the CICYT (Comisi6n Interministerial de Ciencia y Tecnologia), project TAP93-0667-C03-01 and by CIRIT (Comissi6 Interdepartamental de Recerca i Innovacio Tecnolbgica), project GRQ93-2047.
133 (1997) 77-81
81
References 111 MS. Mill&n, J. Campos, C. Ferreira and M.J. Yzuel, Optics Comm. 73 (1989) 277. l21 MS. Milltin, M.J. Yzuel, J. Campos and C. Ferreira, Appl. Optics 31 (1992) 2560. [31 C. Warde, H.J. Caulfield, F.T.S. Yu and J.E. Ludman. Optics Comm. 49 (1984) 241. [41 F.T.S. Yu, White-Light Optical Signal Processing, (Wiley, New York, 1985). [51 F.T.S. Yu and B. Javidi, Optics Comm. 56 (1986) 384. lb1F.T.S. Yu, Z. Yang and K. Pan, Appl. Optics 33 (1994) 2170. [71 J.D. Armitage and A.W. Lohmann, Appl. Optics 4 ( 1965) 399. 181 P. And& C. Ferreira, A. Pons and C. Hem&ndez, Optics Comm. 48 (1983) 103. [91 P. And&, J. Ojeda-Cast&da and J. Ibarra, Optics Comm. 60 ( 1986) 206. [lOI J. Ojeda-Castaiieda, P. And& and J. Ibarra, Optics Comm. 67 (1988) 256.