- Email: [email protected]
A neuromorphic parallel vision with orientation selective receptive field
International Congress Series 1269 (2004) 181 – 184
www.ics-elsevier.com
A neuromorphic parallel vision with orientation selective receptive field Kazuhiro Shimonomura *, Tetsuya Yagi Department of Electronic Engineering, Graduate School of Engineering, Osaka University, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan
Abstract. We have developed a vision system which emulates the orientation selective receptive fields of simple cells in the primary visual cortex. The system consists of a silicon retina and a Field Programmable Gate Array (FPGA). First, the image is filtered by the Laplacian – Gaussianlike receptive field of the silicon retina, and then it is transferred to the FPGA. The orientation selection chip selectively aggregates multiple pixels of the silicon retina, mimicking the feedforward model proposed by Hubel and Wiesel. The present system exhibits the orientation selectivity with even and odd-type responses. The design of the present chip is applicable to implement higher order cells in the primary visual cortex, such as the complex cell. D 2004 Published by Elsevier B.V. Keywords: Visual cortex; Simple cell; Orientation selectivity; Silicon retina; FPGA
1. Introduction In the visual system of the brain, the image is perceived stably in a short time, although the response speed of neurons is extremely slow in comparison to that of semiconductor devices. Moreover, the energy consumption of the neuronal networks is much lower than that of the current digital computer. One of the reasons for such a high computational performance of the brain can be attributed to its hierarchical architecture. In the present study, we developed a vision system to emulate the orientation selective response of the simple cell in the primary visual cortex, aiming at a novel image-processing device, which can be applied to real-time image computations. Previously, several neuromorphic VLSIs with multi-chip configurations have already been fabricated to emulate the receptive field properties of primary visual cortical cells. These systems use the address-event representation (AER) protocol for inter-chip
* Corresponding author. Tel.: +81-6-6879-7785; fax: +81-6-6879-7784. E-mail address: [email protected] (K. Shimonomura). 0531-5131/ D 2004 Published by Elsevier B.V. doi:10.1016/j.ics.2004.05.128
182
K. Shimonomura, T. Yagi / International Congress Series 1269 (2004) 181–184
communication [1,2]. Here we used a more simple technique for inter-chip communication, considering its utility for practical engineering applications. 2. Model Convergence of the response of neurons having a center-surround receptive field can produce an elongated, orientation selective receptive field [3], which is known as the feedforward model. According to recent physiological studies, it is thought that the mechanism for generating the orientation selectivity is more complex. Although this model does not thoroughly explain all properties of a simple cell [4], the results of physiological experiments qualitatively support this idea. Therefore, we used a feedforward connection as the basic structure to realize orientation selectivity. Fig. 1 shows the feedforward model. The multiple pixels, located on a straight line with preferred orientation, are averaged. The receptive field produced by this model is the eventype. The odd-type receptive fields can be obtained by combining neighboring even-type pixels, which are tuned to the desired orientation. 3. System structure Fig. 2 shows a block diagram of the vision system used in the present work. The system consists of a silicon retina, an AD converter and a Field Programmable Gate Array (FPGA). The silicon retina used here has been developed by Kameda and Yagi [5]. The pixel circuit in this chip is arranged in a hexagonal grid. Neighboring pixels are connected by two layers of resistive networks based on the previous model of the vertebrate outer retinal circuit [6]. The spatial response of the silicon retina exhibits a Laplacian– Gaussianlike receptive field. An external image is filtered by the receptive field of the silicon retina. The output of the silicon retina, obtained as analog voltage, is read out pixel by pixel, and converted to 8bit digital signals. Subsequently, the digital signals are sent to the FPGA (Xilinx XCV150). The FPGA circuits aggregate multiple pixels of the retina, mimicking the feedforward model shown in Fig. 1. The configured circuit can generate the response
Fig. 1. A model to obtain an orientation selective receptive field based on the feedforward model proposed by Hubel and Wiesel. N is the number of aggregated pixels.
K. Shimonomura, T. Yagi / International Congress Series 1269 (2004) 181–184
183
Fig. 2. Block diagram of the vision system, which consists of a silicon retina and an FPGA.
orienting at 0j, 60j and 120j selective response. The FPGA circuits also provide the control signals to drive the silicon retina. 4. Results 4.1. Response to a simple image A white bar on a black background was presented to the system. The orientation selective responses of the system are shown in Fig. 3. The white bar is oriented at 0j. The preferred orientation of the system was also set to 0j. The resolution of the silicon retina used here was 40 46 pixels. A is the even-type response. B is the odd-type response generated by subtraction between two even-type responses. For each case, responses of 8 pixels are aggregated. The excited and inhibited subregions appear side by side alternately, similar to the elongated receptive field of simple cells in the primary visual cortex. Low noise responses are obtained because the noise compensation circuit is embedded in the silicon retina. 4.2. Response to a natural image The response to a natural scene was verified under indoor illumination (0.18 W/m2). Here we used a higher resolution version of the silicon retina, which has 100 100 pixels. Accumulation time of the silicon retina was 33 ms. Fig. 4 shows the responses of the system to a hand in a laboratory scene. A is the output of the silicon retina. The edges in
Fig. 3. Responses of the system to a slit pattern. A and B are even-type and odd-type responses, respectively.
184
K. Shimonomura, T. Yagi / International Congress Series 1269 (2004) 181–184
Fig. 4. Response of the system to a hand. A is the output of the silicon retina, B is the odd-type response preferred to 0j orientation (horizontal orientation).
the image are enhanced with a Mach band-like effect. B is the odd-type orientation selective output preferred to horizontal orientation. Only horizontal edges are enhanced, while others such as vertical edges of the monitor are blurred. 5. Conclusions In this study, we have configured the vision system, which consists of a silicon retina and an FPGA. This system provides an orientation selective receptive field constructed by the hierarchical architecture based on the feedforward model. The system can produce the orientation selective response to natural images in real time under indoor illumination. To implement a large-scale parallel vision system, we will design analog VLSI chips, which exhibit orientation selective receptive fields. Acknowledgements The VLSI chip in this study was fabricated in the chip fabrication program of VLSI Design and Education Center (VDEC), the University of Tokyo with collaboration by Rohm and Toppan Printing. References [1] P. Venier, et al., An integrate cortical layer for orientation enhancement, IEEE Journal of Solid-State Circuits 32 (2) (1997) 177 – 185. [2] S. Liu, et al., Orientation-selective a VLSI spiking neurons, Neural Networks 14 (2001) 629 – 643. [3] D. Hubel, T. Wiesel, Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex, Journal of Physiology (London) 160 (1962) 106 – 154. [4] D. Somers, et al., A local circuit approach to understanding integration of long-term inputs in primary visual cortex, Cerebral Cortex 8 (1998) 204 – 217. [5] S. Kameda, T. Yagi, An analog VLSI chip emulating sustained and transient response channels of the vertebrate retina, IEEE Transactions on Neural Networks 14 (5) (2003) 1405 – 1412. [6] T. Yagi, S. Ohshima, Y. Funahashi, The role of retinal bipolar cell in early vision: an implication with analogue networks and regularization theory, Biological Cybernetics 77 (1997) 163 – 171.
www.ics-elsevier.com
A neuromorphic parallel vision with orientation selective receptive field Kazuhiro Shimonomura *, Tetsuya Yagi Department of Electronic Engineering, Graduate School of Engineering, Osaka University, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan
Abstract. We have developed a vision system which emulates the orientation selective receptive fields of simple cells in the primary visual cortex. The system consists of a silicon retina and a Field Programmable Gate Array (FPGA). First, the image is filtered by the Laplacian – Gaussianlike receptive field of the silicon retina, and then it is transferred to the FPGA. The orientation selection chip selectively aggregates multiple pixels of the silicon retina, mimicking the feedforward model proposed by Hubel and Wiesel. The present system exhibits the orientation selectivity with even and odd-type responses. The design of the present chip is applicable to implement higher order cells in the primary visual cortex, such as the complex cell. D 2004 Published by Elsevier B.V. Keywords: Visual cortex; Simple cell; Orientation selectivity; Silicon retina; FPGA
1. Introduction In the visual system of the brain, the image is perceived stably in a short time, although the response speed of neurons is extremely slow in comparison to that of semiconductor devices. Moreover, the energy consumption of the neuronal networks is much lower than that of the current digital computer. One of the reasons for such a high computational performance of the brain can be attributed to its hierarchical architecture. In the present study, we developed a vision system to emulate the orientation selective response of the simple cell in the primary visual cortex, aiming at a novel image-processing device, which can be applied to real-time image computations. Previously, several neuromorphic VLSIs with multi-chip configurations have already been fabricated to emulate the receptive field properties of primary visual cortical cells. These systems use the address-event representation (AER) protocol for inter-chip
* Corresponding author. Tel.: +81-6-6879-7785; fax: +81-6-6879-7784. E-mail address: [email protected] (K. Shimonomura). 0531-5131/ D 2004 Published by Elsevier B.V. doi:10.1016/j.ics.2004.05.128
182
K. Shimonomura, T. Yagi / International Congress Series 1269 (2004) 181–184
communication [1,2]. Here we used a more simple technique for inter-chip communication, considering its utility for practical engineering applications. 2. Model Convergence of the response of neurons having a center-surround receptive field can produce an elongated, orientation selective receptive field [3], which is known as the feedforward model. According to recent physiological studies, it is thought that the mechanism for generating the orientation selectivity is more complex. Although this model does not thoroughly explain all properties of a simple cell [4], the results of physiological experiments qualitatively support this idea. Therefore, we used a feedforward connection as the basic structure to realize orientation selectivity. Fig. 1 shows the feedforward model. The multiple pixels, located on a straight line with preferred orientation, are averaged. The receptive field produced by this model is the eventype. The odd-type receptive fields can be obtained by combining neighboring even-type pixels, which are tuned to the desired orientation. 3. System structure Fig. 2 shows a block diagram of the vision system used in the present work. The system consists of a silicon retina, an AD converter and a Field Programmable Gate Array (FPGA). The silicon retina used here has been developed by Kameda and Yagi [5]. The pixel circuit in this chip is arranged in a hexagonal grid. Neighboring pixels are connected by two layers of resistive networks based on the previous model of the vertebrate outer retinal circuit [6]. The spatial response of the silicon retina exhibits a Laplacian– Gaussianlike receptive field. An external image is filtered by the receptive field of the silicon retina. The output of the silicon retina, obtained as analog voltage, is read out pixel by pixel, and converted to 8bit digital signals. Subsequently, the digital signals are sent to the FPGA (Xilinx XCV150). The FPGA circuits aggregate multiple pixels of the retina, mimicking the feedforward model shown in Fig. 1. The configured circuit can generate the response
Fig. 1. A model to obtain an orientation selective receptive field based on the feedforward model proposed by Hubel and Wiesel. N is the number of aggregated pixels.
K. Shimonomura, T. Yagi / International Congress Series 1269 (2004) 181–184
183
Fig. 2. Block diagram of the vision system, which consists of a silicon retina and an FPGA.
orienting at 0j, 60j and 120j selective response. The FPGA circuits also provide the control signals to drive the silicon retina. 4. Results 4.1. Response to a simple image A white bar on a black background was presented to the system. The orientation selective responses of the system are shown in Fig. 3. The white bar is oriented at 0j. The preferred orientation of the system was also set to 0j. The resolution of the silicon retina used here was 40 46 pixels. A is the even-type response. B is the odd-type response generated by subtraction between two even-type responses. For each case, responses of 8 pixels are aggregated. The excited and inhibited subregions appear side by side alternately, similar to the elongated receptive field of simple cells in the primary visual cortex. Low noise responses are obtained because the noise compensation circuit is embedded in the silicon retina. 4.2. Response to a natural image The response to a natural scene was verified under indoor illumination (0.18 W/m2). Here we used a higher resolution version of the silicon retina, which has 100 100 pixels. Accumulation time of the silicon retina was 33 ms. Fig. 4 shows the responses of the system to a hand in a laboratory scene. A is the output of the silicon retina. The edges in
Fig. 3. Responses of the system to a slit pattern. A and B are even-type and odd-type responses, respectively.
184
K. Shimonomura, T. Yagi / International Congress Series 1269 (2004) 181–184
Fig. 4. Response of the system to a hand. A is the output of the silicon retina, B is the odd-type response preferred to 0j orientation (horizontal orientation).
the image are enhanced with a Mach band-like effect. B is the odd-type orientation selective output preferred to horizontal orientation. Only horizontal edges are enhanced, while others such as vertical edges of the monitor are blurred. 5. Conclusions In this study, we have configured the vision system, which consists of a silicon retina and an FPGA. This system provides an orientation selective receptive field constructed by the hierarchical architecture based on the feedforward model. The system can produce the orientation selective response to natural images in real time under indoor illumination. To implement a large-scale parallel vision system, we will design analog VLSI chips, which exhibit orientation selective receptive fields. Acknowledgements The VLSI chip in this study was fabricated in the chip fabrication program of VLSI Design and Education Center (VDEC), the University of Tokyo with collaboration by Rohm and Toppan Printing. References [1] P. Venier, et al., An integrate cortical layer for orientation enhancement, IEEE Journal of Solid-State Circuits 32 (2) (1997) 177 – 185. [2] S. Liu, et al., Orientation-selective a VLSI spiking neurons, Neural Networks 14 (2001) 629 – 643. [3] D. Hubel, T. Wiesel, Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex, Journal of Physiology (London) 160 (1962) 106 – 154. [4] D. Somers, et al., A local circuit approach to understanding integration of long-term inputs in primary visual cortex, Cerebral Cortex 8 (1998) 204 – 217. [5] S. Kameda, T. Yagi, An analog VLSI chip emulating sustained and transient response channels of the vertebrate retina, IEEE Transactions on Neural Networks 14 (5) (2003) 1405 – 1412. [6] T. Yagi, S. Ohshima, Y. Funahashi, The role of retinal bipolar cell in early vision: an implication with analogue networks and regularization theory, Biological Cybernetics 77 (1997) 163 – 171.