- Email: [email protected]
The Design Of Image Processing System Based On SOPC And OV7670
Available online at www.sciencedirect.com Available online at www.sciencedirect.com
Procedia Engineering
Procedia Engineering 00 (2011) 000–000 Procedia Engineering 24 (2011) 237 – 241 www.elsevier.com/locate/procedia
2011 International Conference on Advances in Engineering
The Design Of Image Processing System Based On SOPC And OV7670 Rui WANGa, Zengshuai MIb,Haihang YUa * , Wei YUANa, Dept. of Information Engineering ,China University of Geosciences,Beijing,10083,China Dept. of Geophysical information technology,,China University of Geosciences,Beijing,10083,China
Abstract With the development of embedded technology, digital image processing technology is widely used in the field of artificial intelligence, intelligent transportation, medical equipment, and how to achieve real-time digital image acquisition, storage and process is a key technology. This paper describes a programmable system on chip SOPC (System On a Programmable Chip) and OV7670 image sensor, using the FIFO and SDRAM under the control of the DMA to directly data-transfer, to achieve high-speed digital image capture, storage, edge detection processing.The design greatly increase the speed of the image date collectiion and storage and is better than the same type of the design especially the anti-noise ability and the effectiveness of the digital image processing
© 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of ICAE2011. Keywords: SOPC;Image Processing;FIFO;DMA
1. The overview of SOPC and image processing technology With the development of the computer and hardware,the image processing technology also has a rapid development,and the realization of the algorithm is the most important in image processing techmology. There is a new solution based on the high-speed,high-capacity Field Programmable Gate Array(FPGA) and the technology of SOPC in recent years in image processing field. The SOPC technology which combines the SOC(System On Chip) technology with the electronic design automation technology is a new design technology for embedded system. It integrates processor,memory，I/O interface,coprocessor and basic user logic into a chip of FPGA and build a programmable SOC. It is more flexible,expanded easily and scalable to the whole design of the system and realize the software and hardware programming with the SOP technology. So this system which used * Corresponding author. Tel.: +0-152-109-65669. E-mail address: [email protected]
1877-7058 © 2011 Published by Elsevier Ltd. doi:10.1016/j.proeng.2011.11.2633
2238
WANG etYU al. Wei / Procedia 24 (2011) 237 – 241 000–000 Rui WANG Zengshuai Rui MI Haihang YUAN/Engineering Procedia Engineering 00 (2011)
the NiosⅡ processor is a general embedded image processing system. It's included the hardware configuration which based on 32-bit soft-core processor NIOS Ⅱ,hardware design,hardware emulation,software design and software debug in the SOPC design. As it is different from the traditional design of the embedded system,the SOPC system development which based on Nios Ⅱis divided into two hardware development and software development process.
Fig 1. The kernel of NIOS Ⅱ
2. The structure scheme of the hardware system module The hardware of the module mainly include the main chip EP2C8Q208C8 which is the Altera's Cyclone II series chip,peripheral clock circuit,reset /reconfiguration circuit,power suplly,SDRAM,FLASH,SRAM memory circuit and so on. The cpu and all of the peripherals of NIOS II kernel is custom-designed by the tool of SOPCBuilder,the module mainly configurates the CPU of Nios,JTAG, UART,avalon tri statebridge,avalon tri statebridge,on chip memory,SDRAM controller,common flash interface. Fig. 1 shows the kernel structure of NIOS II. The CFI FLASH is used to store the code of the NIOS II sofe core,the SRAM which is very similar to the memory the computer is used to store the running program and data,SDRAM is the buffer of the collected digital image data. The JTAG debug module which provides simulation is supported by the Nios II architecture,so the processor is remotely controlled by host. JTAG achieves the following functions:download the program into the memory,start and stop execution,set the breakpoint and watchpoints, analyze the register and memory,real time data acquisition. Because the EP2C8Q208 FPGA structure based on SRAM, you can download the program into the chip to run under the condition of electrification but the data is losed when electricity of instrument turn off. Therefore, so as to save the date when power is off we expand the EPSC4 memory module which
Rui WANG et al. / YU Procedia Engineering 24 Engineering (2011) 237 –00 241 Gang DU Zengshuai MI Haihang Wei YUAN/ Procedia (2011) 000–000
undertake the hardware configuration when the system is power on.
Fig 2. The bus timing of SCCB
Fig. The initialization of OV7670
3. The design of the OV7670 of the image capture module 3.1. SCCB bus SCCB(Serial Camera Control Bus) is simplified I2C protocol, SIO-l is the input of serial clock , SIO-O is serial two-way data cable, which are respectively, equivalent to the SCL and the SDA of I2C protocol. The bus timing of SCCB is about the same as I2C's , the response signal ACK which is called the ninth bit of a transmission unit and is divided into Don't care bitand NA bit. Don't care bit is generated by the slave, NA bit is generated by the host, NA bit must be high due to the SCCB does not support multi-byte reading and writing. In addition, SCCB does not repeat the starting and ending , so the host must send bus stop conditions after it sends out the address of on-chip register in the read cycle of SCCB. Otherwise, the slave will not produce Don't care signal when sending reading command. The bus timing of SCCB is shown in Fig. 2.Initialize the OV7670 into the format of RGB565 by the way of SCCB bus and the timing diagram is shown in Fig. 3. 3.2. Image acquisition Initialize the OV7670 into the format of RGB565 by the way of SCCB bus and the timing diagram is shown in figure 4. The VSYNC generrate step changes in level when the OV7970 starts to transmit a frame image. FPGA detects the changes of the VSYNC level. When the level is high ,OV7670 finishes preparing the data,when it changes into low which means the start of a new frame image data. Decide the actual level of the HREF(horizontal reference signal) at the time of PCLK(pixel clock) generate rise edge,if it is high,the data is efficient , read the data of pixel and the X_Cont(pixel counter sign)plus 1.
Fig. 4. The design of image collection interface
Fig. 5. The structure of the DMA controller
4. The DMA transmission between FIFO and SDRAM
3239
4240
WANG etYU al. Wei / Procedia 24 (2011) 237 – 241 000–000 Rui WANG Zengshuai Rui MI Haihang YUAN/Engineering Procedia Engineering 00 (2011)
The DMA(Direct Memory Access) controller kernel which based on Avalon bus can be used for transmission of bulk data between memory and memory,memory and peripherals,peripherals and peripherals. The NIOS II gives the task of transmission of memory data DMA controller,when it starts to transmit the host peripherals can also do other task. The DMA controller is used for the DMA transmission of data from source address to destination address,the map of source and destination address may be Avalon slave peripherals or a memory. It also used to the two levels of peripherals which supports streaming module transmission and allow the transmission of data which is fixed-length and variable-length. The digital image data of OV7670 is translated form DMA to SDRAM throw the FIFO buffer and then read the data from the SDRAM through DMA to handle in the design. Fig. 5 shows the structure of the DMA controller. 5. Image PRO cessing module in the application of the robot research The application is to proceeds the data that collected by the module by the sobel algorithm and the color detection,so as the robot can recognize different objects and the robot can carry and group objects into classes. The technology can be applied in the industry and achieve high efficiency production. The Sobel operator is used in image processing, particularly within edge detection algorithms. Mathematically, the operator uses two 3×3 kernels which are convolved with the original image to calculate approximations of the derivatives - one for horizontal changes, and one for vertical. If we define A as the source image, and Gx and Gy are two images which at each point contain the horizontal and vertical derivative approximations, the computations are as follows:
1 0 1 G X 2 0 2 * A 1 0 1
1 2 1 G y 0 0 0 * A 1 2 1
(1)
(2)
The x-coordinate is here defined as increasing in the "right"-direction, and the y-coordinate is defined as increasing in the "down"-direction. At each point in the image, the resulting gradient approximations can be combined to give the gradient magnitude, using:
G
Gx2 G y2
(3)
We can get the contour of the object and conform the coordinates within normal field of vision after the processing of the sobel algorithm then filter the background color and get the color characters of the object at last. Feed the coordinates and the color characters into the mcu so as to control the action of the robot. The application flow diagrams of the module is shown in Fig. 6
Rui WANG et al. / YU Procedia Engineering 24 Engineering (2011) 237 –00 241 Gang DU Zengshuai MI Haihang Wei YUAN/ Procedia (2011) 000–000
Fig 6. The application flow diagrams of the module
6. Conclutions This paper puts forward the a image processing design scheme based on SOPC and OV7670,it greatly increase the speed of the image date collectiion and storage and is better than the same type of the design especially the anti-noise ability and the effectiveness of the digital image processing. The robot based on the design did very well in the national robot competition and open Robocup. With the development of the image processing technology, the more complex processing technology will appear constantly,the system in the paper Will speed up research and implementation in image processing system and certainly to have a better development compared with other system. Acknowledgment Thanks for the support of the 2011 annual basic scientific research business expenses special funds of China university of geosciences (Beijing) and the 2011 annual undergraduates innovative experimental project ,The author gratefully acknowledges associate the teachers and studensts for their helpful suggestions and comments. References [1] Altera Corp, Nios II Processor Ref erence Handbook[M] . Altera, 2005. [2] Altera Corp., Nios II Software Developer's Handbook [ M] . Altera, 2005. [3] Altera Corp., SOPC Builder Data Sheet [M] .Alt era, 2005. [4] Bai Yang; Research on Design Strategy of SOPC[J]. Computer Development & Applications, 2007-11 . [5] Christors Davatzikors, "Spatial Transformation and Registration of Brain Images Using Elastically Deformable Models,"in Computer version and Image Understanding. Vol.66, No.2 Mav1991, pp. 207-222. [6] Manish Kumar Birla, "FPGA Based Reconfigurable Platform for Complex Image Proeessing," Technology, 2006 IEEE International Conference , pp. 204-209, May 2006.
Eleetro/information
[7] F Javier Toledo,J.Javier Martlnez, J.Manuel Ferrandez, "FPGA-Based Platform for Image and Video Proeessing Embedded Systems," Programmable Logic,2007.SPL '07.20073rd Southern Conference , pp. 171 一 176, Feb 2007. [8] Takada H, Honda S, NIShiy, Nishiyama R ， Yuyama H, "Hardware/Software co-configuration for Multiprocessor SOPC(work-in-progress report)Software Technologies for Future Embedded Systems," 2003.IEEE Workshop, pp.7-8, May 2003 .
5241
Procedia Engineering
Procedia Engineering 00 (2011) 000–000 Procedia Engineering 24 (2011) 237 – 241 www.elsevier.com/locate/procedia
2011 International Conference on Advances in Engineering
The Design Of Image Processing System Based On SOPC And OV7670 Rui WANGa, Zengshuai MIb,Haihang YUa * , Wei YUANa, Dept. of Information Engineering ,China University of Geosciences,Beijing,10083,China Dept. of Geophysical information technology,,China University of Geosciences,Beijing,10083,China
Abstract With the development of embedded technology, digital image processing technology is widely used in the field of artificial intelligence, intelligent transportation, medical equipment, and how to achieve real-time digital image acquisition, storage and process is a key technology. This paper describes a programmable system on chip SOPC (System On a Programmable Chip) and OV7670 image sensor, using the FIFO and SDRAM under the control of the DMA to directly data-transfer, to achieve high-speed digital image capture, storage, edge detection processing.The design greatly increase the speed of the image date collectiion and storage and is better than the same type of the design especially the anti-noise ability and the effectiveness of the digital image processing
© 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of ICAE2011. Keywords: SOPC;Image Processing;FIFO;DMA
1. The overview of SOPC and image processing technology With the development of the computer and hardware,the image processing technology also has a rapid development,and the realization of the algorithm is the most important in image processing techmology. There is a new solution based on the high-speed,high-capacity Field Programmable Gate Array(FPGA) and the technology of SOPC in recent years in image processing field. The SOPC technology which combines the SOC(System On Chip) technology with the electronic design automation technology is a new design technology for embedded system. It integrates processor,memory，I/O interface,coprocessor and basic user logic into a chip of FPGA and build a programmable SOC. It is more flexible,expanded easily and scalable to the whole design of the system and realize the software and hardware programming with the SOP technology. So this system which used * Corresponding author. Tel.: +0-152-109-65669. E-mail address: [email protected]
1877-7058 © 2011 Published by Elsevier Ltd. doi:10.1016/j.proeng.2011.11.2633
2238
WANG etYU al. Wei / Procedia 24 (2011) 237 – 241 000–000 Rui WANG Zengshuai Rui MI Haihang YUAN/Engineering Procedia Engineering 00 (2011)
the NiosⅡ processor is a general embedded image processing system. It's included the hardware configuration which based on 32-bit soft-core processor NIOS Ⅱ,hardware design,hardware emulation,software design and software debug in the SOPC design. As it is different from the traditional design of the embedded system,the SOPC system development which based on Nios Ⅱis divided into two hardware development and software development process.
Fig 1. The kernel of NIOS Ⅱ
2. The structure scheme of the hardware system module The hardware of the module mainly include the main chip EP2C8Q208C8 which is the Altera's Cyclone II series chip,peripheral clock circuit,reset /reconfiguration circuit,power suplly,SDRAM,FLASH,SRAM memory circuit and so on. The cpu and all of the peripherals of NIOS II kernel is custom-designed by the tool of SOPCBuilder,the module mainly configurates the CPU of Nios,JTAG, UART,avalon tri statebridge,avalon tri statebridge,on chip memory,SDRAM controller,common flash interface. Fig. 1 shows the kernel structure of NIOS II. The CFI FLASH is used to store the code of the NIOS II sofe core,the SRAM which is very similar to the memory the computer is used to store the running program and data,SDRAM is the buffer of the collected digital image data. The JTAG debug module which provides simulation is supported by the Nios II architecture,so the processor is remotely controlled by host. JTAG achieves the following functions:download the program into the memory,start and stop execution,set the breakpoint and watchpoints, analyze the register and memory,real time data acquisition. Because the EP2C8Q208 FPGA structure based on SRAM, you can download the program into the chip to run under the condition of electrification but the data is losed when electricity of instrument turn off. Therefore, so as to save the date when power is off we expand the EPSC4 memory module which
Rui WANG et al. / YU Procedia Engineering 24 Engineering (2011) 237 –00 241 Gang DU Zengshuai MI Haihang Wei YUAN/ Procedia (2011) 000–000
undertake the hardware configuration when the system is power on.
Fig 2. The bus timing of SCCB
Fig. The initialization of OV7670
3. The design of the OV7670 of the image capture module 3.1. SCCB bus SCCB(Serial Camera Control Bus) is simplified I2C protocol, SIO-l is the input of serial clock , SIO-O is serial two-way data cable, which are respectively, equivalent to the SCL and the SDA of I2C protocol. The bus timing of SCCB is about the same as I2C's , the response signal ACK which is called the ninth bit of a transmission unit and is divided into Don't care bitand NA bit. Don't care bit is generated by the slave, NA bit is generated by the host, NA bit must be high due to the SCCB does not support multi-byte reading and writing. In addition, SCCB does not repeat the starting and ending , so the host must send bus stop conditions after it sends out the address of on-chip register in the read cycle of SCCB. Otherwise, the slave will not produce Don't care signal when sending reading command. The bus timing of SCCB is shown in Fig. 2.Initialize the OV7670 into the format of RGB565 by the way of SCCB bus and the timing diagram is shown in Fig. 3. 3.2. Image acquisition Initialize the OV7670 into the format of RGB565 by the way of SCCB bus and the timing diagram is shown in figure 4. The VSYNC generrate step changes in level when the OV7970 starts to transmit a frame image. FPGA detects the changes of the VSYNC level. When the level is high ,OV7670 finishes preparing the data,when it changes into low which means the start of a new frame image data. Decide the actual level of the HREF(horizontal reference signal) at the time of PCLK(pixel clock) generate rise edge,if it is high,the data is efficient , read the data of pixel and the X_Cont(pixel counter sign)plus 1.
Fig. 4. The design of image collection interface
Fig. 5. The structure of the DMA controller
4. The DMA transmission between FIFO and SDRAM
3239
4240
WANG etYU al. Wei / Procedia 24 (2011) 237 – 241 000–000 Rui WANG Zengshuai Rui MI Haihang YUAN/Engineering Procedia Engineering 00 (2011)
The DMA(Direct Memory Access) controller kernel which based on Avalon bus can be used for transmission of bulk data between memory and memory,memory and peripherals,peripherals and peripherals. The NIOS II gives the task of transmission of memory data DMA controller,when it starts to transmit the host peripherals can also do other task. The DMA controller is used for the DMA transmission of data from source address to destination address,the map of source and destination address may be Avalon slave peripherals or a memory. It also used to the two levels of peripherals which supports streaming module transmission and allow the transmission of data which is fixed-length and variable-length. The digital image data of OV7670 is translated form DMA to SDRAM throw the FIFO buffer and then read the data from the SDRAM through DMA to handle in the design. Fig. 5 shows the structure of the DMA controller. 5. Image PRO cessing module in the application of the robot research The application is to proceeds the data that collected by the module by the sobel algorithm and the color detection,so as the robot can recognize different objects and the robot can carry and group objects into classes. The technology can be applied in the industry and achieve high efficiency production. The Sobel operator is used in image processing, particularly within edge detection algorithms. Mathematically, the operator uses two 3×3 kernels which are convolved with the original image to calculate approximations of the derivatives - one for horizontal changes, and one for vertical. If we define A as the source image, and Gx and Gy are two images which at each point contain the horizontal and vertical derivative approximations, the computations are as follows:
1 0 1 G X 2 0 2 * A 1 0 1
1 2 1 G y 0 0 0 * A 1 2 1
(1)
(2)
The x-coordinate is here defined as increasing in the "right"-direction, and the y-coordinate is defined as increasing in the "down"-direction. At each point in the image, the resulting gradient approximations can be combined to give the gradient magnitude, using:
G
Gx2 G y2
(3)
We can get the contour of the object and conform the coordinates within normal field of vision after the processing of the sobel algorithm then filter the background color and get the color characters of the object at last. Feed the coordinates and the color characters into the mcu so as to control the action of the robot. The application flow diagrams of the module is shown in Fig. 6
Rui WANG et al. / YU Procedia Engineering 24 Engineering (2011) 237 –00 241 Gang DU Zengshuai MI Haihang Wei YUAN/ Procedia (2011) 000–000
Fig 6. The application flow diagrams of the module
6. Conclutions This paper puts forward the a image processing design scheme based on SOPC and OV7670,it greatly increase the speed of the image date collectiion and storage and is better than the same type of the design especially the anti-noise ability and the effectiveness of the digital image processing. The robot based on the design did very well in the national robot competition and open Robocup. With the development of the image processing technology, the more complex processing technology will appear constantly,the system in the paper Will speed up research and implementation in image processing system and certainly to have a better development compared with other system. Acknowledgment Thanks for the support of the 2011 annual basic scientific research business expenses special funds of China university of geosciences (Beijing) and the 2011 annual undergraduates innovative experimental project ,The author gratefully acknowledges associate the teachers and studensts for their helpful suggestions and comments. References [1] Altera Corp, Nios II Processor Ref erence Handbook[M] . Altera, 2005. [2] Altera Corp., Nios II Software Developer's Handbook [ M] . Altera, 2005. [3] Altera Corp., SOPC Builder Data Sheet [M] .Alt era, 2005. [4] Bai Yang; Research on Design Strategy of SOPC[J]. Computer Development & Applications, 2007-11 . [5] Christors Davatzikors, "Spatial Transformation and Registration of Brain Images Using Elastically Deformable Models,"in Computer version and Image Understanding. Vol.66, No.2 Mav1991, pp. 207-222. [6] Manish Kumar Birla, "FPGA Based Reconfigurable Platform for Complex Image Proeessing," Technology, 2006 IEEE International Conference , pp. 204-209, May 2006.
Eleetro/information
[7] F Javier Toledo,J.Javier Martlnez, J.Manuel Ferrandez, "FPGA-Based Platform for Image and Video Proeessing Embedded Systems," Programmable Logic,2007.SPL '07.20073rd Southern Conference , pp. 171 一 176, Feb 2007. [8] Takada H, Honda S, NIShiy, Nishiyama R ， Yuyama H, "Hardware/Software co-configuration for Multiprocessor SOPC(work-in-progress report)Software Technologies for Future Embedded Systems," 2003.IEEE Workshop, pp.7-8, May 2003 .
5241