TMCnet News
Embedded ARM9 Image Acquisition System Based on CMOS Image Sensor [Sensors & Transducers (Canada)](Sensors & Transducers (Canada) Via Acquire Media NewsEdge) Abstract: This article is based on CMOS image sensor which is commonly used in image acquisition. It applies 32-bit ARM9 microprocessor S3C2410A as the CPU to control other function modules and designs embedded Arm9 image acquisition system for the realization of machine vision. The main function modules are SDRAM memory cell, image acquisition unit and Ethernet transmission module, UART serial port communication module, Flash module, power supply module, etc. Compared with machine vision systems of the traditional "image acquisition card - PC - terminal control device" model, the image acquisition system has advantages of small volume, low cost, low power consumption, strong real-time performance, etc. And it can be used for practical application in video image monitoring, automatic detection, medical and military detection and so on. In a word, it has a good application prospect. Copyright © 2013 IFSA. Keywords: CMOS Image Sensors, Embedded, ARM9, Image Acquisition. 1. Introduction Machine vision has been widely used in the fields of national economy, scientific research and national defense construction and so on. In the process of mass industrial production, product quality with artificial visual inspection is of low efficiency and precision is not high, while using machine vision can improve production efficiency and automation degree. Image acquisition system is an important part of the machine vision system. Currently two kinds of image sensors commonly used for image acquisition are CCD and CMOS image sensors [2], CCD generally outputs with analog signals which needs video decoding to get the digital signal to be introduced to the microprocessor. However, CMOS image sensor outputs digital signals directly and can be directly connected with the microprocessor [1], The performance of different CMOS image sensors is different. Mainly displays in the difference of size of the image resolution, frame rate, exposure mode, etc. CMOS image sensor can directly set the image resolution size and exposure, gain and other parameters through PC. While CCD image sensor is required to set the video decoder to control the exposure, gain and other parameters of the image. Compared with the CCD image sensor, CMOS image sensor has the advantage of low power consumption, small volume, high speed data transmission and convenient control, etc. Therefore, CMOS image sensor is more suitable for embedded system application [5]. This paper adopts the 32-bit ARM microprocessor, CMOS image sensor and CPLD as the core components and designed the embedded Arm9 image acquisition system which faces the machine vision domain, with its main function modules as SDRAM memory cell, image acquisition unit, Ethernet transmission module, UART serial port communication module, Flash module, power supply module, etc. Compared with the traditional "image acquisition card - PC - terminal control device" model machine vision system, it has the advantages of small size, low cost, low power consumption, strong real-time performance, flexible design, etc. The experimental test results show that the hardware platform design of image acquisition system is reasonable and feasible and with practical reference value. The system can be applied to the video image monitoring, image automatic detection, medical and military detection and other fields, and it has good application prospect. 2. System Structure A typical machine vision system generally includes image acquisition module, image digital module, digital image processing module, light source system, intelligent decision module and mechanical control execution module[3]. The speed of image acquisition and digital image processing module is the performance index for evaluation of hardware design of embedded vision system. There are the following four kinds of common embedded vision system structure: 1) USB interface camera structure. Image acquisition hardware adopts CMOS camera with USB interface. The camera should be directly connected with integrated CPU interface of USB. USB interface can realize high speed serial communication which requires the development of dedicated drive, thus the construction difficulty of the software is greatly increased. 2) Asynchronous FIFO structure. Adopts asynchronous FIFO between the image sensor and master CPU to solve the mismatching of sensor output frequency and master CPU data acquisition frequency. When FIFO is full, CPU realizes the quick read of FIFO data. And it can be dual-port RAM or developed in FPGA. 3) CPLD-cored image acquisition structure. SRAM read and write is directly controlled by CPLD, according to the sequence signals output by image sensor. When one frame of data information collection is completed, send signal to CPU. And CPU needs to process the image and then reads data in ARAM by CPLD. 4) Image acquisition structure by external interrupt. Frame synchronization, line synchronization, and picture element synchronization signal of image sensor are respectively connected to an external interrupt connection of the CPU. The CPU according to the number of interrupt to judge whether a frame image acquisition is completed. By comparison and analysis of several schemes, combined with factors as practical application and developing difficulty, the system structure is shown in Fig. 1. The system consists of three parts of image acquisition, image cache and image processing. In image cache, CPLD shifts the 10-bit data to 32-bit data. Then by external request DMA, the 32-bit data are sent to RAM for preservation by DMA controller of master CPU. This can reduce the wasting of resources in the common low speed image acquisition and also avoid the CPU response lag. 3. Hardware Design Overall system hardware structure is shown in Fig. 2. Image acquisition adopts CMOS monochromatic image sensor KAC9638 of Kodak company. The image cache consists of CPLD chip XC95144 and external expansion SDRM (HY5 7V561620). Hard cores of image processing adopts ARM9(S3C2410A) of Samsung company and extends the UART, Ethernet communication interface, USB port and LCD human-machine interface which is convenient for system debugging and exchange data with other equipment. Workflow of the hardware system is as follows: 1) After the system is powered on, ARM9 initializes image sensor by I2C interface and waits for PC orders. 2) After receiving the PC command, ARM9 first initializes DMA, then sends acquisition command to CPLD. CPLD boots image acquisition and applies DMA of ARM9. And a frame of image data is saved in SDRAM of ARM9. 3) After finishing the image acquisition, the process of ARM9 entering DMA is completed. Then image processing is carried out and the results output by a serial port. 3.1. Memory Selection Addressable storage space of S3C2410A chip is 1 GB and it is divided into eight storage blocks with each storage space of 128 MB. Each storage block is generated by decoding of chip select signal nGCSOnGCS7. The data bus pins are DATA0-DATA31, with a total number of 32. It can be set to data width of 8 bit / 16 bit / 32 bit. The address bus pins are ADDR0-ADDR26, with a total number of 27 and support for 128 MB of space. S3C2410A is 32-bit microprocessor and its external bus is also 32 bits. 32-bit storage system should be adopted to give full play to its 32-bit bus performance advantages. The method is to make parallel connection of two Flash memory chips with 16 bits data width or one Flash memory chip with 32-bit data width. This design adopts two pieces of Intel E28F128J3A to compose the NOR Flash of 16 M x 32 bit. One is of high 16-bits, another of low 16-bits. A SDRAM system of 16 M x 32 bit is made up by two half-word SDRAM (HY57V561620) of Samsung company, to improve the efficiency of its communication with the CPU. 3.2. Communication Interface Design 3.2.1. UART and USB Interface Design UART of S3C2410A provides three independent asynchronous serial I/O port. Each serial port can work independently in interrupt mode and the DMA mode. UART uses the system clock and supports data communication of maximum 230.4 KB/s baud rate. Each UART serial port provides two 16 B FIFO respectively used for sending and receiving buffer. This system adopts the MAX202 chip as the receiver/driver of RS - 232. Connects UARTO and UART1 to 9DB connector and communicates with DSP and PC. S3C2410A has two USB master devices and a USB slave unit. USB device controller allows the DMA mode bulk transfer, interrupt transfer and control transfer. In system design, USB device is for communication and PC machine for debugging. So slave unit interface (compatible USBVerl.l standard) is adopted. 3.2.2. Ethernet Interface Circuit Design There is no Ethernet front-end port on S3C2410A piece, so Ethernet controller must be added to extend. The system adopts 16-bit Ethernet controllerCS8900A and its characteristics is: meets IEEE802.3 standard and supports the full duplex with sending and receiving rate up to 10 MB/s. Built-in SRAM to send and receive buffer, which reduces the overhead of the main processor. Ethernet circuit is shown in Fig. 3. S3C2410A connects CA8900A by 16 bit data line and 20 bit address line. Chip select signal is nGCS3, which means to map the internal registers and frame buffer of CS8900A to the 4 KB storage area of Band4 in S3C2410A. Host computer can have direct access to internal registers and frame buffer of CS8900A through the storage space. CS8900A and RJ - 45 interface is connected directly to a network transformer, which plays a role of level switch and electrical isolation. In addition, CS8900A provides two operating modes: The I/O mode and memory mode. This system selects the work mode by CPLD. 3.3. Circuit Design of Image Acquisition Module Image sensor adopts CMOS monochromatic image sensor KAC9638 produced by Kodak company. KAC9638 is high performance, low power consumption, active pixel sensor with characteristics as follows: 1) KAC9638 is a 1/2 inch, 1024 x 1280 effective image array CMOS image sensor. Under the condition of meeting the resolution ratio of the application needs, it also has a good dynamic range (55 db), minimum illumination (2.40 / Lux - SEC), etc. 2) KAC9638 has a built-in 10 bit A/D converter, directly outputs 8 or 10 bit digital gray value. It ensures accuracy without the need for A/D converter and peripheral circuit is therefore greatly simplified. 3) KAC9638 has good power management function and low power consumption characteristics. The total power consumption of the chip is 150 mw when the data are transferred. In addition, in order to further reduce the power consumption of the system, the chip also has a programmable " power on" and " power off' mode. CMOS image sensor generally adopts I2C bus at present. This system is connected with SCL, SDA of KAC9638 through FC bus of S3C2410A. Completed by S3C2410A directly the initialization of an image sensor. KAC9638 embedded a 10 bit A/D converter, thus it can synchronously output 10 bit digital video stream. But S3C2410A has a 32-bit external bus, 10 data being directly connected to the bus can cause waste of resources and reduce the CPU efficiency. Therefore, CPLD is connected between KAC9638 and S3C2410A. Completed by CPLD the transformation from 10-bit video data to 32-bit video data. Image acquisition interface circuit structure diagram is shown in Fig. 4. To make S3C2410A read the image sensor data like an access memory, map the CPLD to S3C2410A storage unit. CPLD image cache chip select signal is nGCS4, with address mapping to Bank4 of S3C2410A, address range: 0x20000000 ~0x27ffffff. Actually, only one address 0x20000000(DMA source address) is used. The image sensor is provided by the clockoutO pin of S3C2410A. In addition, PCLK, HREF, VSYNC are respectively the pixels, line and frame synchronization signals. OE is the start collecting signal. XINT is external interrupt contact signal of ARM9and CPLD. XDREQ and XDACK are the request and handshake signal of DMA of S3C2410A. 4. System Driver Software Design 4.1. I2C Serial Bus Communication Protocol I2C bus is common network interface in embedded systems, it consists of SCL (serial clock) and SDA (serial data) two buses. The bus has strict timing requirements. When the bus works, clock pulse is sent by a serial clock line SCL and data are transmitted by serial data line SDA. The bus must be controlled by the master device, and the master device produces the transmission direction for serial clock to control bus, also start and stop conditions are generated. I2C bus time series for transferring a byte is shown in Fig. 5. When the master device writes slave units, data transmitted should follow the address of the slave units. But they can't take the initiative to perform data transmission. So when the master device reads slave units, a read request with the slave unit address must be sent. ARM9 is the master device in this system, and image sensor is the slave unit. ARM9 rewrites the value of KAC9638 register through I2C bus to complete the initialization. According to the communication protocol, the process to change a register value is: first send the CMOS sensor (seven addresses + 0) specific address, then send the address of the register which needs to be written and send the data. While the process to read a register value is: first send the CMOS sensor specific write address, then send the address of the register which needs to be read, then send the CMOS sensor (seven address + 1) specific read address and finally receive data. 4.2. Image Acquisition CPLD Sequential Control CPLD collects two 10-bit Image meta data and combines them into a 32-bit (if not enough use 0) and can improve the efficiency of image acquisition. And saving the image data by DMA in the design can reduce CPU overhead. Logic structure diagram with CPLD as the core design of image acquisition device is shown in Fig. 6. Its working principle is as follows: 1) CLKOUTO is the output clock pin of S3C2410A, according to the different 4 ~ 6 bits settings of the internal memory MisCCR in S3C2410A, different clocks can be output, such as the system clock FCLK, AHB (internal) bus clock HCLK and APB bus clock PCLK (external), etc. System inputs MCLK as a master clock of CMOS image sensor. 2) Starting signal START as an I/O of S3CM10A and it is an active high, since the start signal 0E is an active low of KAC9638, so a gate is connected between them. 3) VSYNC is a frame synchronization signal. The signal outputs an effective image with the signal as high level. Use the signal to connect external interrupt XINT of S3C2410A and the CPU can control the start and end of a frame image. 4) D [9:0] is the image sensor output data stream. D[15:10] filled with 0, Composed of 16 bits data input into the buffer. Within the buffer, two cycles shift into a 32-bit data. 5) The buffer is written into clock WRCLK, sensor pixel outputs clock PCLK. Write a 10 bit A/D value to a cycle. So when output 32-bit data at a time, the output clock cycle is a binary frequency of PCLK[6]. And the binary frequency is set as request signal XDREQ of DMA of S3C2410A. 6) Chip Selection signal line nGCS4: As a 32 bit buffer control line, low level is effective. In the data collection, Set the DMA source address within scope of BANK4. Select the buffer when the read operation is performed. 4.3. DMA Program Design for Collecting Image Data Flow chart of system collecting a frame of image data is shown in Fig. 8. 1) Start the image acquisition. Master CPU receives the acquisition order from a serial port or other communication ports, notifies the CPLD acquisition signal by START signal line. CPLD through hardware pin OE starts KAC9638 to collect the image data. 2) DMA initialization. It includes the initialization of DMA source address, destination address, DMA frequency, etc. 3) The interrupt. When a frame data acquisition is completed, inform the CPU via the external interrupt. 4) DMA data transmission does not need CPU intervention, the interrupt is generated after completion of a frame image. 5) Only when the DMA interrupt and external interrupt are generated, a frame image acquisition is thus completed. And then it goes to a master CPU processing. If only one of the interrupt is generated and timeout waiting for another interrupt, the result is the missing of data and collecting image failure. 5. Hardware Debug 5.1. Hardware Debugging Environment In the system hardware debugging, it applies the integrated development environment with JATG simulator for debugging, which is a most adopted debugging method currently [4]. Integrated development environment chooses ADS 1.2 of ARM company. JTAG simulator is also known as JTAG debugger, which is a device for debugging through the JATG boundary scan mouth of ARM chip. It belongs to complete non plug-in (i.e., do not use the on-chip resources) debugging. 5.2. Hardware Debugging Steps and Results 1) BootLoad system pilot testing. First use the Flash programming tool developed by Embest company to program Flash of the BootLoad. If log-on message returned by serial port of target board is correctly received in PC super terminal, it represents the normal operation of system. ADS downloader can be applied for debugging of SDRAM[7]. 2) Image sensor test. a) I2C configuration testing. Download the bin documents compiled by ADS to SDRAM of the target board, if the super terminal receives the correct ID of returned image sensor, I2C communication is normal. Configurate the size of image as 3(H)x5(V). Use the oscilloscope to measure the relationship between frame synchronization, line synchronization and the pixel clock. As shown in Fig. 9, 11 pixel clocks (set 3 pixels each line plus 8 all black pixels at the beginning) corresponds to a line synchronous clock. Fig. 10 shows the relationship between the line synchronization signal and frame synchronization signal. Five line synchronous clocks correspond to a frame synchronous clock. Test results show that S3C2410A can correctly configurate the working mode of the image sensor. b) Image acquisition test. For the convenience of test of the image obtained, send the image data collected by ARM through UART interface to PC terminal and then combine the data into image display. The received data is shown in Fig. 11. This system applies the combination of CMOS image sensor, DMA of CPLD and ARM9 to complete image acquisition. The method improves the image acquisition speed and reduces the CPU overhead. CMOS image sensor has the advantages of moderate price, simple peripheral, and integrated I2C interface facilitates the programming control. CPLD shifts the CMOS sensor output data into 32 bits data, which can make the sensor output at a higher speed. DMA of ARM9 is responsible for image acquisition and allows the CPU to handle other tasks. Experimental results show that the hardware platform design of the image acquisition system is reasonable and feasible. In practice, the system can be used in the video image monitoring, image automatic detection, medical and military detection and so on, it has a good application prospect. References [1] . L. Zhi, Research on CMOS image sensor application in star sensors, PhD Thesis, Chinese Academy of Sciences, 2004. [2] . S. Min, H. Xinkai, Z. Yarn, CCD and CMOS image sensor detecting performance comparison, Semiconductor Optoelectronics, Vol. 25, 2004, pp.1-4. [3] . L. Wensen, L. Zhongshen, H. Jian, Image acquisition system design based on ARM and CMOS, Electronic Testing, Vol. 5,2008, pp. 12-16. [4] . L. Jijun, D. Yungang, Z. Lihua, On research progress of CMOS image sensor, Laser & Optoelectronics Progress, Vol. 04,2009, pp. 45-52. [5] . C. Rongting, P. Meijia, Base and application of CCD/CMOS image sensor, Beijing Science Press, 2006. [6] . D. Chunlei, ARM system structure and programme, Tsinghua University Press, Beijing, 2003. [7] . L. Jie, Z. Tiancheng, Wireless image sensor network review based on CMOS technology, Microcomputer Information, Vol. 28,2012, pp. 133-135. Niqin JING, Yu Wang Beijing Vocational College of Electronic Science, 100029, China Tel: 13466616192 E-mail: [email protected] Received: 29 October 2013 /Accepted: 22 November 2013 /Published: 30 December 2013 (c) 2013 International Frequency Sensor Association |