TMCnet News
Studies on the Driver Violations via Collaborative Perception [Sensors & Transducers (Canada)](Sensors & Transducers (Canada) Via Acquire Media NewsEdge) Abstract: Aiming at the general phenomena of drivers' illegal behaviors in urban road traffic, this paper adopts the advanced computer technology for collaborative perception and monitoring of drivers' major illegal behaviors in the process of driving to lower the incidence rates of traffic accidents. It has presented a method based on embedded multi-source information fusion technology to deal with driver's main illegal behaviors. It mainly uses the speed sensors, alcohol sensors and CCD camera to acquire the data of drivers' major illegal behaviors such as speeding, drunk driving and fatigue driving. Then it uses the GPS to locate the vehicle, warning and disposing driver's illegal behaviors real-timely. And it will also send the driver's information to the administration of traffic information database by the means of 3G network. Finally, a simulation experiment has been presented to prove the correctness and effectiveness of the method on the development board with the kernel of Cortex-A8. Copyright © 2014 IFSA Publishing, S. L. Keywords: Cortex-A8, Drivers, Sensors, Embedded, Collaborative perception. (ProQuest: ... denotes formulae omitted.) 1. Introduction According to the analysis of years motor vehicle traffic, about 80 % of the total number of accidents caused by driver's illegal driving behaviors. While, the major forms of illegal driving behaviors are speeding, drunk driving, fatigue driving, driving without license and running the red light [1]. In the view of that this phenomena happen regularly, in addition to strengthen penalties and compulsory learning to eliminate the hidden peril of traffic accident, the sensor technology can be adopted to collaboratively perceive and monitor drivers' major illegal driving behaviors. Then, the traffic administration will punish for the driver4s illegal driving behaviors later. In accordance with the statistics of extraordinary serious traffic accidents, fatigue driving occupies a significant proportion. Adopting the "domain" wireless sensor put on the driver, we can effectively monitor the driver's physiology states [2]. And an embedded driver fatigue monitoring system which uses the cascaded classifier of the Haar feature to detect the face region from the image has been designed to judge the state of fatigue [3]. Otherwise, owing to the low sensitivity of the traditional drunk driving detection system, there is an embedded drunk driving intelligent identification system based on the internet of things that can detect the alcohol concentration of the driver and other passengers respectively [4]. In recent years, the traffic accidents accounts for speeding in highway occur frequently, and the toll rises year by year. Therefore, many scholars at home and abroad have researched and explored the problem for a long time. An automatic detection method of the network intelligent transportation network has been designed based on the road traffic incidents in Internet of Things [5]. The paper has presented an optimizing processed method which uses all kinds of sensor devices, including camera, alcohol sensor, Hall sensor and GPS, to collaborative perceive the driver4s illegal driving behaviors along with the information fiision technology. 2. The Structure of Perception Module The collaborative process module framework about the driver's illegal behaviors including speeding, fatigue driving and drunk driving is shown in Fig. 1. In the Fig. 1, the system consists of the CortexA8 microprocessor, alcohol sensor module, Hall sensor module, CCD module, LED module, 3G module, voice alarming module and GPS module. The alcohol sensor is used for acquiring the alcohol concentration of the driver's breath gas. The Hall sensor is used to collect the information about changes in the velocity of the car. And CCD camera is able to get the information of the driver's face. By the image processing, we can analysis whether the driver's eyeball has a tendency to become smaller or even closed, and also counts the frequency of driver's wink. At last, through the fusion of the information by the Cortex-A8 microprocessor, the system can judge whether the driver has an illegal behavior. If it happened, automatic alarm device would be triggered, and the driver's information and his location would be sent to the transportation station by the 3 G network. 3. The Key Technology 3.1. Determination of the Fatigue Driving The acquisition of driver's fatigue driving information mainly includes image capturing, image pre-processing, face location, eyes location, extracting the value of eyes characteristic parameter, analyzing sate of eyes closed, etc. First, the driver's driving image is acquired. And the sharpness of the image will directly affect the accuracy of the arithmetic. By the means of image preprocessing, the standard image format can easily distinguish the characteristics between face and eyes. The most important step is face detection, because involving in the later real-time debugging of the embedded system, the choosing of the algorithmic should comprehensively consider from the aspects of precision, real-time and the portability for porting. In the end, comparing the value of the eyes' characteristic parameter, the fatigue driving can be judged by the PERCLOS standard and the warning information can be sent out. After the video image has been acquired by the CCD camera, every frame image of the video should be analyzed. While every frame image is colorized and has carried lots of useless information, so it is difficult to deal with. Therefore, the face image should be preprocessed. It mainly includes graying, homomorphism filtering, threshold adjusting, image binaryzation, and morphology filtering. Graying is a process to transform the color image containing brightness and color into the grayscale image. And it is an important step in most pictures processing, because its result is the basis for the subsequent processing. Since the rule to determine whether the driver is in the fatigue state is the times of diver's blinks and the degree of eyes' opening, the color image can be transformed into grayscale image of 256 levels that can simplify the computation of image processing. It is known that color images are composed of red, green, blue the three primary colors. And due to the different sensitivity to the colors of human's eyes, the weight of the three primary colors is different when transforming color image into gray. The specific conversion formula is represented as follows: ... (1) Homomorphic filtering is designed to eliminate the influence of uneven illumination and enhance image details. By the means of Homomorphic filter built in frequency domain, the reduction of the image quality caused by the lack of light can be avoided. And it is easier to enhance the scene we are interested in. Thus, it will not only can keep the original of the image but also enhance the details in a large part. Seeing from the result of image processing, Butterworth high pass filter highlights the changes in details. But the grayscale of whole image has decreased, which makes the dark face image less clear. Having had Homomorphic filtering on the same image again, not only the dynamic range of the image has been compressed, but also the dark face details have been enhanced especially the details of dark face. The Homomorphic filtering algorithmic is represented as follows: ... (2) In this equation, the x and y respectively represent the abscissa and ordinate; T(x, y) is the reflectivity of the point (s, y); S(x, y) is the value before the filtering; I(x, y) is the illumination intensity of the point whose value is gotten through the convolution between the original image and a large low pass filter: ... (3) In the equation, the 2N+1 are used to describe the order of low pass filter. Because this design is for the image of 320x240, the N = 10. At the beginning of the driver's eyes localization, the external contours of the driver eyes can be seen as composed of two period of circular arc form, and eyes can be seen as inscribed and incomplete round with black pieces. The geometric features are shown in Fig. 2. According to the university and symmetry of facial features, the position of the human eyes could only commonly appear in a fixed area. Thus, setting a relative position on the face region, eyes area can be intercepted probably. Once this part of the information is obtained, the human eyes will be extracted further. The size of the defined area is 37x33. For the upper face image region, the relative starting coordinate for cutting out is (25, 13). The result of initial eyes localization is shown as Fig. 2. Just as the Fig. 2 shown, using relative coordinate for the human's eyes localization can only find out eyes' region initially. And we need to get rid of the interface information just as the eyebrows or the part of orbital edge miscellaneous points, so that we can only extract the needed eyeballs. There the image region growing method will be used to extract the eyeball, which includes the localization of eye's region growing point and the region growing. The localization of eyes' region growing is to extract eyeball completely, so finding a reasonable regional growing point is critical. We will use the integral projection both on the abscissa and ordinate, and finding out the values of abscissa and ordinate which has the biggest integral values in the X axis and Y axis. And the point combined by the two values is the needed regional growing point. First we need calculate the integral projection of the eye's image white pixels on both the vertical and horizontal direction. Second, a peak value will appear on the X axis in the vertical direction of integral projection. We just see the abscissa of the point as the abscissa of the regional growing point. And the integral projection in the Y axis of the horizontal direction will be double peaks, so we select the first peak's ordinate from down to up as ordinate of the regional growing point. Thus, the regional growing point has been confirmed. The benefit of this way is that we can confirm whether the regional growing point is in the eyeball basically. If not, we cannot extract the eyeball. As for the judgment of eyes state, having had Homomorphic filtering and binaryzation processing, the opened eyes image had a great difference with the closed eyes on the maximum in the integral of vertical direction. There are at least 7 white dots in a certain vertical direction for the normal opened eyes, while there are only no more than 3 for the closed. We can steadily distinguish both of the states according to the difference between those integrals. Therefore, there are two methods to judge the fatigue state: the one is counting the times of blink within 20 seconds; the other is counting opening and closing states in the pre-set number of frames. In the first way, if the numbers of blink have been more than 8 within 20 seconds, the system will see it as a signs of fatigue, and a warning would be sent out. In the second method, if the state of driver's eyes is still closing in the pre-set number of frames, the warning will also be triggered. When the counts of fatigue warning have been more than 8 times, the driver will be considered in a depth tired state, and the system will perform an automatic deceleration control. 3.2. Determination of Drinking Driving We use the MQ-3 type gas sensor to detect the concentration of the alcohol gas, as shown in Fig. 5 [6]: The MQ-3 gas sensor belongs to a kind of MQ series gas sensitive element, its detection ranges from 100 ppm to 2000 ppm. And it has plenty of advantages such as high sensitivity, the output signal is level V, fast response speed, the power consumption is less than 0.75 W and so on. The sensitive part of MQ-3 gas sensor is made up of N-type semiconductor microcrystal sintering layer of metallic oxide (stannic oxide). When its surface is absorbed by gas alcohol molecules, the proportion of the conduction electrons in the surface will change, thus the resistance on the surface will change with the measured gas concentration. Because this change is reversible, we can repeatedly use it. The on-board alcohol detection modules are mainly composed of center control module, alcohol detection module, storage module, engine control module and the signal transmission circuit module [7]. Using the 8051 single chip as control chip, the center control module is the core of the on-board alcohol detection terminal, which is used to control other module's work. The master control module can control the alcohol detection module to detect the alcohol concentration of the driver's breath gas regularly. When the alcohol detected concentration is more than the standard threshold for drunk driving, the engine will be prohibited if it is stopped, or the driver will be warned if the engine is running. If they do not listen to discourages, the car will slow down gradually and automatically, and the engine will also be shut off automatically within a set-time. At same time, the system will start 3G mobile wireless communication network, and send the information which includes the location of the car, the driver's information, the alcohol concentration information to the traffic police basal detection station in a form of text messages by the 3G module. The alcohol detection module based on the electronic heating system which considers alcohol sensor as the core is made up of multiple sensor array. After the alcohol detection module preheating, it will begin to work and detect the alcohol concentration periodically in the car at a set-time. After the analog signal output of the alcohol sensor being operated and amplified, it will be sent to the single-chip. After the data have been dealt with, the alcohol concentration data would be displayed on the LCD and a voice alarm device would be triggered soon. And in the storage module, the E2PR0M chip is used to store the result of alcohol detection, the vehicle's license number and the owner information. The signal emission includes GPS and 3G mobile communication module. And the GPS is used to locate their position which includes latitude, longitude and time information through the antennas. The 3G module connects the central control through the serial and it through the AT commands to exchange information with central module. And then, it sends the GPS positioning information, car's owner information and the detection alcohol concentration to the basic traffic police detection station or monitoring central database by the mobile communication network in short message. The engine control circuit and the switch are in series in the circuit. And the two-way light switch is in a parallel in the double light and alarm circuit. When the alcohol concentration is out of the limit, the switch of engine will be cut off and drive the double flash warning the other cars from the driving car. If the traffic police station receives the information, they will upload the information to the database management system. And according to the longitude and latitude, they can locate the peccant vehicle and make some corresponding processing. The traffic police station may through analyzing the driving statistical information to achieve the comprehensive and scientific management such as strengthening the management of drunken driving black spots. 3.3. Determination of Speeding The Principe of the Hall sensor measuring the speed is that the sensor will produce a switch signal when its surface close to the magnetic field. So we install the sensor in a fixed position of the automotive axle, and install a small magnet relatively on the rotating parts. Whenever there is one revolution of the rotating parts, the sensor will produce a switch signal, so we can get the speed with a switch signal detector. The over-speed alarm module is mainly made up of 8051 single-chip microcomputer control module and sensor module. And the 8051 mainly controls the peripheral hardware and achieves some calculation function, while the sensor is used to signal sampling [8]. The Hall sensor is type of SC-3019, and this device has high reliability, strong anti-jamming capability, wide adaptation to the power and good compatibility. And its internal mainly consists of the voltage regulating circuit, hall potential element, differential amplifier, Schmitt trigger and drive circuit. And its output is pulse signal which can be input microcontroller directly without shaping circuit [9]. The speed monitoring circuit consists of hall sensor and a small piece of permanent magnet fixed on the transmission shaft of automobile gearbox. When the car starts, the magnet will also rotate with it. And the produced magnetic field acts on the hall sensor, which will produce the counting pulse signal one by one. Then this signal will be sent to the single-chip microcomputer after voltage regulation. The microcontroller counts by the timer and counter, and gets the shaft revolutions after conversion. At last, the speed after processing will be sent to the display circuit [10]. 4. Realization of Collaborative Perception 4.1. Determination of hardware environment 1) ARM Cortex-A8 Processor. The research based on embedded multi-sensor fusion technology is to deal with the speeding, drunk driving and fatigue driving in the process of the driving, and can also have the alarming and displaying function. Cortex-A8 is the first super scalar processor designed by ARM. The processor contains new technology which was used to increase code effectiveness and performance. Cortex-A contains one technology named NEON directed for multimedia and signal processing. Cortex-A8's speed can reach 600 MHz to 1 GHz which makes it the best processor of all time. Cortex-A8 could meet every mobile device's demand, and its power dissipation is 300 mW or lower while performance could reach up to 2000 MIPS. 2) REAL210 Development Board. REAL210 development board is an ARM board and designed by Guangzhou RunEr Corporation, its processor is Samsung's S5PV210 whose core is ARM Cortex-A8 and instruction set is ARMV7. E210COREV3 could support such interface as CMOS (CCD), WIF, Bluetooth, GPS, 3G, and CAN. The platform is one REAL210 board based on CORTEX-A8, the hardware is composed of three modules which are data acquisition module, data analysis, and acquisition module and system decision module. Data acquisition module need acquire three routes of signal. The status of driver's eyes is transferred to core processor through video picture processor and A/D after gathered by CCD/CMOS .The speed is gathered by Hall sensor and transferred to core processor by CAN-bus. The alcohol concentration of drivers is gathered by alcohol sensor and transferred to core processor by USB. Data analysis and acquisition module chooses ARM which apply 5-assembly-lines and separated instruct/data unit to increase CPU's speed. The core processor analyzes eyes' status to pick up information of fatigue driving, and analyzes alcohol concentration to pick up information of drunk driving, and analyzes the speed to pick up information of over speed. With the help of information fusion technique, system decision module could judge drivers' illegal behavior by totally considering drivers' status and speed then make correct measures. If the driver is slightly fatigue driving, the platform will alarm, if the driver is fatigue driving seriously, the platform will light and warn the driver and limit the speed at the same time. The other two kind of illegal behaviors will also get corresponding measures. 4.2. Linux Development Environment Cross-compiler environment is essential to embedded system which doesn't have the ability for software development, so that we have to develop embedded system software on host machine then debug the software by copying it to embedded platform. The operation system of embedded platform and host machine is Ubuntu-10.01.1. We use serial Portsmouth and network to connect Linux host machine and development board. There are three steps to build cross-compiler environment that are configuring QT integrated development environment, configuring arm-linux-gcc and configuring QT for Embedded Linux. The Linux operation system is transplanted to development board after configuring compilation environment correctly. The embedded Linux operation system is composed of Linux core, Boot-loader, hardware device/root file system and application. 4.3. Simulation of Driver's Illegal Behaviors 1) Drunk driving. Begin the driving, we set up the system model first, and then doing a choice according to the button. If the button is pressed, we will detect the alcohol concentration of the driver. If the result is out of the limit, the engine will be limited to start. If the alcohol concentration is detected out of the limit during the driving, the system will start the 3G and GPS, and the information will be sent to the traffic police station. In the alcohol testing experiment, 20 people have drunken beers and wine of different concentration. And we measured the alcohol concentration of their breath gas every 10 minutes after drinking. Each one was tested 50 times, and the success rates can be 90 % to 95 %. The result is shown in Fig. 4 below. When the alcohol concentration of the breath has over the limit 200 mg/100 ml, the car will be forbidden and warn drunk driving. 2) Fatigue driving. We had invited 20 people to imitate the nap of drives. And the Fig. 6 shows 3 test results of it. The curve of article 1 shows that the frequency of continuous blinks is higher than the normal; the second curve shows that the times of blinks are more than the normal in a certain period time; the last curve shows that there is a longtime close of eyes. In the experiment, when any of the three kinds of circumstance had happened, the system would give a voice alarming. And the accuracy of the experiment result can reach about 90 %. 3) Speeding. There, we install the Hall sensors on each bearing of the wheels. And then, according to the calculation and transformation, we can get the speed of the vehicle timely and compare the speed with the result shown on the screen. It is shown in Fig. 6. 4) Comprehensive simulation on speeding, drunk driving and fatigue driving. Speeding, drunk driving and fatigue driving is the major illegal behaviors in the road traffic. Based on Hidden Markov Model and the classification judgment of data collected by the sensors real-time, multiple early warning mechanism of parameters constrain can be put forward [11]. Fist testing the alcohol density in the vehicle, if the alcohol detected concentration is more than the standard threshold for drunk driving, the engine will be prohibited if it is stopped or the driver will be warned if the engine is running. If they do not listen to discourages, the car will slow down gradually and automatically, and the engine will also be shut off automatically within a set-time. In the process of the car, the system can judge whether the drive is speeding or fatigue driving based on the transition probability of Hidden Markov Model and warn the driver according to different situations. The results of simulation are shown in Fig. 7, and the alarming success rates can be more than 90 % [12]. 5) The analysis of experiment results. We have tested the three major illegal behaviors in the Cortex-A8 embedded development board, including hardware circuit test and environment simulation test. According to the results of the experiment, we can find the designer of the system can reach a great high accuracy. So this system is worth to be promoted and used widely. 5. Conclusions The paper based on the driver's three illegal behaviors in the city roads designs a cooperative perceive platform and have been verified on the development board. The alcohol detection module detect the alcohol concentration of driver's breath gas real-timely; the CCD camera is used to confirm the fatigue state; the speed sensor can confirm the situation of speeding; GPS module can receive the positioning information and calculate the longitude and latitude of the car. If there are illegal behaviors, it will warn and start 3 G network automatically and send the information to the traffic police station. Our next step is ready to experiment in a real environment and realize productization if it is proved to be feasible. Acknowledgements The work is supported by NSF of Hubei Province (2012FFB05006) a National Undergraduate Training Programs for Innovation and Entrepreneurship 20131049710001). References [1]. Deng Mingyang, Fu Rui, Chen Guangjiao, Collection and analysis about background factors of road traffic accidents index, Journal of Chongqing Jiaotong University (Natural Science), Vol. 31, No. 4, 2012, pp. 852-855. [2]. Wu Yizhi, Ding Yong-Sheng, Xu Hongan, Bao XuLi, A design and Qos study of body area medical wireless sensor networks, Computer Engineering and Applications, Vol. 43, Issue 14, 2007, pp. 124-126. [3]. Liu Danghui, Sun Peng, Yin Yunxia, Design and implementation of an embedded system for driver fatigue surveillance, Measurement & Control Technology, Vol. 29, Issue 5,2010, pp. 27-31. [4] Li Xiaojing, Liu Shulun, The embedded drunk driving intelligent identification system based on the internet of things, Manufacturing Automation, Vol. 34, Issue 11,2012, pp. 33-36. [5] Wang Hao, Tan Guozhen, Yang Jixiang, Research on urban traffic emergency detection based on IoT, Computer Science, Vol. 39, Issue 2, 2012, pp. 262-267. [6] Li Fupeng, Design and research of intelligent vehicle-mounted alcohol detection control system, Master Thesis, Zhejiang Normal University, 2009. [7] Sun Dan, Design of in-vehicle preventing alcoholimpaired driving system based on GPS/GSM, Master Thesis, Dalian University of Technology, 2013. [8] Cao Yang, System of the monitoring of lager trucks speeding and overloading, Master Thesis, University of Electronic Science and Technology, 2010. [9]. Feng Chungui, Design of vehicle speeding automatic monitoring system based on RFIDF, Southwest University, 2013 [10]. Wang Hai, Wang Xiaonei, Design of over speed warning system of vehicle, Machine Building & Automation, Vol. 39, Issue 4,2010, pp. 172-175. [11]. Zhang Nan, Ma Fuheng, Zhang Shuai, Sun Xiaoran, Chu Dongdong, Research on gray Markov chain forecasting method of Dam behavior, Water Resources and Power, Vol. 30, Issue 8, 2012, pp. 70-73. [12]. Yu Weiwei, Wu Qing, Applying HMM-based driver fatigue recognition in smart vehicle space, Computer Applications and Software, Vol. 28, Issue 10, 2011, pp. 43-46. Yuan Chao, Xu Yi, Cao Junhui, Zhu Yaguang, Mao Zhenhua, Liu Shu jin, Mao Chuyang Department of Computer Science and Technology, Wuhan University of Technology, 430063, China Tel.: 13871297761 E-mail: [email protected] Received: 25 March 2014 /Accepted: 30 June 2014 /Published: 31 July 2014 (c) 2014 IFSA Publishing, S.L. |