Open access
1. Introduction
Moving target detection has always been a significant interest in temporal image analysis field since it rose up to the present. It has widespread application prospect in dealing the emergency event, detecting moving object in big square, and rescuing in natural disaster. Based on the common single lens camera imaging, for the restraint of the photographable screen and the expression of plane image, we have to change the actual spatial information from three-dimension to two-dimension, and meanwhile we need the excellent handling ability to protect image high frequency detail. When we detect moving target on computer, we have to transform the two-dimensional information to the three dimensional attribute to reappear the actual target. Such “3-2-3” dimensional conversion has large computation, low real time and bad precision, so it hinders the development of moving target detection. Insect ‘s compound eye system has the unique superiority in detecting moving target. It can detect moving goal with little information by distinguishing outline characteristic of object. And it does not have the visual dimension transformation question.
The traditional visible light and infrared target detection systems mostly use the existing imaging sensors (for example camera and so on) to gain the image, and then use computer signal processing method to achieve recognition and localization for target. The precision of system has a positive correlation with the computation payload. The precision is higher and the need of the computation load is larger. However, the existing processor cannot completely satisfy the demand of the system for the time request. Therefore, many researchers want to develop new methods for high temporal resolution remote sensing imaging while they continue to research the original processing mode. Moving target detection, with less computation, is an important development direction in future. In this aspect, some insects, just like dragonfly and honeybee with compound eye, have provided good example for us.
Because insect’s compound eye has the characteristics of small volume, light weight and large viewing field, it has the unique superiority in the moving object detection and track. According to the research, the honeybee’s reaction time to the object which appears suddenly only needs 0.01s, but the human eye actually needs 0.05s. Dynamic target detection ability of dragonfly reaches 100%, and capture ability reaches 99%. It is definite to establish one kind of new movement object detection system by simulating this kind of characteristic of insect. In this aspect, many foreign researchers have done a lot of work, and have studied many outstanding algorithms and system. American Martin Marietta Company and the Texas instrument company developed the sensor automatic target recognition module, also developed the miniaturized sensor module which is used for performance flight. American ID Image Company, Missouri/ Columbia University have united to study the staring algorithm (GATOR) system which has been used to face the track recognition. Britain and Norway have united to develop the image formation infrared sensor and the programmable phantom processor. In 1991, B.Moslehi, W.Parkyn et al. of the American Ghali Furry Asia Physical optics Company invented one kind bionic compound eye vision system.
Some research has already been done in this domain in China. Li Wenyuan of Tianjin University used the computer to simulate the image effect that real image pass through the compound eye optical system. Lu Liming and Wang Guofeng et al. of the Northwestern Industrial University used two modulators to simulate the function of the compound eye localizing the target. They integrate the image formation detector to construct more than one mold to guide the missile homing head. This research has proven the feasibility and value application of the movement object detection using bionic compound eye. Tian Weijian and Yao Shengli et al. of Chinese Academy of Science Xi'an Optics Precision machinery Research institute proposed one kind multichannel imaging system which has been used in the moving target detection. Huang Zhuxin, Xu Guili et al. of Nanjing University of Aeronautics and Astronautics have designed one kind bionic compound eye measurement system which is composed of 7 cameras. This system has big field of view, but its volume is also huge. The combination mode has not jumped out of the ordinary single aperture camera image thought completely.
2. Research on moving target detection based on compound eye
The traditional video moving target detection technology mainly refers to detect moving goal with static camera photographs. When the camera is loaded on moving carrier, there will be relative movement between camera and moving goal. So the electronic image stabilization is need for accurate detection moving goal.
Target monitoring scope is limited with single aperture camera. Image is much smaller after the cutting out visible scope with electronic image stabilization, and it is not favor for the wide range detection. Compound eye has unique superiority in moving target detection. Based on compound eye image mechanism and method, a bionic system that mimics large field of view and spherical imaging characteristics of compound eye at structure level is proposed. In view of high temporal resolution remote sensing, our research include: moving target outline is rapidly set with changeable resolution detection mode; the geometrical structure between cameras is given out; method for panoramic mosaic and calculation of overlap ratio of adjacent cameras is given out, and the synchronization of different cameras is discussed.
The authors intend to research new method for moving target detection with imitating insect vision system. We do lots of research on bionic compound eye mechanism and sum up the basic principle of moving target detection based on insect compound eye. The principle is shown in Fig. 1.
Figure 1.
Principle of bionic compound eye moving target detection.
Based on anatomy of insect compound eye and electrophysiological data, the similar experimental apparatus is researched on. By comparing insect compound eye with experimental data acquired from the apparatus, the mechanism of moving target detection is revealed in theory to establish the corresponding mathematical model. In final, the new algorithm of moving target detection is modeled to improve temporal resolution of remote sensing imaging. The framework of research is shown in Fig. 2.
Figure 2.
Framework of bionic compound eye moving target detection.
Moving target detection by compound eye is imitated at the two levels of function and principle. A bionic moving target detection algorithm is proposed, which consists of two main stages. In the first stage, principle of coarse detection of moving targets by low resolution area that out of acute zone of compound eye is mimicked. A motion window construction method is proposed by merging minimal bounding rectangle of adjacent changing area to roughly point out positions of targets. In the second stage, principle of accurate target extraction in acute zone of compound eye is mimicked. A target segmentation method is proposed, which in the motion window uses the complementarities of edge detection and symmetrical difference to get a coherent outline at first and then calculates convex hull of the outline to extract accurate target. The algorithm decreased the probability of occurrence of more than one target in a single motion wind. It also makes the targets detected more integrated. As a result of coarse-to-fine detection, high temporal efficiency is achieved.
3. Bionic compound eye hardware equipment
We research bionic compound eye equipment in reference of dragonfly eye structure. The equipment comprises multi-channel ray system and each ray system is similar to an aperture of compound eye. The ray peak is small lens which correspond an independent photosensitive part. The photosensitive part equipped with cpu process the data and connect with the next advanced treatment equipment. Equipment processing mode simulate two levels of processing pattern of compound eye.After the aperture data is sampled and extracted by the first treatment unit, it is submitted to the second treatment unit. The flow is shown in Fig. 3.
Figure 3.
Compound eye two levels of signal processing schematic drawing.
We imitate dragonfly eye structure to set up bionic compound eye equipment. The equipment is used to test moving target detection algorithm, and its schematic drawing is shown in Fig. 4.
Figure 4.
Compound eye equipment spatial structure.
4. Electronic image stabilization of bionic compound eye equipment
The rocking of image will reduce detection precision when aiming the moving target. So the electronic image stabilization before detection is crucial for the following automatic detection. The main research of electronic image stabilization includes three sections: identify overall motion parameter of moving target; forecast active movement value; compensation and splicing of image sequence. The electronic image stabilization goal is to eliminate rocking noisy and make image clear and stable. The key step of electronic image stabilization is calculating movement value between images and its flow is shown in Fig. 5.
Figure 5.
Flow of electronic image stabilization.
Behind electronic image stabilization, image mosaic of adjacent cameras is done. Its destination is to expand the view field of compound eye system. Image mosaic software researched by us can splice several images to a complete image, and the result is shown in Fig. 6.
5. Simulate compound eye visual function to construct moving target detection model
Small and medium-sized compound eyes of insects are not evenly distributed, and sensitive vision areas lie in the regions with dense compound eye. The authors research sensitive vision areas to find its effect for detecting moving target and to give qualitative analysis.
Figure 6.
Automatic image mosaic by three high resolution cameras.
With restriction of the number of small eyes, spatial resolution is not too high on compound eye imaging.But this is advantagement that compound eye structure is favorable to detect moving target. In most cases, moving target detection only needs information of targets outline. Compound eye could quickly extract objects profile with little information.In this case, high resolution will not only reduce the efficiency of contour extraction, but also limit object in a small area and can not recognize the overall targets, resulting in the error detection. Therefore, it is an important part that quickly extracts moving object contour mechanism with low resolution imaging based on insect compound eye.
Compound eyes structure and bionics research have shown that there is a disparity that adjacent compound eyes test on the same point. According to the relative position of small eyes, they detect the same point time difference parameters which can be quickly calculated the value of changes in external optical flow and the moving targets location and speed. When compound eye movement with background fixed, compound eyes could also be detected insect own speed and relative position. Therefore the second part of this section is exploring insect to utilize compound eye which could detect changes in the external optical flow and locating moving target position, and to find the main impact factors on moving targets detection, such as velocity, distance and small eye visual muscle movement.
In the 1950s, Hassenstein and Reichardt proposed compound eye elementary motion detection model (HR model), shown in Fig. 7. The model describes process of inter-related operations on two adjacent photoreceptor cells (or small eyes),which the HPF represent high-pass filter, LPF on behalf of low-pass filter, π stand for multiplication, Σ represent addition and subtraction. The model has better explanation on compound eye which receives the primary optical signal, but the model is not still perfect now. In addition, HR model aim at part process. In the basis of these studies, on the one hand we will combine with the existing experimental data on the HR model to improve it further; on the other hand we will focus on simulation of the overall signal processing neurons, and analysis function on moving target recognization in sensitive areas. We also want to establish compound eye imaging mathematical model and give calculative method of parameters values in model.
Figure 7.
Hassenstein-Reichardt elementary motion detection model.
Based on insect visual perceptive mechanism, we establish the corresponding mathematical model by simulating bionic compound eye experiments with experimental device. For common single aperture image, large amount of useful experimental data (such as three-dimensional information, speed information etc.) are almost completely lost in the imaging process with multi-dimensional conversion. During the phase of recognizing moving target, two-dimensional image need to be reverted multi-dimensional information. The process needs large volume of calculation which is difficult to ensure real-time. In order to make up lost information, we introduce additional knowledge which will create additional noise. An important part of research is to find how to image directly which utilize the target multi-dimensional information at the same time. In addition, subject to structural constraints, resolution of compound eye imaging is far lower than the human eye and the existing high-resolution imaging. However, low-resolution imaging is precisely conformed to its limited brain processing capacity of insect which could detect targets with small computation.
6. Design compound eye moving target detection algorithm
To design and simulate moving targets detection algorithm of compound eye imaging, we establish models with computer simulation software. By changing different settings and different parameters of the handler functions, we can get different simulation results. Compared simulation results with the actual measurement data of compound eyes, we improve algorithm further. We do some experiment to test the algorithm based on the completion of installation. We select two types of indoor and outdoor scenes to verify effectiveness of algorithm. The algorithm, combined indoor with outdoor scenes, will form a complete system for moving target detection. The flow of algorithm is shown in Fig. 8.
Figure 8.
Compound eye moving target detection algorithm flow.
During the first phase of the algorithm, we simulate insect compound eye to detect moving target outline and extract targets in a rough area.In the second phase of algorithm, we simulate compound eye mechanism to accurately extract moving object.
Based on the above research, we developed electronic image stabilization and moving target detection software, shown in Fig. 9.
7. Conclusion
Generally speaking, the former research mostly focused in imitation of equipment structure, rather than in compound eye mechanism, which cause the research to be relatively slow. This project team has done some foundation work in insect morphology, pattern recognition and digital imaging domain. Based on former foundation, this project continues to do some work, such as simulating dragonfly compound eye imaging system, establishing elementary mathematical model developing compound eye equipment and introducing the electronic image stabilization. The work will obviously enhance system performance. With the breakthrough in this project, it will lead to any transformation in methods and means for the intelligence vision system.
Figure 9.
Software interface of aerial image stabilization and moving target detection.
Acknowledgments
1. Project supported by the National Key Technologies R & D Program of China
Project No.: 2004BA104C
2. Project supported by: National Natural Science Foundation
References
- 1.
Gao P. Yan L. Zhao H. 2007 A new method for stabilization of video images with large moving object . EIMTV, 2007. - 2.
Gao P. Yan L. Zhao H. 2007 Aerial remote sensing automatic control system for UAV. ASSAIII, 2007. - 3.
Gao P. Yan L. Zhao H. Lu S. 2007 Study on shooting control algorithm of remote sensing control system for UAV . IGARSS2007. - 4.
Kumar R. Sawhney H. Samrasekera S. Hsu S. Tao H. Guo Y. Hanna K. Pope A. Wildes R. Hirvonen D. Hansen M. Burt P. 2001 Aerial video surveillance and exploitation[C]. Proceeding of IEEE, October 2001, 89(10): 1518-1539 - 5.
Wu J. Zhou G. 2006 Real-time UAV video processing for quick-response to natural disaster[C]. Proceedings of IEEE International Conference on Geoscience and Remote Sensing Symposium, July 31-Aug. 4, 2006, Denver, USA.2006 976 979 . - 6.
Srinivasan S. Latchman H. Shea J. 2004 Airborne traffic surveillance systems-video surveillance of highway traffic[C]. Proceedings of the ACM 2nd International Workshop on Video Surveillance & Sensor Networks, October 2004, New York, USA.2004 131 135 . - 7.
Shastry A. Schowengerdt R. 2002 Airborne video registration for visualization and parameter estimation of traffic flows[C]. Proceedings of Pecora 15/Land Satellite Information IV/ISPRS Commission I/FIEOS Conference, 2002. - 8.
Wei D. Zhou G. 2008 Real-time UAV ortho video generation[C]. Proceedings of IEEE International Conference on Geoscience and Remote Sensing Symposium, Boston, USA, July 2008, 5: 510-513. - 9.
Shastry A. C. Schowengerdt R. A. 2005 Airborne video registration and traffic-flow parameter estimation[J]. IEEE Transactions on Intelligent Transportation Systems, 2005, 6(4): 391- 405. - 10.
http://www.cs.cmu.edu/~vsam/ - 11.
Zhao H. Gao P. Yan L. Lu S. 2005 The real time image merge method for the remote sensing image acquired from the UAV . IGARSS2005. - 12.
Zhao H. Gou Z. Gao P. Cheng Y. 2007 No ground control point making the orthophoto for the UAV remote sensing system . ISPDI, 2007 - 13.
Yan L. Gao P. Zhao H. He K. Liu J. Yang S. Tong Q. 2006 Data collection of UAV remote sensing system in cloudy and rainy area. APCATS’2006. - 14.
Yan L. Ding J. Qin Q. Liu Y. Zhao H. Liu D. Zhao S. Gao P. Lian Z. 2007 A ground-based teaching and experimental simulation system for earth observing digital aerial remote sensing. ISITAE’07, Nov. 2007:388 390 . - 15.
Arambel P. Silver J. Krant J. Antone M. Strat T. 2004 Multiple-hypothesis tracking of multiple ground targets from aerial video with dynamic sensor control[C]. Signal processing, sensor fusion, and target recognition XIII, Orlando FL, 12-14 April 2004. 2004, 5429: 23-32. - 16.
http://www.sarnoff.com/sarnoff.com/press-room/news/2007/03/12/pyramid-vision-receives-videodetective-order-from-dhs - 17.
http://www.pyramidvision.com/products/terrasight - 18.
Chung Y. He Z. 2007 Low-complexity and reliable moving objects detection and tracking for aerial video surveillance with small UAVs[C]. Proceeding of IEEE International Symposium on Circuits and Systems, 2007.5,27 2670 2673 . - 19.
Kim J. Kim Y. 2008 Moving ground target tracking in dense obstacle areas using UAVs[C]. Proceeding of the 17th IFAC World Congress, Seoul, Korea, 2008.7. - 20.
Helble H. Cameron S. 2007 OATS: Oxford aerial tracking system[C]. Proceeding of IEEE International Conference on Robotics and Automation, 2007 - 21.
Cohen I. Medioni G. 1999 Detecting and tracking moving objects for video surveillance[C]. IEEE Proceeding of Computer Vision and Pattern Recognition, Fort Collins CO, June23 25 . - 22.
Cohen I. Medioni G. 1998 Detection and tracking of objects in airborne video imagery[C]. CVPR’98, 1998 - 23.
Rodriguez P. A. Geckle W. J. Barton J. D. Samsundar J. Gao T. Brown M. Z. Martin S. R. 2006 An emergency response UAV surveillance system[C]. Proceedings of AMIA 2006 Symposium, 2006: 1078 - 24.
Loveard T. Boxer P. http://www.auvsi.org/unmannedscience/newsletter/attachments/86/Loveard_T.pdf - 25.
Mirchandani P. Hickman M. Angel A. Chandnani D. - 26.
Collins R. T. Lipton A. J. Kanade T. Fujiyoshi H. Duggins D. Tsin Y. Tolliver D. Enomoto N. Hasegawa O. Burt P. Wixson L. 2000 A system for video surveillance and monitoring[R]. Technical report CMU-RI-TR-00-12, Robotics Institute, Carnegie Mellon University, May 2000 - 27.
Higgins C. M. Douglass J. K. Strausfeld N. J. 2004 The computational basis of an identified neuronal circuit for elementary motion detection in dipterous insects[J]. Visual Neuroscience, 2004, 21: 567-586. - 28.
Pant V. 2003 Modular Neuromorphic VLSI architecture for visual motion and target tracking[D]. The University of Arizona, USA, 2003