Moving target detection has always been a significant interest in temporal image analysis field since it rose up to the present. It has widespread application prospect in dealing the emergency event, detecting moving object in big square, and rescuing in natural disaster. Based on the common single lens camera imaging, for the restraint of the photographable screen and the expression of plane image, we have to change the actual spatial information from three-dimension to two-dimension, and meanwhile we need the excellent handling ability to protect image high frequency detail. When we detect moving target on computer, we have to transform the two-dimensional information to the three dimensional attribute to reappear the actual target. Such “3-2-3” dimensional conversion has large computation, low real time and bad precision, so it hinders the development of moving target detection. Insect ‘s compound eye system has the unique superiority in detecting moving target. It can detect moving goal with little information by distinguishing outline characteristic of object. And it does not have the visual dimension transformation question.
The traditional visible light and infrared target detection systems mostly use the existing imaging sensors (for example camera and so on) to gain the image, and then use computer signal processing method to achieve recognition and localization for target. The precision of system has a positive correlation with the computation payload. The precision is higher and the need of the computation load is larger. However, the existing processor cannot completely satisfy the demand of the system for the time request. Therefore, many researchers want to develop new methods for high temporal resolution remote sensing imaging while they continue to research the original processing mode. Moving target detection, with less computation, is an important development direction in future. In this aspect, some insects, just like dragonfly and honeybee with compound eye, have provided good example for us.
Because insect’s compound eye has the characteristics of small volume, light weight and large viewing field, it has the unique superiority in the moving object detection and track. According to the research, the honeybee’s reaction time to the object which appears suddenly only needs 0.01s, but the human eye actually needs 0.05s. Dynamic target detection ability of dragonfly reaches 100%, and capture ability reaches 99%. It is definite to establish one kind of new movement object detection system by simulating this kind of characteristic of insect. In this aspect, many foreign researchers have done a lot of work, and have studied many outstanding algorithms and system. American Martin Marietta Company and the Texas instrument company developed the sensor automatic target recognition module, also developed the miniaturized sensor module which is used for performance flight. American ID Image Company, Missouri/ Columbia University have united to study the staring algorithm (GATOR) system which has been used to face the track recognition. Britain and Norway have united to develop the image formation infrared sensor and the programmable phantom processor. In 1991, B.Moslehi, W.Parkyn et al. of the American Ghali Furry Asia Physical optics Company invented one kind bionic compound eye vision system.
Some research has already been done in this domain in China. Li Wenyuan of Tianjin University used the computer to simulate the image effect that real image pass through the compound eye optical system. Lu Liming and Wang Guofeng et al. of the Northwestern Industrial University used two modulators to simulate the function of the compound eye localizing the target. They integrate the image formation detector to construct more than one mold to guide the missile homing head. This research has proven the feasibility and value application of the movement object detection using bionic compound eye. Tian Weijian and Yao Shengli et al. of Chinese Academy of Science Xi'an Optics Precision machinery Research institute proposed one kind multichannel imaging system which has been used in the moving target detection. Huang Zhuxin, Xu Guili et al. of Nanjing University of Aeronautics and Astronautics have designed one kind bionic compound eye measurement system which is composed of 7 cameras. This system has big field of view, but its volume is also huge. The combination mode has not jumped out of the ordinary single aperture camera image thought completely.
2. Research on moving target detection based on compound eye
The traditional video moving target detection technology mainly refers to detect moving goal with static camera photographs. When the camera is loaded on moving carrier, there will be relative movement between camera and moving goal. So the electronic image stabilization is need for accurate detection moving goal.
Target monitoring scope is limited with single aperture camera. Image is much smaller after the cutting out visible scope with electronic image stabilization, and it is not favor for the wide range detection. Compound eye has unique superiority in moving target detection. Based on compound eye image mechanism and method, a bionic system that mimics large field of view and spherical imaging characteristics of compound eye at structure level is proposed. In view of high temporal resolution remote sensing, our research include: moving target outline is rapidly set with changeable resolution detection mode; the geometrical structure between cameras is given out; method for panoramic mosaic and calculation of overlap ratio of adjacent cameras is given out, and the synchronization of different cameras is discussed.
The authors intend to research new method for moving target detection with imitating insect vision system. We do lots of research on bionic compound eye mechanism and sum up the basic principle of moving target detection based on insect compound eye. The principle is shown in Fig. 1.
Based on anatomy of insect compound eye and electrophysiological data, the similar experimental apparatus is researched on. By comparing insect compound eye with experimental data acquired from the apparatus, the mechanism of moving target detection is revealed in theory to establish the corresponding mathematical model. In final, the new algorithm of moving target detection is modeled to improve temporal resolution of remote sensing imaging. The framework of research is shown in Fig. 2.
Moving target detection by compound eye is imitated at the two levels of function and principle. A bionic moving target detection algorithm is proposed, which consists of two main stages. In the first stage, principle of coarse detection of moving targets by low resolution area that out of acute zone of compound eye is mimicked. A motion window construction method is proposed by merging minimal bounding rectangle of adjacent changing area to roughly point out positions of targets. In the second stage, principle of accurate target extraction in acute zone of compound eye is mimicked. A target segmentation method is proposed, which in the motion window uses the complementarities of edge detection and symmetrical difference to get a coherent outline at first and then calculates convex hull of the outline to extract accurate target. The algorithm decreased the probability of occurrence of more than one target in a single motion wind. It also makes the targets detected more integrated. As a result of coarse-to-fine detection, high temporal efficiency is achieved.
3. Bionic compound eye hardware equipment
We research bionic compound eye equipment in reference of dragonfly eye structure. The equipment comprises multi-channel ray system and each ray system is similar to an aperture of compound eye. The ray peak is small lens which correspond an independent photosensitive part. The photosensitive part equipped with cpu process the data and connect with the next advanced treatment equipment. Equipment processing mode simulate two levels of processing pattern of compound eye.After the aperture data is sampled and extracted by the first treatment unit, it is submitted to the second treatment unit. The flow is shown in Fig. 3.
We imitate dragonfly eye structure to set up bionic compound eye equipment. The equipment is used to test moving target detection algorithm, and its schematic drawing is shown in Fig. 4.
4. Electronic image stabilization of bionic compound eye equipment
The rocking of image will reduce detection precision when aiming the moving target. So the electronic image stabilization before detection is crucial for the following automatic detection. The main research of electronic image stabilization includes three sections: identify overall motion parameter of moving target; forecast active movement value; compensation and splicing of image sequence. The electronic image stabilization goal is to eliminate rocking noisy and make image clear and stable. The key step of electronic image stabilization is calculating movement value between images and its flow is shown in Fig. 5.
Behind electronic image stabilization, image mosaic of adjacent cameras is done. Its destination is to expand the view field of compound eye system. Image mosaic software researched by us can splice several images to a complete image, and the result is shown in Fig. 6.
5. Simulate compound eye visual function to construct moving target detection model
Small and medium-sized compound eyes of insects are not evenly distributed, and sensitive vision areas lie in the regions with dense compound eye. The authors research sensitive vision areas to find its effect for detecting moving target and to give qualitative analysis.
With restriction of the number of small eyes, spatial resolution is not too high on compound eye imaging.But this is advantagement that compound eye structure is favorable to detect moving target. In most cases, moving target detection only needs information of targets outline. Compound eye could quickly extract objects profile with little information.In this case, high resolution will not only reduce the efficiency of contour extraction, but also limit object in a small area and can not recognize the overall targets, resulting in the error detection. Therefore, it is an important part that quickly extracts moving object contour mechanism with low resolution imaging based on insect compound eye.
Compound eyes structure and bionics research have shown that there is a disparity that adjacent compound eyes test on the same point. According to the relative position of small eyes, they detect the same point time difference parameters which can be quickly calculated the value of changes in external optical flow and the moving targets location and speed. When compound eye movement with background fixed, compound eyes could also be detected insect own speed and relative position. Therefore the second part of this section is exploring insect to utilize compound eye which could detect changes in the external optical flow and locating moving target position, and to find the main impact factors on moving targets detection, such as velocity, distance and small eye visual muscle movement.
In the 1950s, Hassenstein and Reichardt proposed compound eye elementary motion detection model (HR model), shown in Fig. 7. The model describes process of inter-related operations on two adjacent photoreceptor cells (or small eyes),which the HPF represent high-pass filter, LPF on behalf of low-pass filter, π stand for multiplication, Σ represent addition and subtraction. The model has better explanation on compound eye which receives the primary optical signal, but the model is not still perfect now. In addition, HR model aim at part process. In the basis of these studies, on the one hand we will combine with the existing experimental data on the HR model to improve it further; on the other hand we will focus on simulation of the overall signal processing neurons, and analysis function on moving target recognization in sensitive areas. We also want to establish compound eye imaging mathematical model and give calculative method of parameters values in model.
Based on insect visual perceptive mechanism, we establish the corresponding mathematical model by simulating bionic compound eye experiments with experimental device. For common single aperture image, large amount of useful experimental data (such as three-dimensional information, speed information etc.) are almost completely lost in the imaging process with multi-dimensional conversion. During the phase of recognizing moving target, two-dimensional image need to be reverted multi-dimensional information. The process needs large volume of calculation which is difficult to ensure real-time. In order to make up lost information, we introduce additional knowledge which will create additional noise. An important part of research is to find how to image directly which utilize the target multi-dimensional information at the same time. In addition, subject to structural constraints, resolution of compound eye imaging is far lower than the human eye and the existing high-resolution imaging. However, low-resolution imaging is precisely conformed to its limited brain processing capacity of insect which could detect targets with small computation.
6. Design compound eye moving target detection algorithm
To design and simulate moving targets detection algorithm of compound eye imaging, we establish models with computer simulation software. By changing different settings and different parameters of the handler functions, we can get different simulation results. Compared simulation results with the actual measurement data of compound eyes, we improve algorithm further. We do some experiment to test the algorithm based on the completion of installation. We select two types of indoor and outdoor scenes to verify effectiveness of algorithm. The algorithm, combined indoor with outdoor scenes, will form a complete system for moving target detection. The flow of algorithm is shown in Fig. 8.
During the first phase of the algorithm, we simulate insect compound eye to detect moving target outline and extract targets in a rough area.In the second phase of algorithm, we simulate compound eye mechanism to accurately extract moving object.
Based on the above research, we developed electronic image stabilization and moving target detection software, shown in Fig. 9.
Generally speaking, the former research mostly focused in imitation of equipment structure, rather than in compound eye mechanism, which cause the research to be relatively slow. This project team has done some foundation work in insect morphology, pattern recognition and digital imaging domain. Based on former foundation, this project continues to do some work, such as simulating dragonfly compound eye imaging system, establishing elementary mathematical model developing compound eye equipment and introducing the electronic image stabilization. The work will obviously enhance system performance. With the breakthrough in this project, it will lead to any transformation in methods and means for the intelligence vision system.