Performance comparison of difference fusion schemes Multi-focus image
1. Introduction
Image fusion can synthesize many images from different sensors into a picture which can meet specific application by using a mathematical model. It can effectively combine the advantages from different images and improve the analysis ability (Blum et al., 2005). In recent years, the image fusion in automatic target recognition, computer vision, remote sensing, robots, medical image processing and military fields has a very wide range of applications.
In many of the image fusion technology, image fusion based on multi-resolution analysis has become the focus of research and hotspot now. At present, the image fusion base on multi-resolution method can be divided into three kinds (Hu et al., 2008): the first kind is based on the pyramid decomposition (such as Laplace pyramid, ratio low-pass pyramid, contrast pyramid, gradient pyramid, etc.). The second type is based on wavelet decomposition, such as discrete wavelet transform (Li &Wu, 2003), wavelet and wavelet packet framework (Wang, 2004), multi-wavelet transform (Zhang et al., 2005), integer wavelet transform (Wang et al., 2008), FILWT (Li&Zhu, 2007), the dual tree complex wavelet transform (Yang et al., 2007), etc. The third type is new multiresolution methods, such as image fusion based on finite ridgelet transform (Liu et al., 2007), curvelet transform(Filippo et al., 2007) and contourlet transform (Li et al., 2008; Yang & Jiao, 2008).
In the multi-resolution fusion process, the choices of rules and operators is crucial and it will affects the quality of fusion image. However, the existing multiresolution image fusion research and experiment are basically only for two images and the model is only suitable for 2 image fusion. It cannot be generalized to many images (two above ) fusion. Although in the multiresolution image fusion methods, the existing simple fusion rules model can be extended to the fusion which has more than two image. Such as average accurate measurement, pixel absolute value choose big quasi measurement and based on region characteristics (such as regional energy, entropy, variance, average gradient, contrast and markov distance etc) choice of large prospective measurement, etc, but the result is very limited.
Therefore, this paper, a novel fusion algorithm of multiple images based on fast integer lifting wavelet transform is proposed. The algorithm may consider both of the result and speed of the fusion with tools of lifting wavelet. Also, according to the image promotion wavelet transformation different sub-bands characteristics, two kinds of new high and low frequency fusion strategy are proposed respectively. The result of experiment shows that this algorithm is not only suitable for rebalanced source image fusion, and can achieve good fusion result and faster fusion speed.
2. Integer lifting wavelet transform
Sweldens from Bell Labs proposed a method which does not depend on the Fourier transform of ascension wavelet construction in the mid 1990's, it not only inherited the time-frequency localization features of traditional wavelet but also has some other advantages (Li&Zhu, 2007; Lin, 2005)
2.1. The basic principle of integer lifting wavelet transform
In spatial domain, the realization process of lifting wavelet transform mainly divided into three steps:
Where the “
Where the “
After the finite layer of ascension, even sequences represent wavelet decomposition of low-frequency even after the ascending sequence represents wavelet decomposition low-frequency and odd sequence represents the high coefficients of low-frequency do the same operation can be get to the next level transformation. Ascension wavelet transform inverter change just put the above steps in turn.
2.2. Biorthogonal symmetric 9/7 wavelet decomposition of integer ascension
Any limited long wavelet filters can be decomposed by factor its corresponding mechanism of promoting realization. This paper used for image fusion of ascension wavelet transforms using image processing use most (9/7) filter. If the input signal recorded as
Where
For image promotion wavelet transform method usually adopts the ranks of image is hierarchically, first did them ascension decomposition of matrix transformation, again to results obtained by columns are ascending decomposition transform, after a layer decomposes available source image smooth characteristics respectively reflect the low-frequency sub image and reflect brightness mutation and details of the three characteristics in horizontal and vertical direction (in) and diagonal high-frequency sub image. Compared with the traditional based on convolution computation of wavelet transform compared, adopting lifting scheme can wavelet transform the calculation average halved, in two-dimensional image data processing, can reduce about 3/4 computation (Li&Zhu, 2007; Lin, 2005). Figure 1 (Top) shows a two dimensional example of visible light image for integer lifting wavelet decomposition with depth 2 and with depth 3.
3. Image fusion based on integer lifting wavelet transform
Assume
3.1 The fusion rules of the low frequency area
First, suppose the ABS of the Low-frequency coefficients of
Then defining a matching degree
Finaly, determine fusion operators. Defining a threshold
Where, the complete computational algorithm of weighting coefficient
If
Where,
By above knowable, while the ABS of the low-frequency coefficients have large differences,then choose the coefficient absolute value larger as the fusion of pixels; while the ABS of the low-frequency coefficients have rarely differences,adopt WA-Weighted Average determine fusion of low-frequency coefficients. So the low-frequency fusion rules may, according to the characteristics of the image itself dynamically select the weighted average method and the pixel of absolute value chosen 51, thus namely suitable for low frequence part complementary stronger image, and suitable for low frequence part more similar, complementary poor image.
3.2. The fusion rules of the high frequency area
First, we can compute respectively the local average value variance
Where, M、N indicate the local area respectively the rows and columns number (General took for odd);
By the reason of
Then, we can normalize the mean variance
Then, a matching degrees is defined as given below.
At last, a matching degree threshold
Where, the complete computational algorithm of weighting coefficient
Namely select with current processing pixels (
Where,
Above fusion rules indicate, while the high frequency component of source image have large difference in local mean variance in the corresponding decomposition layers and the corresponding on the direction, the corresponding decomposition layers and the corresponding on the direction of local mean variance large difference, that one of the source image contains rich details information, and other source image containing less detailed information, then use local mean variance choose big fusion rules. And in other hands, while the high frequency component of source image have less difference in local mean variance in the corresponding decomposition layers and the corresponding on the direction, it explained that the high coefficients contain rich details information, when using the weighted average fusion operators determine after fusion of wavelet coefficients. This can be clearly retention of image significantly signal details, can avoid again missing information, reduce the noise again at the same time ensure the consistency of the fused image.
3.3. The image fusion scheme
The fusion framework using integer lifting wavelet transform is shown in Fig.2. The approach to image fusion in ILWT (integer lifting wavelet transform) domain is as follows.
4. Experimental study and analysis
In order to verify the algorithm the effect and the performance, we focus more on three pieces of the image fusion simulation experiment. At the same time, also with the other two traditional wavelet images fusion algorithms of results were compared. In this algorithm, the source image on three-layer integer ascension wavelet decomposition, take a Form with
size of 3×3 in local region; the fusion when matching degree threshold of high frequency part and the low frequency part are separately took to be 0.75 and 0.65. In order to facilitate comparison, here the other two traditional wavelet image fusion algorithms are recorded as: wavelet fusion algorithm I and wavelet fusion algorithm II. Including, wavelet fusion algorithm of I high and low frequency component used respectively to local variance choose big norms and weighted average of low-frequency fusion rule, and the wavelet fusion algorithm of high and low frequency component II respectively by means of absolute value choose big norms and take an average of low-frequency fusion rules。In addition, the fusion algorithm of wavelet II and algorithm II, choose sym type 4 as multi-scale wavelet image decomposition and reconstruction tool, but wavelet decomposition layers and the local window size of the region and the algorithm are consistent.
Figure 3 gives more focused on three pictures of image fusion result. Figure 3 (a) and (b), (c) respectively, focusing on the left, middle and the right target for fusion source image, its image size are 512 x 512, and with precise registration. Figure 3 (a) from left of target clearer, figure 3 (b) closer to the middle goal on the relatively clear, figure 3 (c) on the right target clearer. Fusion purpose is to get a picture on the left, and right among both imaging of fused image is very clear. Figure 3 (d) and (e), (f) in order to take advantage of this paper fusion algorithm, wavelet and wavelet I fusion algorithm fusion algorithm II income of fused image. From the fusion results can see, three algorithms can get quite satisfactory visual effect, as far as possible to eliminate the source image focusing of the difference, raises the fused image of overall clarity. But by comparing it was evident that this algorithm can income of fused image is best effect, the fused images in all goals are the most clearly. In order to better compare three algorithm is superior, figure 2 shows three algorithm of fused image left, income from middle and right to local area of large PIC。Among them, the figure 4 (a) and (d), (g) respectively derived from the algorithm income of fused image left, and right among local regions; Figure 4 (b), (e), (h) respectively from fusion algorithm I income fused images, left, and right among the local area; Figure 4 (c), (f), (I) respectively from fusion algorithm respectively from II income fused images, left, and right among the local area. By figure 4 can see, this algorithm can income of fused image left, center and local area are very clear right and other details are well reserve. For example, figure 4 (a) the rings and the cord (dotted line box labeled part) was obviously reservation; Figure 4 (d) of saw tooth and background passage clearly visible; Figure 4 (g) the edge of the ball was the most obvious, the most clear, the details to double eliminate the most clean。Relative to character, the algorithm of wavelet fusion algorithm I income of fusion result commercial and wavelet fusion algorithm II income of fusion result worst, such as figure 4 (c) of the rings and the cord almost invisible, figure 4 (f) the zigzag and background line and figure 4 (I), the ball is not too clear.
Subjective visual evaluation can be given fusion result intuitive contrast, but are susceptible to personal experience and visual psychology factors, so need and objective evaluation standard combining comprehensive evaluation. Here we use the information entropy (IE), average gradient (AG) and SD as objective performance evaluation standard (Blum et al., 2005), is presented in table 3 the above image fusion result the objective performance evaluation results. From table 3 shows an objective evaluation index data can see: relative wavelet fusion algorithm, this algorithm can I and II obtained relatively good objective evaluation index and faster fusion. From figure 1 of the fusion results can see, relative to other two kinds of wavelet fusion method, this chapter algorithm of fused image edge details of income is more significant. In order to more intuitive to assess the algorithm can well reserve the detail information source image edge, Laplace operator selects the edge extraction of fused image in figure 3, and the edge extraction of fused image is shown in Fig.5. Obviously the algorithm fusion image edges detail information richer, more conducive to image segmentation, identification further treatment.
5. Conclusion
Based on the analysis of the existing multi-resolution fusion method, based on most current fusion effect good multi-resolution fusion model is only suitable for two images fusion, unable to better suit many images fusion of the problem, this paper proposes an integer ascension based on wavelet transform many images fusion algorithm. For many, blurring the amplitude image fusion experimental simulation and the experimental results were compared and analyzed. Experimental results show that the algorithm not only suitable for real-time rebalanced source image fusion, and can obtain rapid visual effect is better, details, and more abundant of fused image. In addition, can put the image fusion effect the objective evaluation index is introduced to this paper matching degree of the threshold of adaptive selection process, the matching degree of the threshold of optimal choice problem is transformed into an optimal problem, so using intelligent optimization algorithm (such as immune genetic algorithm, cloning algorithm and the particle swarm optimization algorithm, etc) to realize the matching degree of the threshold of optimal choice, it needs to be further research.
Fused image | Objective evaluation index | Fusion time consuming /S | ||
Entropy | Average gradient | Standard deviation | ||
Fig.3(d) | 7.1869 | 22.9037 | 58.1062 | 1.819 |
Fig.3(e) | 7.1043 | 20.4479 | 56.8517 | 2.358 |
Fig.3(f) | 7.0795 | 19.0536 | 55.9841 | 2.137 |
References
- 1.
Blum R. S. Xue Z. Z. Zhang M. 2005 Multi-sensor image fusion and its applications , ,0-84933-417-9 Raton, USA - 2.
Hu G. Liu Z. Xu X. 2008 Research and Recent Revelopment of ImageFusion at Pixel Level . ,25 3 (March 2008),650 655 , (in Chinese) - 3.
Li M. Wu S. J. 2003 ANew Image Fusion Algorithm Based on Wavelet Transform , ,154 159 , Phoenix, Arizona, USA, May 17-22, 2003 - 4.
Wang H. 2004 A New Multiwavelet-based Approach to Image Fusion . ,21 3 (March 2004),177 192 . - 5.
Zhang X. Pan Q. Zhao Y. 2005 Image Fusion Method Based on Stationary Multi-wavelet Transform . ,16 5 (May 2005),605 609 . - 6.
Li W. Zhu X. 2007 An Image Fusion Algorithm Based on Second Generation Wavelet Transform and Its Performance Evaluation . ,33 8 (August 2007),817 822 . - 7.
Yang X. Jin H. Jiao L. 2007 Adaptive Image Fusion Algorithm for Infrared and Visible Light Images Based on DT-CWT . ,26 6 (June 2007),419 424 . - 8.
Wang Z. Yu X. Zhang L. B. 2008 A Remote Sensing Image Fusion Algorithm Based on Integer Wavelet Transform[J] . ,19 11 (November 2008),1542 1545 . - 9.
Filippo N. Andrea G. Stefano B. 2007 Remote Sensing Image Fusion Using the Curvelet Transform . ,8 2007,143 156 . - 10.
Liu K. Guo L. Li H. H. 2007 Image Fusion Based on Finite Ridgelet Transform [J] . ,18 11 (November 2008),1382 1385 . - 11.
Yang L. Guo B. L. Ni W. 2008 Multimodality Medical Image Fusion Based on Multiscale Geometric Analysis of Contourlet Transform[J] . ,72 2008,203 211 . - 12.
Yang X. H. Jiao L. C. 2008 Fusion Algorithm for Remote Sensing Images Based on Nonsubsampled Contourlet Transform[J] . ,34 3 (March 2008),274 281 . - 13.
Lin Z. X. D. 2005 Study on Algorithms of Wavelet Transform for Image Processing Via Lifting Scheme , In: ,47 68 , S.D. Xidian University, Xi’an, China