Registration of Radar and Optical Satellite Images Using Multiscale Filter Technique and Information Measure

1. Introduction

Since a spectacular series of missions in the context of the Earth Observing System (EOS) by NASA beginning from the late 1990’s, the significance of the satellite remote sensing has been recognized all over the world (Kafatos & Qu, 2007; Kaufman et al., 1998). In particular, the applications on hazard mitigation and resource exploration have been widely regarded as one of basic approaches over the past years (e.g., Barrett et al., 1991, Chuvieco, 2008, Fu et al., 2004, Ninomiya et al., 2005, 2006, Realmuto, 2000, Sato et al., 2006, Teeuw, 2007, Urai et al., 2007). In general, remote sensing, from different points of view, includes many branches, or exactly speaking many application fields, such as environmental and ecological remote sensing, geological remote sensing, and military remote sensing. In this chapter, we focus our research on geological applications. However, the proposed algorithms and approaches might be applicable to every fields associated with image registration processing. Although remotely sensed optical images from satellite sensors can meet most needs in the practical applications, considerable weather-dependence limits its functional deployment under some circumstances. For instance, during the period of the devastating Ms 8.0 Wenchuan earthquake in the summer of 2008 (Fu et al., 2009), the most optical images from ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) (Yamaguchi et al., 1998) sensors on the NASA’s Terra satellite can hardly be used to do some refined applications just because of heavy clouds contaminated. However, SAR (Synthetic Aperture RADAR) images are not influenced by climate and time. In practical applications, the optical satellite images, in particular with high resolutions from sensors such as SPOT (Chevrel et al., 1981) and IKONOS (Tanaka & Sugimura, 2001), provides excellent legibility, but they may be affected by the clouds and weather conditions. On the other hand, SAR images are not influenced by climate and they can be obtained day-and-night, but they suffer from a serious intrinsic speckle noise (Franceschetti & Lanari, 1999, Lampropoulos & Boulter, 1997). Therefore, the joint application of these two different kinds of data information will be great interest for many geological problems associated with the remote sensing (e.g., Chen et al., 2003). The first major processing for such a combination is to finish an accurate registration. That is why registration of images coming from different sources is of increasing importance (Le Moigne et al., 2002, Li, 2006, Li et al., 2006c, Li et al., 2008, Schowengerdt, 2006).

Up to now, many registration methods have been proposed to register SAR and optical satellite images manually or (semi-)automatically (e.g., Ali & Clausi, 2002, Cheng et al., 2004, Curlander & Kober, 1990, Dare & Dowman, 1996, 2001, Galland et al., 2005, Hong & Schowengerdt, 2003, 2005, Inglada & Vadon, 2005, Lampropoulos et al., 2003, Lampropoulos et al., 2002, Li et al., 1993, 1995, Li et al., 2007c, Mao et al., 2007, Raucoules & Carnec, 1999, Shu & Tan, 2007; Shu et al., 2005, Thepaut et al., 1998, Vornberger & Bindschadler, 1992, Wang & Chen, 2003, Wegner & Soergel, 2008a, 2008b, Wu & Maitre, 1990, Yang et al., 2005, Zamora et al., 1998,Zhang et al., 2007, Zhang et al., 2004, Zhao & Chen, 2003). In general, these methods can be classified into two major categories: feature based method and intensity based method (Li et al., 2006c). The feature based registration method has been widely implemented into the commercial software, such as ENVI and ERDAS Imagine, initially for the mono-modality image registration, but it encounters significant difficulties when to-be-matched features are not easily extracted from different source images. For example, it is very difficult to extract the features (e.g., roofs and crossroads) from ASTER image for very rural area (e.g., Horonobe, Hokaido, Japan) rather than for metropolitan area (e.g., Tokyo). Furthermore, for the radar image (e.g., RADARSAT-2), this difficulty becomes even heavier than the optical images. Therefore, the corner reflector is often needed to install on the ground so as to enhance the accurate registration process (Li et al., 2009). On the other hand, the intensity based registration method may be easily implemented as a semi- or fully automatic manner, but it is seriously dependent on the choice of similarity measures or metrics, and usually needs large amount of computation (Inglada & Giros, 2004).

Due to the distinct and intrinsic differences of imaging properties between radar and optical images, it is not easy to extract corresponding features from them. Moreover, serious speckle noise existing in radar images further aggravates this problem. Thus, it is a very challenging problem to precisely and automatically register them with a satisfactory accuracy. Wu & Maitre (1990) proposed multiresolution approach to register the SPOT-XS and SEASAT SAR images by matching the contour lines in different scale space representations, but the problems of no distinct features in SAR images still remain. Vornberger & Bindschadler (1992) conducted multispectral research of ice sheets over an area of Greenland by registering Landsat TM and SAR imagery. They found that significant corrections to the SAR data were required to account for range-darkening, non-square pixel dimensions, speckle, and relief distortion. The exposed rock was available to be used as corresponding control points in one area, while it was absent in another area and lakes and streams were used. Li et al. (1993) used an elastic active contour to register the optical and SAR images. Using the contours from the optical image as the initial condition, accurate contour locations in the SAR image are obtained by applying an active contour model. They found that this snake method outperformed manual registration in terms of root mean square error at the control points. Dare & Dowman (1996) tried to develop an automatic system for registering SAR data to optical data by feature matching. In order to enhance the extraction of features from SAR images, they tested various speckle reduction filters and segmentation procedures to aid this procedure. Thepaut et al. (1998) proposed an automatic registration method of multidate and multisensory, i.e. ERS SAR and SPOT, images based on a multichannel consensual segmentation scheme. Zamora et al. (1998) also used the segmentation technique to register multi-sensor images by adaptive clustering prior to performing preprocessing and cepstrum operation to determine the translational displacement. Raucoules & Carnec (1999) proposed the use of local correlation to deal with the co-registration of ERS SAR and SPOT orthoimage by taking the interferometric phases as additional information. Dare & Dowman (2001) introduced an improved model, which is based on their previous work (1996), for automatic feature-based image registration, and presents a robust system for automatically registering SAR and SPOT imagery by incorporating multiple feature extraction and feature matching algorithms. Their experiments on both small and large images showed that in each case a large number of accurate tie points could be identified fully automatically across the images. Ali & Clausi (2002) conducted an investigation on the development of a fully automatic registration system for SAR and optical remote sensing images. The registration method is based on the extraction and matching of common features that are visible in both images. The algorithm involves the five steps, i.e. noise removal, edge extraction, edge linking pattern extraction and pattern matching. The experiments showed that accurate ground control points (GCPs) could be identified automatically. Hong & Schowengerdt (2003, 2005) proposed an automated precise registration approach of ERS2 SAR and Landsat5 TM images using visually-located control points based pre-registration and large-scale edge gradient contours based precise-registrtion. Lampropoulos et al. (2003) presents a proof of concept demonstration to perform image registration from dissimilar sources based on multiple transformations of two quite dissimilar images into new domains, where local or global similarities are extracted. Wang & Chen (2003) proposed an automatic SAR and optical image registration approach based on fuzzy linear feature extraction and neural network controlled resampling and image transformation. Cheng et al. (2004) proposed a SVM (support vector machine) supported edge feature extraction technique to register SAR and optical images. They used a modified Hausdorff distance as a similarity measure and genetic algorithm as the search strategy. Galland et al. (2005) first presented a SAR-to-optical image registration method by using a perfect knowledge of the sensor parameters. They suggested that the feature extraction can be used to perform the refinement of parameters, and the registration can be approximated by simple polynomial transformations. Inglada & Vadon (2005) proposed a fully automatic approach for the fine registration of SPOT5 and ENVISAT/ASAR IMS data. Shu et al. (2005) implemented a mutual information based registration method to match SAR and SPOT images. They used Gabor filters to extract orientation at four directions to enhance the registration process. The experimental results showed that the approach is much better than coarse manual registration. Yang et al. (2005) presents a SAR-to-optical image registration method by using extracted multiple features to improve the control points. Mao et al. (2007) used the approach being similar with Cheng et al. (2004) to register SAR and optical images based on linear features using Hausdorff distance and genetic algorithm. Shu and Tan (2007) proposed a mutual-information based method to register SAR and SPOT images. The novelty is to calculate local contrast of 5x5 windows centred at each point in both images, and to obtain two contrast images to be used to accurately register by assigning the contrast values to each pixel. Zhang et al. (2007) proposed a region-feature based automatic registration workflow for ENVISAT/ASAR and Landsat TM images. Wegner and Soergel (2008) proposed a semi-automatic registration approach based on line features for the improvement of the bridge scene of interest. They used a road extraction approach based on measuring spectral angels in the optical image, and thresholding and morphological operators for bridge extraction in the SAR image. The distance maps are calculated to transform discrete line segments to continuous two-dimensional information, and such distance maps are finally registered using a global transformation and a cross-correlation metric.

Summarizing the past research, we can find most studies used the feature based registration method (e.g., Wang & Chen, 2003, Wu & Maitre, 1990). The key problem of such kind of methods is how to enhance the extraction process of the corresponding features. The registration workflow is usually complicated, and it depends on many assistant means to lubricate the whole process. However, few researchers have tried to automate the SAR-to-optical image registration by using the intensity based approach (e.g., Mao et al., 2007, Shu et al., 2005). This kind of registration methods does not need many assistant means, and fully utilize the intensity information contained in images. The whole registration workflow is driven by an optimization process for similarity measure based object function. This search procedure for the transformation parameters usually needs large amount of computation. In this chapter, we put forth a novel intensity based method to register PALSAR (Kimura & Ito, 2000) and ASTER images based on our previous work (Li et al., 2007c). The intensity based mutual information is used as the similarity measure to automate the registration process. The multiscale steerable Simoncelli filter (Simoncelli & Adelson, 1990, Simoncelli & Farid, 1996, Simoncelli & Freeman, 1995, Simoncelli et al., 1992) is implemented to lubricate the registration process. A hybrid search technique is used to enhance the optimization process of transformation parameters. The experimental results showed that the proposed registration scheme is competent and feasible for PALSAR and ASTER images.

The whole chapter consists of five sections. The first section shortly reviews the research background of radar and optical image registration, and the latest registration techniques, and our novel registration methodology. The second section addresses a generalized mathematical model with mutual-information based similarity measure and multiscale steerable Simoncelli filter technique. The third section gives the description of PALSAR and ASTER satellite imagery. The fourth section discusses the experiment design, process, and results. The fifth section lays out the concluding remarks.

2. Algorithm

2.1. Image registration

The task of image registration is fundamental in image processing. Therefore, it is common problems in nearly all the scientific fields associated with image applications (Brown, 1992, Goshtasby, 2005, Maintz & Viergever, 1998, Pluim et al., 2003, Zitova & Flusser, 2003). In the literature, image registration is also termed image alignment or image matching (e.g., Chen et al., 2007, Chen et al., 2003, Szeliaki, 2005). From a mathematical point of view, image registration can be defined as a process to search a transformation which determines a mapping that is the best match of two or more images of the same object field, acquired by different sensors, or taken by the same sensor at different times (Li, 2006).

For an image pair ( $I_F$ , $I_R$ ) to be registered, the definition of registering the float image ( $I_F$ ) to the reference image ( $I_R$ ) can be expressed mathematically as (1):

$$I_R(x,y)=\zeta (I_F(T_{\alpha }(x,y)))$$ E1

where $T_{\alpha }$ is a transformation function, which maps two spatial coordinates $x$ and $y$ , to the new spatial coordinates $x^{\prime }$ and $y^{\prime }$ by the set of parameters $\alpha$ as (2):

$$(x^{\prime },y^{\prime })=T_{\alpha }(x,y)$$ E2

$\zeta$ is a one dimensional intensity or radiometric interpolation function.

The intensity based image registration can be analyzed as a non-convex optimization problem (Li, 2006, Modersitzki, 2004). This can be expressed mathematically as (3):

$${\alpha }^{*}=Arg\underset{T_{\alpha }}{optima}(S_{T_{\alpha }}(I_F,I_R))$$ E3

where $S$ is the similarity measure, and ${\alpha }^{*}$ is the optima estimated by the search algorithm.

The generalized registration process using eqns. (1-3) for the intensity based method can always be depicted in Figure 1 (Li et al., 2006c, Yoo, 2004). It can be found that the intensity based image registration is an iterated process to search for the optimized transformation parameters.

Figure 1.

A generalized framework for intensity based image registration

2.2. Mutual information

As pointed in Figure 1, similarity measure is an element of the image registration. It is used to construct an object function associated with image intensities for the optimization step. It should be noted that the term of similarity measure is different from the term of similarity metric (Cover & Thomas, 2006). In this chapter, we used the former. Starting in 1995, with the successful implementation of mutual information as a novel similarity measure to the multimodality medical image registration (Maes et al., 1997, Viola & Wells, 1995), the achievement of the intensity based automated image registration becomes possible. Based on the information theory (Cover & Thomas, 2006), the standard definition of mutual information, $MI(I_F,I_R)$ , of two images $I_F$ and $I_R$ can be written as (4):

$$MI(I_F,I_R)=H(I_F)+H(I_R)-H(I_F,I_R)$$ E4

where $H(I_F)$ and $H(I_R)$ are the marginal entropies of $I_F$ and $I_R$ , and $H(I_F,I_R)$ is their joint entropy. Considering the definition in (4), the mutual information is maximal when the two images are totally geometrically aligned by a certain transformation matrix. In practice, the normalized version (NMI) of the standard mutual information is popular. It may be defined as (5):

$$NMI(I_F,I_R)=\frac{H(I_F,I_R)}{H(I_F)+H(I_R)}$$ E5

In order to compute of mutual information, the marginal entropies and joint entropy of the image pair should be calculated. These entropies can be calculated from the probability density functions. Furthermore, these probability density functions can be estimated from the histogram of images or other tricks such as Parzen windows (Glavinovic, 1996, Parzen, 1962). The detailed computation is left out in this chapter and it can be found in (Maes et al., 1997, Viola & Wells, 1995).

.

2.3 Simoncelli filter

The multiresolution decomposition techniques using wavelet-like filters are usually adopted to enhance the image registration (Le Moigne et al., 2002). Because the steerable Simoncelli filters are more robust to translation, rotation and noise than the standard Daubechies wavelet filters (Cole-Rhodes et al., 2003), it enables us to use it for registration of SAR and optical satellite images (Li et al., 2007c).

According to the definition of steerable Simoncelli filters (Simoncelli & Freeman, 1995), the steerable pyramid is a multiscale representation that is translation-invariant, but that also includes representation of orientation. Furthermore, the representation of orientation is designed to be rotation-invariant. The basis and projection functions are oriented (i.e., steerable) filters, localized in space and frequency. It is overcomplete to avoid aliasing. It is also "tight frame", i.e. the projection functions and basis functions are identical, though it is not an orthogonal representation.

The diagram for steerable Simoncelli pyramid may be depicted in Figure 2. The filters {Fhi0, Flo0} are used to initially split the image into a highpass residual band and a lowpass subband. This lowpass band Flo0 is then split into some lowerpass bands {Flo1, Flo2,...}. {FB0, FB1,...} represent the oriented subbands which ensure that the representation is rotation-invariant. In order to ensure some translation-invariance, the outputs of the high-pass filter and of the band-pass filters are not subsampled. The resulting transform is overcomplete by a factor of 4k/3, where k is the number of oriented band-pass filters. The scale tuning of the filters is constrained by the recursive system diagram. The orientation tuning is constrained by requiring the property of steerability (Cole-Rhodes et al., 2003, Li et al., 2007c, Simoncelli & Freeman, 1995).

Figure 2.

A steerable Simoncelli pyramid with three-level decompositions of original image

2.4. Workflow

As shown in Figure 1, the computational process of intensity based image registrations can be partitioned into four major modules, i.e., intensity interpolation, mapping transformation, similarity measure, and optimization strategy for the parameter space. Till now, our developed registration system has implemented 7 interpolation algorithms, 25 similarity measures, 11 optimization algorithms, and it supported 7 transformations from rigid to polynomial mapping (Li et al., 2007a). The whole workflow of image registration in this chapter can be depicted in Figure 3.

Figure 3.

Workflow of SAR-to-optical image registration

3. Data

3.1. ASTER

The Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) sensor instrument was launched in December 18, 1999, onboard the first NASA’s EOS series of satellites, Terra. ASTER covers a wide spectral region with 14 bands from visible to thermal infrared with high spatial, spectral and radiometric resolution. Three visible and near infrared (VNIR) bands, six shortwave infrared (SWIR) bands, and five thermal infrared (TIR) bands have a spatial resolution of 15 m, 30 m, and 90 m, respectively. In addition, ASTER has a stereoscopic capability for the bands (bands 3N and 3B) in near infrared region (Iwasaki & Fujisada, 2005, Yamaguchi et al., 1998), and so far, it can generate Digital Elevation Model (DEM) data with high accuracy (Fujisada et al., 2005).

ASTER-TIR is the first satellite-borne multispectral TIR remote sensing system with spectral, spatial and radiometric resolutions adequate for geological applications (e.g., Ninomiya et al., 2005). Compared with two bands of Landsat TM or ETM in SWIR region (between 1.6 to 2.5 microns), ASTER sensor has 6 bands in this region and provides an opportunity to identify mineral component of surface rocks in the semi-arid to arid region (e.g., Rowan & Mars, 2003). Therefore, ASTER imaging system can provide some important capabilities to identify lithologic and mineralogic features on earth surface (e.g., Fu et al., 2007, Ninomiya et al., 2005). Especially, ASTER multispectral TIR sensor can provide an important tool for monitoring heat flow related to volcanic activities (e.g., Urai et al., 2007). Therefore, ASTER can provide a potential tool for mapping the wide multiple-aim geologic products from regional to global scales because high-resolution multispectral dada obtained by ASTER can cover almost throughout earth surface.

3.2. PALSAR

PALSAR is the abbreviation of Phased Array type L-band Synthetic Aperture Radar. It is an active microwave sensor using L-band frequency, operated at all-weather conditions regardless of day and night, launched with ALOS satellite on January 24, 2006 in Japan. It is improved based on SAR onboard the first earth observation satellite (JERS-1) with multi-mode observation functions of multi-polarization, variable off-nadir angle, and switching spatial resolution and swath width observation (Igarashi, 2000, Kimura & Ito, 2000). It provides higher performance than the JERS-1's SAR with a totally new advantageous observation mode (i.e., ScanSAR). PALSAR has incorporated many highly advanced observation technologies, and is expected to contribute greatly in areas such as resource exploration, environmental monitoring on earth and monitoring of natural disasters (e.g., Rosenqvist et al., 2007, Takada et al., 2009). The signals are recorded in complex notation on PALSAR sensor from which their amplitude and phases could be computed. The specifications of PALSAR sensor can be summarized in Table 1. It should be noted that PALSAR sensor can not observe the areas beyond 87.8 Degrees north latitude and 75.9 Degrees south latitude, when the off-nadir angle is 41.5 Degrees (http://www.eorc.jaxa.jp/ALOS/en/about/palsar.htm).

3.3. Image set

In this chapter, all image pairs were extracted from each of the full scene ASTER and PALSAR data. The ASTER L1B Band 1 data and PALSAR fine mode data were used in the experiments. The extracted sub-images for experiments have 128x128, 256x256, 512x512, 1024x1024 pixels, respectively. The research area covers the part of the city of Tokyo, Japan. One sub-image pair has been shown in Figure 4. The very clear difference in terms of visual appearance of features can be observed. The same spatial features can not be easily found from both images. PALSAR image is inevitably contaminated by the speckle noise and strongly scattered signals from any corners on the earth surface. In principle, physical properties and viewing geometries between ASTER and PALSAR images are intrinsic different. However, the effect resulted from the viewing geometry will be alleviated in flat regions.

Mode	Fine		ScanSAR	Polarimetric ^*
Center frequency (MHz)	1270 (L-band)
Chirp bandwidth (MHz)	28	14	14, 28	14
Polarization	HH VV	HH+HV VV+VH	HH VV	HH+HV+VH+VV
Incident angle (Degree)	8-60	8-60	18-43	8-30
Range resolution (m)	7-44	14-88	100 (Multi look)	24-89
Observation swath (km)	40-70	40-70	250-350	20-65
Bit length (bits)	5	5	5	3 or 5
Data rate (Mbps)	240	240	120, 240	240
NE sigma zero ^** (dB)	<-23 (Swath width 70km) <-25 (Swath width 60km)		<-25	<-29
S/A ^*** (dB)	"/16 (Swath width 70km) "/21 (Swath width 60km)		"/21	"/19
Radiometric accuracy	Scene: 1dB / Orbit: 1.5dB

Table 1.

Due to power consumption, the operation time will be limited.** Valid for off-nadir angle with 34.3 Degrees (Fine mode), 34.1 Degrees (ScanSAR mode), 21.5 Degrees (Polarimetric mode).*** S/A level may deteriorate due to engineering changes in PALSAR Characteristics of PALSAR

Figure 4.

ASTER (left) and PALSAR (right) images of Tokyo bay, Japan with 512x512 pixels

4. Experiments

The registration scheme of our proposed method (Figure 3) includes three major stages similar with our previous work (Li et al., 2007c). Both SAR and optical image are firstly decomposed into a steerable pyramid. Subbands FBi (Figure 2) are utilized to extract features in the provided image set. The partial volume intensity interpolation (Li et al., 2006a) is adopted for the estimation of probability density functions. The particle swarm optimization (Li & Sato, 2007) is used to globally search the parameter space of the registration function at the coarsest level. The local stochastic gradient search, which is conducted by a simultaneous perturbation stochastic approximation technique (Li et al., 2006b, 2007b), is implemented to optimize the registration function at other levels. The result of the global optimization is used as the initial guess of the local stochastic gradient search.

Figure 5 shows the visualization check of one registration result. By visual comparison, we can note that the registration of the proposed scheme is much better than the manual method conducted in some commercial software. To do a numerical comparison, we manually locate 5-10 pairs of check points with a good distribution and evaluate the registration accuracies according to RMSE (root mean squared error). The RMSE values are 5-7 pixels for manual registration and 1-3 pixels for our registration. This indicates that the registration accuracy is greatly improved by our proposed scheme. In particular, the results are better than our previous work on the registration of JERS-1 SAR and ASTER images (Li et al., 2007c).

Figure 5.

Visualizing check of PALSAR and ASTER image (Left: before registration; Right: after affine registration)

5. Conclusions

The experimental results showed that proposed scheme can be capable to register radar and optical satellite images such as PALSAR and ASTER images. Compared to the traditional manual method, the scheme using multiscale filter technique and information measure can greatly enhance the registration process. The hybrid search/optimization approach is relatively less sensitive to initial guess, and make the registration process robust. Meanwhile, our method maintains comparable accuracy comparing with the traditional manual method. Except for the computing time, the scheme might encounter difficulties when the two images with very time differences.

Future work should include two parts. The first one is to conduct many more experiments on different regions. The other one is to incorporate some feature based technique to speed up the search process.

Acknowledgments

This research has been mainly funded under the research contract with the Ministry of Economy, Trade and Industry (METI), Japan as the part of the R&D of Remote Sensing Technologies for Non-renewable Resources (SEKITOKU). QL would like to thank I. Sato, Y. Murakami, M. Urai, Y. Ninomiya, F. Sakuma, and S. Okuyama (AIST) for all of their help. ASTER and PALSAR data were provided by the Earth Remote Sensing Data Analysis Centre (ERSDAC), Japan.