Evaluation of the four commonly used features for SAR image registration in terms of several criteria.
An investigation on the appropriate feature and parameter retrieval algorithm is conducted for feature-based registration of synthetic aperture radar (SAR) images. The commonly used features such as tie points, Harris corner, SIFT, and SURF are comprehensively evaluated. SURF is shown to outperform others on criteria such as the geometrical invariance of feature and descriptor, the extraction and matching speed, the localization accuracy, as well as the robustness to decorrelation and speckling. The processing result reveals that SURF has nice flexibility to SAR speckles for the potential relationship between Fast-Hessian detector and refined Lee filter. Moreover, the use of Fast-Hessian to oversampled images with unaltered sampling step helps to improve the registration accuracy to subpixel (i.e., <1 pixel). As for parameter retrieval, the widely used random sample consensus (RANSAC) is inappropriate because it may trap into local occlusion and result in uncertain estimation. An extended fast least trimmed squares (EF-LTS) is proposed, which behaves stable and averagely better than RANSAC. Fitting SURF features with EF-LTS is hence suggested for SAR image registration. The nice performance of this scheme is validated on both InSAR and MiniSAR image pairs.
- extended fast least trimmed squares (EF-LTS)
- feature-based image registration
- parameter estimation
- speeded up robust feature (SURF)
- synthetic aperture radar (SAR)
Synthetic aperture radar (SAR) as an irreplaceable remote sensing technique has been used for earth observation and environment monitoring for a long time due to its all-weather and all-day operational capability. A large number of airborne and spaceborne SAR sensors have been deployed recently. Nevertheless, the difference in sensors and imaging geometries will always introduce a geometrical warp between images which should be compensated before any joint application of multiple SAR images for accurate apperception and understanding of target and scene. Image registration is just dedicated to retrieve the warp function to align the same pixel position in each SAR image to the same target in the global system.
A lot of SAR image registration techniques have been developed hitherto. In this chapter, we focus on the algorithms that conduct registration based on image features, such as contour, region, line, and point. Contour, region, and line as well as their combination are often used for registration of multi-modality images. For SAR images with geometrical distortion and speckle, point feature is generally much clearer and easier extracted. Tie points, corner, and keypoint are the commonly used features in SAR image registration. Tie points usually refer to the features extracted from tie patches in SAR image registration [1, 2, 3, 4]. The tie patches are first matched by region-based algorithms, and the tie points are then located by extracting the geometrical centers or centroids of the matched patches. Corner denotes another kind of point feature which has two dominant but different edge directions in local neighborhood. In SAR image registration, Harris corner  is the commonly used point feature [2, 6] whose response function is the weighted addition of the determinant and squared trace of the first-order moment matrix which describes the local neighboring gradient distribution of a point. Keypoint refers to the point differing in brightness or color compared with the surrounding. It is identified to further enable a complementary description of image structure that cannot be characterized by corner. The scale invariant feature transform (SIFT)  and the speeded up robust feature (SURF)  are the widely used keypoints in SAR image registration. SIFT was developed by Lowe  to extract features based on the automatic scale selection theory. Lindeberg  found that the only possible scale-space kernel under a variety of reasonable assumptions is the Gaussian function, and he experimented with both the traces of Hessian matrix, i.e., the Laplacian of Gaussian (LoG) and the determinant of Hessian (DoH) matrix, to detect the blob-like structures. To extract keypoints efficiently, Lowe  simplified LoG with the difference of Gaussian (DoG) further. SIFT enables not only a feature detector, but also a 128D vectorized descriptor of gradient and orientation. Mikolajczyk and Schmid conducted a comparative study on 10 different local descriptors and found that SIFT performs the best on treating the common image deformations . SIFT has been widely used in SAR image registration [11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]. Chen et al.  systematically evaluated the application of SIFT to SAR and displayed its usefulness for image registration. Schwind et al.  further indicated that SIFT is a robust alternative for point feature-based SAR image registration. The bottleneck of SIFT is the speed [8, 13, 15], which hinders its application to general SAR image registration. To accelerate SIFT, Schwind et al.  proposed to skip features detected at the first octave of the scale space pyramid (SSP) because matches extracted from this octave have the highest matching false alarm rate (MFAR). This can save the processing time without reducing the number of correct matches greatly. However, the first scale octave in SSP of SIFT refers to the image of original size or doubled size which has the highest resolution in SSP. Thus, the features extracted from this octave are more accurate for image registration . Therefore, the discarding of matches from the first octave may influence the final registration accuracy. Based on the same scheme as SIFT, SURF developed by Bay et al.  uses a combination of novel detection, description, and matching methods to simplify SIFT. SURF extracts feature based on DoH instead of its trace because DoH bears slightly better scale selection property under non-Euclidean affine transformation than LoG. Bay et al. used a Fast-Hessian detector with box filters to approximate DoH. The SURF descriptor is a 64D vector composed by the Harr wavelet responses of the square area around keypoint. SURF has been demonstrated to outperform SIFT on speed, repeatability, distinctiveness, and robustness . It has been used for multispectral satellite image registration , seabed recognition based on sonar images , and SAR image registration [26, 27, 28, 29].
The next procedure after feature extraction is to match the features for correspondences. For tie points, this procedure is unnecessary because they have already matched when extracted. For other features, the correspondences are usually constructed by optimizing certain merit function, such as maximizing the similarity or minimizing the difference. The warp function can then be retrieved by fitting the obtained correspondences. For correspondences without any mismatches, the retrieval can be easily conducted by fitting them with the least squares (LS). However, for the general registration cases, the initial correspondences often contain mismatches. Therefore, the robust retrieval algorithms which are insensitive to outliers are needed. In many existing literatures on feature-based SAR image registration [15, 16, 26, 27], the random sample consensus (RANSAC)  has been widely used and recommended for warp function retrieval. RANSAC conducts the estimation by randomly sampling a minimal sampling set (MSS) to achieve an estimation of the warping, and the entire datasets are then checked on the estimation for a consensus set (CS) of correspondences. These two steps are iterated until the largest CS is achieved . Besides this, the least median squares (LMedS)  and the fast least trimmed squares (Fast-LTS)  have also been used [4, 34, 35]. There are also some other approaches which use different matching and retrieval algorithms with different features, which can be referred to the related reviewing articles [36, 37, 38].
Although lots of approaches have been developed for feature-based SAR image registration, there are still some open problems that have not been perfectly solved yet. In this chapter, we concentrate on two problems, i.e., which feature is more appropriate and which retrieval algorithm performs much better? The first problem is related to the feature operator, which is focused in Sections 2 and 3. We give a detailed evaluation to tie points, Harris corner, SIFT, and SURF in terms of the geometrical invariance of feature and descriptor, extraction and matching speed, localization accuracy, robustness to decorrelation, and flexibility to speckle. SURF is identified to outperform others. Particularly, we find that SURF is flexible to speckle for the close relationship between Fast-Hessian detector and refined Lee speckle filter. SURF is thus more competent for SAR image registration. The second problem is posed in Section 4 with the reason that the widely used RANSAC is found instable for parameter estimation in the registration of an interferometric SAR (InSAR) image pair. The uncertainty arises from its inappropriate loss function and estimation strategy. Based on the scheme of Fast-LTS, an extended Fast-LTS (EF-LTS) is presented for 2D robust parameter estimation. Experiment on InSAR image pair demonstrates that EF-LTS is more stable and robust than RANSAC. It is more appropriate and competent for SAR image registration. Based on these, we recommend fitting the SURF features with EF-LTS to conduct the registration. We further evaluate this scheme in Section 5 by processing the MiniSAR image pair, and the result complies with our expectation. Section 6 concludes the chapter finally.
2. Comparative analysis on the commonly used features for SAR image registration
SAR image is acquired with intensity and phase, which should be transformed into the real one before feature detection by taking the intensity or the logarithmic intensity of the image. Instead of proposing a novel feature for SAR image registration, we identify the appropriate feature from the widely used tie points, Harris corner, SIFT, and SURF by evaluating them on several criteria. In this section, the features will be evaluated on the following six factors, i.e., the geometrical invariance of feature, the extraction speed, the localization accuracy, the geometric invariance of descriptor, the matching speed, and the robustness to decorrelation, while the impact of SAR speckles will be particularly focused and analyzed in Section 3.
2.1 Geometrical invariance of feature
The geometrical invariance of feature refers to which degree of warping a same feature can still be extracted from the warped images by a detector. Cross-correlation (CC) is sensitive to image rotation and scaling, hence the CC-based tie points are only invariant to the following translation transformation:
where <·> denotes the ensemble average;
where the weight
The Harris response
2.2 Feature extraction speed
The extraction speed is mainly influenced by the computational load of detector. Tie points are identified by traversing all potential offsets to calculate CC. The resulted computational load is heavy. The Harris response
When applied in practice, Gaussians should be discretized and cropped. The corresponding discretized and cropped
2.3 Localization accuracy of feature
Image registration accuracy is closely determined by the localization accuracy of feature. Tie points achieve subpixel accuracy by oversampling the image patches  or CC obtained in coarse registration . Higher sampling rate indicates higher accuracy, but it also signifies larger data sets, heavier computational load, and more severe aliasing. Keypoint in SIFT and SURF is first located as the extrema using the non-maximum suppression technique, and is then refined to subpixel and sub-scale accuracy by Taylor fitting a 3D quadratic to the scale function DoG (for SIFT) or the approximated DoH (for SURF) in the scale space :
Therefore, SIFT and SURF can obtain the highest accuracy. However, it should be noted that although the subpixel feature localization is the precondition of accurate image registration, it cannot guarantee a subpixel image registration. For high accurate SAR image registration, we should further evaluate the features carefully, and this will be detailed in Section 3.4.
2.4 Geometrical invariance of descriptor
Feature descriptor is usually a vector depicting the neighboring information of a feature. It plays a key role in feature matching. The descriptor’s geometrical invariance determines the degree of warping to which features can still be successfully matched. Harris corner and tie points have no descriptor. From feature matching point of view, however, they both adopt template matching by selecting the image square centered around the feature as descriptor, which is only invariant to translation. Thus, tie points and Harris corner can be successfully matched only under weak warping. SIFT and SURF descriptors enable a good compromise between feature complexity and the robustness to commonly occurring deformation such as weak affine transformation [7, 8, 43]:
2.5 Matching speed of feature
Feature matching is usually conducted based on certain merit function of the descriptors. In feature-based SAR image registration, the merit function is to maximize the similarity (such as CC ) or minimize the differences (such as Euclidean distance [7, 8]). A correspondence is detected if it can optimize the merit function. For SIFT and SURF, the merit of an optimal correspondence has also to be certain times larger than the second optimal merit. Matching speed is mainly determined by the calculation of merit. For tie points and Harris corner, the merit function is the maximum of CC, which can be obtained on complex data or magnitude data , referring to coherent CC or incoherent CC, respectively. The registration accuracy attained by coherent CC is much higher than that by incoherent CC . If
The merit function in SIFT and SURF is the minimum of the Euclidean distance. If
2.6 Robustness to decorrelation
SAR decorrelation sources can be classified into two categories, i.e., the geometrical warping and radiometric warping. Geometrical warping will lead to decorrelation and influence the CC-based feature matching, which relates to the geometrical invariance of feature discussed above. Here, we focus on the radiometric warping-induced decorrelation. Such decorrelation is resulted because CC is only invariant to affine changes in scattering. Target scattering in microwave band is sensitive to frequency, bandwidth, and polarization. All these introduce a complex nonlinear radiometric warping, which degrades SAR information and aggravates image registration by impacting the localization of tie points. The localization accuracy of tie points is measured by the error standard deviation
3. Impact of SAR speckles on accurate feature extraction
SAR image is acquired by actively measuring and coherently processing the electromagnetic scattering of target. The interference of scatterings from scatterers within each resolution cell produces a pixel-to-pixel variation in image intensity and results in the so-called speckle. In this section, we first conduct a qualitative evaluation on the flexibility of existing features to speckles. An experimental evaluation of the identified feature is then conducted and some necessary improvements are developed for high accurate SAR image registration.
3.1 Flexibility to image speckling
For CC-based tie points, the assumption that the scattering is locally stationary and ergodic may not be tenable in the existence of speckles. As a result, the correlation estimation as well as the localization and matching of the feature will be biased. For the geometrical texture-based detectors such as Harris, SIFT, and SURF, speckles may lead to false texture and high MFAR. To achieve stable features from the speckle-contaminated SAR image, a conceivable method is to suppress speckle beforehand. Schwind et al.  suggested adopting the ISEF filter, but they indicated that ISEF filter and any other filter may slightly affect feature localization and registration quality. Hence, a better strategy is to conduct speckle suppression while feature extraction, i.e., the detector should be flexible to speckling.
Harris detector obtains features using the first-order image derivatives which are not robust to speckles. As a result, Harris detector may extract many features, but most of the extracted features are speckles with only a few correct matches. This influence has been also observed by Schwind et al.  when using SIFT to SAR: only very few matches are constructed at the first octave of SSP although with extensive number of extractable features, and the matches from this octave have the highest MFAR of all the octaves. The first scale octave refers to the original or double-sized images which are of the highest resolution and the largest number of extractable keypoints. The highest MFAR at this octave clearly indicates the bad flexibility of SIFT to speckles, while the lower MFAR at higher octaves is just due to the fact that larger image smoothing reduces the speckle. Different from SIFT, SURF can deal with speckle very well because of the relationship between Fast-Hessian detector and refined Lee speckle filter.
3.2 Refined Lee speckle filter
An ideal speckle filter should adaptively smooth speckle, retain the sharpness of boundaries and edges, and preserve the subtle but distinguishable details. The most widely used boxcar filter replaces a pixel with the mean of its windowed neighborhood. This filter can be easily implemented and works very well in homogeneous area, but will degrade spatial resolution in inhomogeneous area due to the indiscriminate averaging . To solve this, many filtering techniques have been proposed. The refined Lee speckle filter is just such a filter which uses the local statistics to suppress speckles without degrading image. To identify pixels with the similar texture, Lee devised the eight non-square edge-aligned windows, as shown in Figure 2. In the course of filtering, one of the windows is matched to calculate local statistics based on edge direction, and the minimum mean square algorithm is then adopted for filtering. As a result, this filter can effectively reduce the speckle without degrading the edge .
3.3 Relationship between Fast-Hessian detector and refined Lee filter
As mentioned previously, SURF extracts features based on the box filter displayed in Figure 1. Box filter not only speeds up feature extraction, but also enables SURF to extract features while reducing speckles. In
which corresponds to DoH in (7), where the constant 0.9 is used to balance the expression for the Hessian’s determinant. Then, SSP in SURF just indicates that we adopt a series of box filters of different size to filter speckles and extract features of different scales. Hence, SURF is very flexible to deal with speckle.
3.4 Evaluation of SURF for SAR image subpixel registration
As listed in Table 1, according to the comparative analysis in Sections 2 and 3.1 on several criteria, we can obtain that for the general registration of SAR images
SURF outperforms others in terms of the considered criteria.
SIFT is applicable when no strict requirement for speed.
Harris may be appropriate for coarse registration.
Tie points are fit for images with slight distortion and weak decorrelation and require heavy computation load.
|Items||Tie points||Harris corner||SIFT||SURF|
|Geometrical invariance of feature||Translation||Rotation and translation||Scaling, rotation, and translation||Scaling, rotation, and translation|
|Feature extraction speed||Slower||Faster||Slow||Fast|
|Feature localization accuracy||Subpixel*||Pixel||Subpixel||Subpixel|
|Geometrical invariance of feature descriptor||Translation||Translation||Affine transform||Affine transform|
|Feature matching speed||Slow||Slow||Fast||Faster|
|Robustness to decorrelation||Worse||Bad||Good||Good|
|Flexibility to image speckle||Good||Bad||Bad||Better|
From these, we can see that SURF is more appropriate and competent for general SAR image registration. Nevertheless, SAR applications, like DEM retrieval and deformation estimation usually impose a strict requirement for registration accuracy. To ensure an acceptable result, the registration accuracy should be subpixel. To evaluate the capability of SURF for subpixel image registration, we devise a comparative experiment on some contrived SAR image pairs. Figure 3 shows a SAR image of Enta Volcano acquired by SIR-C/X-SAR. We treat this image as the master and transform it to model an affine geometrical warp for the slave image:
where “#” denotes “the number of.”
We evaluate the two SURF detectors on four image pairs with different transformations, the retrieved warp matrix parameters,
|Detectors||Estimated affine warping parameters||Correct match number and |
|0.7164||0.0415||−0.0481||0.8059||2.3151||3.6269||42 (0.1923)||(0.7109, 0.8770)||1.3725|
|0.7195||0.0425||−0.0347||0.8067||2.0444||1.3480||22 (0.2414)||(0.5887, 0.7854)||1.1070|
|0.7186||0.0458||−0.0395||0.8085||1.3565||2.2052||73 (0.1300)||(0.6219, 0.6602)||0.3949|
|0.7192||0.0450||−0.0403||0.8088||1.4752||2.3425||129 (0.1164)||(0.3001, 0.4602)||0.2321|
|0.7181||0.0453||−0.0402||0.8093||1.6070||2.1746||188 (0.1754)||(0.2580, 0.3790)||0.2439|
|0.7186||0.0457||−0.0398||0.8094||1.4895||2.1085||176 (0.1619)||(0.2819, 0.3874)||0.3596|
|0.9370||0.1887||−0.1576||1.0908||−10.5603||−3.7546||55 (0.0678)||(0.6040, 0.7075)||0.3598|
|0.9298||0.1909||−0.1603||1.0868||−9.9304||−2.2059||25 (0.2188)||(0.7949, 1.2405)||1.3231|
|0.9352||0.1898||−0.1618||1.0940||−10.4452||−3.4432||170 (0.0449)||(0.3821, 0.5200)||0.0698|
|0.9361||0.1890||−0.1613||1.0937||−10.4329||−3.4817||419 (0.0141)||(0.2267, 0.3080)||0.1058|
|0.9361||0.1890||−0.1617||1.0937||−10.4252||−3.4490||735 (0.0252)||(0.1703, 0.2143)||0.0894|
|0.9360||0.1890||−0.1616||1.0938||−10.4227||−3.4476||893 (0.0262)||(0.1601, 0.2273)||0.0908|
|1.1387||0.0984||−0.0736||1.3238||−2.2175||1.7098||47 (0.0408)||(0.7131, 0.9156)||3.7101|
|1.1402||0.1055||−0.0805||1.3160||−3.6578||3.5156||29 (0.1212)||(1.0153, 0.9129)||2.1610|
|1.1361||0.1038||−0.0897||1.3160||−2.3833||5.6308||157 (0.0427)||(0.3856, 0.4829)||0.3166|
|1.1363||0.1038||−0.0895||1.3165||−2.4408||5.4805||476 (0.0206)||(0.1902, 0.3197)||0.1784|
|1.1365||0.1037||−0.0894||1.3159||−2.4575||5.5336||983 (0.0180)||(0.1616, 0.2378)||0.1954|
|1.1363||0.1037||−0.0894||1.3160||−2.4582||5.5270||1293 (0.0300)||(0.1432, 0.2119)||0.1903|
|1.2033||0.0744||−0.0753||1.3054||−4.2454||2.4055||52 (0.0545)||(0.7247, 0.7616)||1.3900|
|1.1959||0.0695||−0.0650||1.3079||−1.8954||0.0939||24 (0.0769)||(0.9570, 1.1486)||3.6836|
|1.2075||0.0778||−0.0735||1.3066||−5.0414||2.0074||172 (0.0227)||(0.3858, 0.4182)||0.5695|
|1.2076||0.0766||−0.0719||1.3077||−5.0751||1.6740||514 (0.0172)||(0.2207, 0.3552)||0.2844|
|1.2078||0.0777||−0.0718||1.3077||−5.1181||1.6590||1052 (0.0177)||(0.1506, 0.2289)||0.2416|
|1.2077||0.0778||−0.0719||1.3078||−5.1297||1.6397||1451 (0.0176)||(0.1343, 0.1998)||0.2203|
4. Appropriate retrieval algorithm for SAR image registration
The next procedure after feature extraction is to retrieve the warp function from the attained correspondences. Due to the influences of spatial/temporal decorrelation, system noise, and environmental interference, or the non-robustness in the depiction and matching of features, there are always mismatches in the constructed correspondences. It is difficult to get
Furthermore, unlike the pinhole imaging of optical camera, SAR acquires the imagery using a slant-range geometry which cannot be modeled as a central projection . As a result, the warp model between SAR images is dependent on the system parameter, imaging geometry, and target relief, and we cannot adopt a global homography or essential matrix to model the geometrical warping then. Nevertheless, when the system parameter and imaging geometry are fixed and the area-of-interest has gentle topography, we can conventionally approximate the warp function as a low-order polynomial . This indicates our strategy in the retrieval of registration parameters, to focus on the global registration instead of local discontentment.
4.1 Evaluation of RANSAC for SAR image registration
RANSAC  has been widely used in feature-based SAR image registrations for parameter retrieval [15, 16, 26, 27]. Unlike LS which uses all the available data to estimate parameters, RANSAC conducts the estimation using a few-to-many strategy or a local-to-global strategy. A MSS is randomly sampled from the constructed correspondences to achieve an estimation of the warp function firstly. The cardinality of MSS, i.e., the smallest sufficiency to determine the warp parameters, is just related to the degree of freedom (DoF) of the warp function. For example, the cardinality will be 3 for affine transformation of 6 DOFs. The entire dataset are then checked for those correspondences consistent with the retrieved warping to construct a larger CS. These two steps are repeated until the largest CS is finally achieved for parameter estimation. This local-to-global strategy is tenable only if any MSS of inliers can generate the “true value” of warp parameters . But it is often hard to keep this in real registration due to the unavoidable noise and local distortion, i.e., a different estimation of parameters will be achieved from a different MSS configuration of inliers. This uncertainty is even more severe in SAR image registration because SAR warping varies from pixel to pixel and the low-order polynomial approximation only accounts for global registration instead of local contentment. The local-to-global strategy may then magnify the local distortion, aggravate the estimation uncertainty, and damnify the global registration accuracy although a largest CS is identified. To demonstrate this, we devise an experiment to coregister a spaceborne InSAR image pair as shown in Figure 4(a) and (b). The two images are acquired by RadarSat-2 on May 4 and 28, 2008, respectively. The scene is within South Phoenix, AZ, USA with some buildings and vegetable lands. We first use
|RANSAC||0.9992||2.5228 × 10−4||1.8796 × 10−4||2.4879 × 10−4||1.3770 × 10−4||3.3408 × 10−4||0.5462||0.0046|
|EF-LTS||0.9990||0.0000||2.5485 × 10−4||0.0000||9.2440 × 10−5||0.0000||0.5483||0.0000|
|RANSAC||0.9996||2.2264 × 10−4||−2.6068||0.1955||0.5133||0.1509||−37.36||0.1126|
The uncertainty of RANSAC in SAR image registration just comes from its retrieval strategy and loss function. To achieve a stable registration for SAR images, a feasible improvement is to estimate the parameters with more correspondences to reflect the true support than just a MSS, and to apply an appropriate loss function. This leads us another direction to the robust parameter regression.
The widely used LS is now being criticized more and more for lack of robustness. To tackle with this, some robust regression approaches were developed, like LMedS  and the least trimmed squares (LTS) . LMedS implements the regression by minimizing the median of residual squares. This makes LMedS so robust that it can still obtain a reasonable estimation even if 50% of the dataset are outliers. So the breakdown point of LMedS is as high as 50%. LTS is a modification of LS with the same breakpoint as LMedS. It also fits the linear model:
It has been proved that
The trimming constant
4.3 EF-LTS for SAR image registration
Fast-LTS is appropriate for 1D linear regression formulated in (19). However, for SAR image registration, what we need to do is to fit a 2D polynomial regression
Then construct the initial
Carry out two C-steps on
and estimate the error scales
The credible correspondence in both directions of
where “&” denotes the logical AND operator. The final estimations
which in fact indicates the weighted LS.
Step 33 makes EF-LTS obtain more accurate and stable estimation than the original LTS. The logical AND in (29) shows that only the feature correspondence which is correctly matched in both
In Fast-LTS, the random sampling number
Since the trimming constant
Therefore, if a required false alarm rate
Thus, iteration in EF-LTS is controlled by the inlier percentage rather than the inlier number. Table 4 shows the sampling number
The inlier percentage
Thus, besides introducing more iterations and computation load, higher
When the correspondence number
To evaluate EF-LTS for SAR image registration, we also use it to the InSAR image pair given in Figure 4(a) and (b). Similarly, the feature correspondences are first constructed by SURF with
5. Experiment and analysis
Based on the finding in Sections 2–4, we propose to conduct high accurate SAR image registration by using EF-LTS to fit the SURF correspondences. The scheme works as follows:
Actually, this scheme has been put into practice in the above experiments. In this section, we further devise an experiment to check it on MiniSAR pair. The images we use are two high-resolution SAR images of the entrance gate of the Sandia Research Park acquired by the Ku-Band MiniSAR system developed by the Sandia Laboratory . The images are taken from different tracks with different incidences and squints, as listed in Table 5, while the platform altitude is just beyond 1 km. All these reveal the nontrivial target relief-induced geometrical warping between images, which, however, cannot be compensated beforehand for lack of ground truth such as DEM and target height. Besides this, the images also experience a very large intensity variation. To enhance the texture, we use the logarithmic intensity of original complex images, as shown in Figure 7(a) and (b). To achieve a more precise approximation to the real warping, we divide the image pair into four 500 × 500 patch pairs. The geometrical warping on each patch pair is approximated as an affine transformation (the higher order polynomial has also been used to model the warp function, but unsatisfactory registration result is attained). We adopt
|Parameters||Master image||Slave image|
|Azimuth resolution||0.1016 m||0.1016 m|
|Range resolution||0.1016 m||0.1016 m|
|Global track angle||158.3687°||153.0825°|
|Central frequency||16.8 GHz||16.8 GHz|
|Platform altitude||1.6715 km||1.6715 km|
To further evaluate the registration performance of the scheme, in the following we focus on the two pole-like target areas 1 and 2 in Figure 7(d) with their corresponding Google optical images shown in Figure 8(g) and (h), respectively. Figure 8(i) portrays the details of Pole 2 in the Street View of Google Maps. The target is shown to be the power transmission pole. Figure 8(a)–(c) exhibits the SAR imagery of Pole 1 in the master image, coregistered slave image, and overlapped image, respectively. The corresponding SAR imageries of Pole 2 are displayed in Figure 8(d)–(f), respectively. It is known that the darker pole-like feature in each SAR image is not the real pole scattering, but its shadow under the irradiation of radar. The actual scattering center of the pole is overlapped with its ground position because of the dominant dihedral backscattering between the pole and ground. From Figure 8(c) and (f), we can find that the shadows of the two poles are still separated after registration due to the volume-induced warping. According to our estimate, the separations are about 6.5 and 5°, respectively, which approach to the actual track angle 5.2862°. Except for these shadows, the poles and other area are accurately overlapped. Nice registration is still achieved despite the large local distortion and decorrelation. Moreover, the experiment also validates the strategy for general feature-based SAR image registration, i.e., to focus on the global registration and to neglect the local discontentment. The accurate registration of each pixel is impossible and unnecessary. It should be noted that the conventional SAR image registrations including the feature-based approaches focused in current chapter are mainly appropriate for images with approximated low-order polynomial geometrical warping. For SAR images taken from area of rough topography with long baseline, we need some more complex approaches with the
Nevertheless, by taking the squared Frobenius norm of matrix
we can then obtain the total power (also known as
SAR coherent imaging unavoidably brings about geometrical distortion and speckle into the acquired images and makes the registration of SAR images much more complicated. In this chapter, we focus on two important procedures in general feature-based SAR registration, i.e., the feature extraction and the parameter retrieval by identifying the appropriate feature and the appropriate estimation algorithm. As for the former, we conduct a detailed evaluation on the commonly used features such as tie points, Harris corner, SIFT, and SURF. We find that SURF outperforms others in terms of the geometrical invariance of feature, extraction speed, accuracy of localization, geometrical invariance of descriptor, matching speed, robustness to decorrelation, and flexibility to image speckling. Among these criteria, feature’s flexibility to speckle is particularly focused because speckle impacts the feature extraction and matching, while speckle filtering may change the feature position and impact the subpixel localization. The Fast-Hessian detector of SURF has a potential relation with the refined Lee speckle filter. SSP in SURF just indicates that we use a series of box filters of different size to filter speckles and extract features of different scales. Thus, SURF is very flexible to deal with SAR speckle. In view of the application with strict requirement for registration accuracy, we suggest using the SURF detector of
Parameter retrieval in SAR registration is difficult because spatial or temporal decorrelation will always introduce mismatches into the obtained feature correspondences. The estimator should be robust to outliers. We find that the commonly used RANSAC may trap into local occlusion and result in uncertain parameter retrieval. This uncertainty is more severe in SAR image registration because SAR geometrical warping varies from pixel to pixel, but the low-order polynomial approximation can only account for global registration instead of the local contentment. The local-to-global strategy in RANSAC may thus magnify the local distortion, aggravate the estimation uncertainty, and damnify the global registration accuracy although a largest CS is obtained. To achieve a stable registration for SAR images, we should estimate the parameters with more correspondences to reflect the true support than just a MSS, and apply an appropriate loss function. This leads us to EF-LTS, which improves Fast-LTS from 1D regression to 2D regression, and provides us an adaptive determination of the number of random sampling instead of setting it as a constant 500. EF-LTS conducts registration by LS fitting at least half of the correspondences to minimize the squared residual. It behaves very stable and is averagely better than RANSAC. Hence, we recommend conducting SAR image registration by fitting SURF features with EF-LTS. Experiments on both InSAR and MiniSAR image pairs validate the nice performance of this registration scheme.
This work is supported by China Manned Space Program along with the Youth Innovation Promotion Association, Chinese Academy of Sciences under Grant No. 2014131. The authors thank the International Society for Optics and Photonics (SPIE) for the permission to reuse materials that have appeared in Proceedings of SPIE (Li D, Zhang Y. On the appropriate feature for general SAR image registration; The appropriate parameter retrieval algorithm for feature-based SAR image registration. SAR Image Analysis Modeling and Techniques XII. Vol. 8536, 2012.)