Image Enhancement Methods for Remote Sensing: A Survey

Abstract

The quality of the images obtained from remote sensing devices is very important for many image processing applications. Most of the enhancement methods are based on histogram modification and transform based methods. Histogram modification based methods aim to modify the histogram of the input image to obtain a more uniform distribution. Transform based methods apply a certain transform to the input image and enhance the image in transform domain followed by the inverse transform. In this work, both histogram modification and transform domain methods have been considered, as well as hybrid methods. Moreover, a new hybrid algorithm is proposed for remote sensing image enhancement. Visual comparisons as well as quantitative comparisons have been carried out for different enhancement methods. For objective comparison quality metrics, namely Contrast Gain, Enhancement Measurement, Discrete Entropy and Average Mean Brightness Error have been used. The comparisons show that, the histogram modification methods have a better contrast improvement, while transform domain methods have a better performance in edge enhancement and color preservation. Moreover, hybrid methods which combine the two former approaches have higher potential.

1. Introduction

Widely used remote sensing applications, such as mapping, classification, soil moisture detection, target detection and tracking, etc. require high quality images. To meet the increasing need for higher quality images, image enhancement methods which improve the contrast and edge information of the input image are applied to the raw input images.

Images provided by remote sensing devices have to be enhanced by special methods instead of standard enhancement methods. Since applications like classification, target detection and target tracking are automated applications, the original reflectance values of the input image should be preserved as much as possible, which makes enhancing the remotely sensed image a challenging problem [1, 2]. Remote sensing image enhancement techniques should improve the visibility, contrast and edge information of the image while preserving the original reflectance values.

In recent years, many remote sensing image enhancement methods have been developed to increase the quality of these images. Image enhancement methods can be divided into two main groups as direct and indirect methods [3, 4, 5]. Direct methods aim to enhance the images by using a defined contrast measure [6, 7, 8, 9], while the indirect methods try to improve the dynamic range of the images without a contrast measurement [10, 11, 12, 13, 14, 15].

In direct methods, contrast measurements can be global or local. In general, local measurements have better results [9]. Dhnawan et al. [6] proposed a local contrast function based on the relative difference between a central region and a neighboring region for a given pixel. Beghdad and Negrate [7] introduced an improvement of [6] by defining the contrast with the consideration of edge information. Laxmikant Dash and Chatterji [8] proposed an adaptive contrast enhancement method where contrast amplification is based on the brightness estimated by local image statistics. Cheng and Xu [9] proposes a another adaptive enhancement method based on the fuzzy entropy principle and fuzzy set theory.

The direct methods have a low computational cost but accordingly show a poor image enhancement performance. The state of art methods are generally indirect methods which provide better enhancement performances compared to the direct methods. The indirect methods can be divided into two sub categories as histogram modification based methods [3, 4, 16, 17, 18, 19, 20, 21, 22] and transform domain methods [1, 2, 21, 23, 24, 25].

The simplest histogram modification method is Histogram Equalization (HE) [16]. In this method, the histogram distribution of the input image is aimed to have uniform distribution. This method is able to improve the contrast. However, the HE based enhanced images generally suffer from undersaturation or oversaturation, which results in poor quality images. To fix this problem, more efficient histogram modification methods have been proposed in recent years such as Bi-Histogram equalization (BHE) [17] based and Recursive Mean-Separate histogram equalization (RMSHE) [18]. In both methods, the original histogram of the input image is divided into sub-histograms. After obtaining the sub-histograms, separate histogram equalizations are applied to these sub-histograms. Finally, the divided histograms are merged to obtain the enhanced image [17, 18]. The images obtained by these methods have higher quality compared to the classical HE method, however the undersaturation and oversaturation problems are not resolved. 2-D histogram based methods have also been proposed for image enhancement [19, 20]. These methods provide better results than the methods aforementioned, however the computational cost of 2-D histogram creation is too high, which makes these methods not suitable for automated applications. Moreover, there are faster methods with higher enhancement performances. Another method proposed in this sub-category is Adaptive Gamma Correction with Weighting Distribution (AGCWD) method [4]. In this method, a weighted distribution of the original histogram of the input image is obtained followed by Gamma correction. The most important benefit of this method is its ability to preserve the original reflectance values which are needed for remotely sensed image enhancement, however this method too suffers from saturation artifacts. Moreover, the edge information is lost especially in the brighter regions [2, 21]. Histogram modification methods have a good performance if the histogram of the input image is smoother. Moreover, these group eliminate the lower-scale details [22]. The histogram modification methods have a higher performance for low resolution images and images containing larger-scale details.

Transform domain based image enhancement methods use certain transformations to decompose the image into subbands and improve the contrast by modifying specific components [1, 2, 23, 24, 25]. The first method in this category uses a combination of discrete wavelet transform and singular value decomposition (DWT-SVD) [23]. In DWT-SVD method, first discrete wavelet transform (DWT) is applied to both the input image and to the equalized input image by a general histogram equalization method. Since the details and edge information are kept in high pass sub bands, the method concentrates on the approximation sub bands. After obtaining both approximation sub bands, a Singular Value Decomposition (SVD) is applied to the approximation sub bands of the input and the equalized images. The singular value calculated from the input image is weighted by the singular value of the equalized image to obtain an enhanced singular value. Finally, inverse SVD followed by inverse DWT are applied to obtain the enhanced image. A more recent transform domain method uses the Bilateral Filtering (BF) for image enhancement [1]. The input image is decomposed into its approximation and detail layers by a multiscale BF. Finally, the obtained detail layers are added to the original image with a weighted manner to obtain edge enhanced image. Another method is the Remote Sensing Image Enhancement based on the hazy image model [2]. In this method, the commonly used hazy image model [26] is adapted for image enhancement applications. Here, the two unknown parameters of the hazy image model, namely airlight and transmission, are estimated with simple statistical properties of the input image to obtain the enhanced image. A more recent work is based on Robust Guided Filtering [24]. In this method, a robust guided filter described in [27] is applied to the input image and the difference between the original image and filtered image is considered as a detail sub-band as in DWT. The detail sub-bands are amplified and added to the original image to obtain the final enhanced image. Although they show a better performance, the methods in this group suffer from blocking artifacts or, in some cases, they are unable to enhance the image globally [22]. The overall performance of transform domain methods is better than the histogram modification methods. Moreover, the performance of this group of methods is significantly better for high resolution images and images containing both low and high scale details. There are hybrid methods combining histogram and transform methods. One hybrid method is based on a Regularized Histogram Equalization and Discrete Cosine Transform (RHE-DCT) [25]. In this technique, first a global enhancement is applied to the input image by a Regularized Histogram Equalization (RHE). Here, the equalization is made by using the sigmoid function. After obtaining the equalized image, Discrete Cosine Transform (DCT) is applied to the equalized image to obtain DCT coefficients. After this, the coefficients are modified to locally improve the contrast of the image. Finally, inverse DCT is applied to obtain the enhanced image.

In addition to all these methods, a hybrid algorithm combining [1] and HIM [2] methods has been proposed. In this proposed hybrid algorithm, the BF method described above is applied to the image to obtain a global enhanced image. Then, the HIM method is applied block by block to this globally enhanced image to obtain a local enhancement.

2. Remote sensing image enhancement methods

The quality of remote sensing images depends upon numerous factors such as noise, illumination or equipment conditions during the acquisition procedure [28]. The data obtained by optic sensors (multispectral, hyperspectral, panchromatic sensors) are degraded by atmospheric effects and instrumental noises, namely thermal (Johnson) noise, quantization noise and shot (photon) noise which cause corruption in the spectral bands by varying degrees [29]. On the other hand, SAR images (radar sensors), which offer many benefits such as working 7/24 and in all weather conditions, suffer from multiplicative speckle noise [28].

These degradations reduce the contrast in the resulting images and can highly affect human perception or the accuracy of computer assisted applications [25]. Thus, contrast enhancement, besides noise removal, constitutes a primary step for various applications of remote sensing image processing for better information representation and visual perception.

2.1 Adaptive gamma correction with weighting distribution (AGCWD)

In this method, a weighting distribution of the original histogram of the input image is obtained followed by Gamma correction.

First an Adaptive Gamma correction is made to the input image as:

$$T\left(l\right)=l_{\mathit{max}}{\left(\frac{l}{l_{\mathit{max}}}\right)}^{\gamma }=l_{\mathit{max}}{\left(\frac{l}{l_{\mathit{max}}}\right)}^{1-F\left(l\right)}$$E1

Here, $l$ is the intensity value of the current pixel and $l_{\mathit{max}}$ is the maximum intensity value of the input image. $\gamma$ is a varying adaptive parameter which is equal to $1-F\left(l\right)$, and $F\left(l\right)$ is the cumulative distribution function. The reason to use cumulative distribution function for adaptive Gamma correction is to guarantee the Gamma parameter to follow the changes between the pixels of the image.

In order to avoid the adverse effects, a weighting distribution function is used so as to slightly modify the histogram as follows:

$$f\left(l\right)=f_{\mathit{max}}{\left(\frac{f\left(l\right)-f_{\mathit{min}}}{f_{\mathit{max}}-f_{\mathit{min}}}\right)}^{\alpha }$$E2

Here, $\alpha$ is the adjustment parameter, $f$ is the probability density function and $f_{\mathit{max}}$ and $f_{\mathit{min}}$ are the maximum and minimum of the $f$. Using (2), the modified cumulative distribution function $F$ is evaluated by:

$$F_{\omega }\left(k\right)=\frac{{\sum }_{l=0}^k{f}_{\omega }\left(l\right)}{\sum f_{\omega }}$$E3

where

$$\sum f_{\omega }=\sum _{l=0}^{l_{\mathit{max}}}{f}_{\omega }\left(l\right)$$E4

Finally, the Gamma parameter of (1) is modified as:

$$\gamma =1-F_{\omega }\left(l\right)$$E5

The modified Gamma parameter and Eq. (1) is used to obtain the enhanced image.

2.2 Discrete wavelet transform and singular value decomposition based method (DWT-SVD)

In this method, a combination of discrete wavelet transform (DWT) and singular value decomposition (SVD) are used for enhancement purposes. In the classical one-dimensional (1D) DWT, the input signal is decomposed into its low ($L$) and high ($H$) frequency components. In order to perform a two-dimensional (2D) transform, the 1D DWT is applied to the row of the images followed by the columns of the image, or vice versa. After applying the 2D DWT, four different subbands are obtained, namely $\mathit{LL}$, $\mathit{LH}$, $\mathit{HL}$, and $\mathit{HH}$. The approximation subband $\mathit{LL}$ contains low frequency components, while the diagonal subband $\mathit{HH}$ contains high frequency components for both rows and columns of the image. The horizontal and vertical subbands $\mathit{LH}$ and $\mathit{HL}$ contains low frequency component for the rows and high frequency components for the columns and vice versa, respectively.

SVD is used to decompose a matrix into two orthogonal square matrices ($U$ and $V$) and a diagonal matrix containing the singular values $\left(\mathrm{\Sigma }\right)$ as shown:

$$I=U_I{\mathrm{\Sigma }}_I{V}_I^T$$E6

The enhancement method firstly applies a general histogram equalization to the input image $I$ to obtain equalized image $\tilde{I}$. Then discrete wavelet transform is applied to both the input and equalized images so as to obtain the subbands ${\mathit{LL}}_I$, ${\mathit{LH}}_I$, ${\mathit{HL}}_I$, ${\mathit{HH}}_I$ and ${\mathit{LL}}_{\tilde{I}}$, ${\mathit{LH}}_{\tilde{I}}$, ${\mathit{HL}}_{\tilde{I}}$, ${\mathit{HH}}_{\tilde{I}}$, respectively.

Since the rough information about the images are present in the $\mathit{LL}$ subbands, SVD is applied to these subbands to obtain the singular values. As aforementioned, the singular values contain the intensity of the image. Therefore, equalization is made for the singular values. Here, the $\mathrm{\Sigma }$ components of the ${\mathit{LL}}_I$ and ${\mathit{LL}}_{\tilde{I}}$ is weighted as to obtain a correction coefficient ($\xi$).

$$\xi =\frac{\mathit{max}\left({\mathrm{\Sigma }}_{{\mathit{LL}}_{\tilde{I}}}\right)}{\mathit{max}\left({\mathrm{\Sigma }}_{{\mathit{LL}}_I}\right)}$$E7

where ${\mathrm{\Sigma }}_{{\mathit{LL}}_{\tilde{I}}}$ is the singular value matrix of the equalized image derived from its ${\mathit{LL}}_{\tilde{I}}$ subband and ${\mathrm{\Sigma }}_{{\mathit{LL}}_I}$ is the singular value matrix of the input image obtained from its ${\mathit{LL}}_I$ subband. After determining the correction coefficient, the corrected singular value matrix $\mathrm{\Sigma }$ is obtained as:

$$\mathrm{\Sigma }=\xi {\mathrm{\Sigma }}_{{\mathit{LL}}_I}$$E8

Here, $\mathrm{\Sigma }$ is the corrected singular value matrix. The new $\mathit{LL}$ subband is constructed as:

$$\mathit{LL}=U_{{\mathit{LL}}_I}\mathrm{\Sigma }{V}_{{\mathit{LL}}_I}^T$$E9

After constructing the new $\mathit{LL}$ subband, the enhanced image is obtain by performing the inverse DWT to this new $\mathit{LL}$ subband and detail subbands of the original image.

2.3 Regularized histogram equalization and discrete cosine transform based method (RHE-DCT)

This method basically consists of two steps: Regularized Histogram Equalization (RHE) followed by Discrete Cosine Transform (DCT). The first one performs a global contrast enhancement and the second one enhances the local contrast.

RHE aims to perform a histogram equalization to the input image by a regularized manner as:

$$f\left(k\right)=s\left(k\right)\left(1+h\left(k\right)\right)$$E10

Here $f\left(k\right)$ is the probability density function of the equalized histogram, $h\left(k\right)$ is the normalized histogram of the input image, $s\left(k\right)$ is the sigmoid function defined as:

$$s\left(k\right)=\frac{1}{1+e^{-\left(k-1\right)}}-\frac{1}{2}$$E11

By this modification, the minimum value of the equalized image is assured to be equal to $0$. The $f\left(k\right)$ obtained is normalized as:

$$f\left(k\right)\leftarrow \frac{f\left(k\right)}{{\sum }_{t=1}^{K}f\left(t\right)}$$E12

Here $K$ is the number of the gray levels. The cumulative distribution function $F\left(k\right)$ is obtained as:

$$F\left(k\right)=\sum _{t=1}^{K}f\left(t\right)$$E13

and new gray levels are evaluated as:

$$y\left(k\right)=\left[F\left(k\right)\left(y_{\mathit{max}}-y_{\mathit{min}}\right)+y_{\mathit{min}}\right]$$E14

Finally the equalized image is obtained by using a standard lookup table based HE procedure to obtain $Y_{\mathit{eq}}$.

In order to perform a local enhancement, the DCT coefficients of the globally equalized image is used. For this purpose, first the DCT is applied to the equalized image as:

$$C\left(u,v\right)=c_h{c}_{\omega }\sum _{k=0}^{M-1}\sum _{l=0}^{N-1}{Y}_{\mathit{eq}}\left(k,l\right)\mathit{cos}\left[\frac{\left(2k+1\right)\mathit{h\pi }}{2M}\right]\mathit{cos}\left[\frac{\left(2l+1\right)\mathit{\omega \pi }}{2N}\right]$$E15

$c_h$ and $c_{\omega }$ are computed by:

$$c_h=\left\{\begin{array}{cc}\sqrt{\frac{1}{M}},& h=0\\ \sqrt{\frac{2}{M}},& 1\le h\le M-1\\ & \end{array}\right.$$E16

$$c_{\omega }=\left\{\begin{array}{cc}\sqrt{\frac{1}{N}},& \omega =0\\ \sqrt{\frac{2}{N}},& 1\le \omega \le N-1\\ & \end{array}\right.$$E17

The lower absolute values of $C$ should be adjusted to perform local enhancement while higher values should be maintained to avoid drastic changes. By this way new DCT coefficients are obtained as:

$$D^{\prime }\left(h,w\right)=\left\{\begin{array}{cc}D\left(h,w\right),& D\left(h,w\right)>0.01D\left(0,0\right)\\ \mathit{\alpha D}\left(h,w\right),& D\left(h,w\right)\le 0.01D\left(0,0\right)\end{array}\right.$$E18

Here $\alpha$ is the adjustment parameter and is automatically determined as:

$$\alpha =1+\sqrt{\left(\mathit{std}\left(Y_{\mathit{\text{global}}}\right)\right)-\mathit{std}\left(X\right)/\left(2^B-1\right)}$$E19

After obtaining the new DCT coefficients, inverse DCT is applied to obtain the final enhanced image.

2.4 Bilateral filtering based method (BF)

This method is basically based on multiscale bilateral filtering. In classical bilateral filtering, the filter output can be determined as:

$$\mathrm{BF}\left[I\right]=\left(\frac{1}{W}\right)\sum _{q\in s}{G}_{{\sigma }_s}\left(‖p-q‖\right)G_{{\sigma }_r}\left(‖I_p-I_q‖\right)I_q$$E20

where

$$W=\sum _{q\in s}{G}_{{\sigma }_s}\left(‖p-q‖\right)G_{{\sigma }_r}\left(‖I_p-I_q‖\right)$$E21

Here ${\sigma }_s$ and ${\sigma }_r$ are the Gaussian kernels controlling the spatial and range of the input image. $I_p$ is the intensity value of the pixel at location $p$, $I_q$ is the intensity value of the neighboring pixels within the window $S$ at location $q$. The difference between the input image and the filter output gives the detail layer of the image.

$$D^1=I-\mathrm{BF}\left[I\right]$$E22

Here, $D^1$ is the first detail layer of the image. In order to carry on the decomposition, bilateral filtering is applied again to the filter output. Here, to guarantee the shift invariance, ${\sigma }_s$ is doubled and ${\sigma }_r$ is halved. In order to obtain level detail layer, two adjacent filter outputs are subtracted as:

$$D^j={\mathrm{BF}}^j\left[I\right]-{\mathrm{BF}}^{j-1}\left[I\right]$$E23

Here, $j$ corresponds to the decomposition level.

In order to reconstruct the input image from an $L$ levels of decomposition, one can simply add all detail layers to the final filtering output as:

$$I=\sum _{j=1}^L{D}^j\left[I\right]+{\mathrm{BF}}^L\left[I\right]$$E24

Bilateral filtering based method firstly decomposes the input image by (24).

After obtaining the detail layers for L levels. The details are amplified and added directly to the original image as:

$$I_E={\mathrm{BF}}^L\left[I\right]+\sum _{j=1}^L{\omega }_j{D}^j\left[I\right]$$E25

Here, $I_E$ is the enhanced image and ${\omega }_j$ are the weighting factors for the corresponding detail subbands $D^j\left[I\right]$.

The parameter determination is very important in order to achieve a good enhancement result. Therefore, ${\sigma }_r$, ${\sigma }_s$, and S parameters of the bilateral filter, as well as the decomposition level and weights have to be determined. To achieve this, a comparison between the enhancement results obtained by differing parameter are made. As a result of this comparison, ${\sigma }_r$ is chosen as 0.6, ${\sigma }_s$ is chosen as 1.8, $S$ is chosen as a window sized $5\times 5$, the decomposition level is chosen as $4$, and the weights (${\omega }_1$, ${\omega }_2$, ${\omega }_3$, ${\omega }_4$) are chosen as $2$ [1].

2.5 Adaptive cuckoo search based enhancement algorithm (ACSEA)

In this method, the image enhancement is performed by optimizing a predefined enhancement kernel [30]. The enhancement process of ACSEA is given below:

$$I_{E_{\left(i,j\right)}}={\left({\mu }_{\left(i,j\right)}^L\right)}^{\alpha }+F_{\left(i,j\right)}^e\left(I_{i,j}-c_{{\mu }_{\left(i,j\right)}}^L\right)$$E26

where

$$F_{\left(i,j\right)}^e=k\frac{{\mu }^G}{{\sigma }_{\left(i,j\right)}^L+b}$$E27

Here, $F_{(}\left(i,j\right)){}^e$ is calculated by the mean value and standard deviation of the image and called as the image enhancement function. $\left(i,j\right)$ is the location of the current pixel. ${\sigma }_{(}\left(i,j\right)){}^L$ is the local standard deviation and ${\mu }_{(}\left(i,j\right)){}^L$ is the local mean value calculated in a window sized $N\times N$ centered at $\left(i,j\right)$, while ${\mu }^G$ is the global mean value. The method focuses on optimizing the parameters $\left(a,b,c,k\right)$, where $0\le a\le 1.5$, $0\le b\le 0.5$, $0\le c\le 1$, and $0.5\le k\le 1.5$.

In order to optimize the enhancement formula given in (26), a chaotic initialization is made and an objective fitness function is used as given below:

$$F\left(I_E\right)=log\left(log\left(E\left(I_E^8\right)\right)+e\right)\frac{N_e\left(I_E^8\right)}{\mathit{MN}}{e}^{H\left(0\right)}$$E28

where

$$I_E^8=\sqrt{{\left({\nabla }_x{I}_E\right)}^2+{\left({\nabla }_y{I}_E\right)}^2}$$E29

In (28), $E\left(.\right)$ is the expected value operator and $H\left(.\right)$ is the entropy operator. In (29), ${\nabla }_x$ and ${\nabla }_y$ are the gradients. $I_E^8$ is the Sobel edge detected image.

In order to optimize the enhanced image $\left(I_E\right)$, the objective function given in (29) is optimized with a chaotic initialization so as to obtain the best enhancement result.

2.6 Hazy image model based enhancement (HIM)

This method is based on the commonly used hazy image model [26, 31].

$$I=\mathit{Jt}+A\left(1-t\right)$$E30

where $I$ is the input image, $A$ is the airlight coefficient, t is the transmission map and $J$ is the haze free image. In order to obtain haze free image $J$, $A$ and $t$ have to be estimated.

For dehazing purposes airlight coefficient is generally estimated from the brighter pixels of the input image. For enhancement, instead of the brighter pixels the mean of the image is assumed to be the airlight coefficient [2].

$$A=1/\mathit{KL}\sum _{k=1}^K\sum _{l=1}^{L}I\left(k,l\right)$$E31

$I$ is the input image with dimensions of $K\times L$ and $I\left(k,l\right)$ is the intensity value of the pixel at location $\left(k,l\right)$. In general, in dehazing algorithms, the transmission map is estimated using the airlight coefficient and normalized input image. The normalized image is obtained by estimated airlight coefficient. Following the similar manner, this method also normalizes the image with the estimated airlight and estimate the transformation as:

$$t=1-\omega \left(\frac{1}{A}\right)$$E32

Here, $\omega$ is an arbitrary coefficient. The coefficient can be determined as the standard deviation $\left(\sigma \right)$ of the input image [2]. Finally, the enhanced image is obtained by simply taking out $J$ out of (30) as:

$$J=\frac{I-A\left(1-t\right)}{t}$$E33

2.7 Robust guided filtering based method (SDF)

This method uses the Robust Guided Filtering described in [27] which uses two guidance images namely dynamic guidance and static guidance. In order to perform Robust Guided Filtering, the following cost function should be minimized:

$${\in }_u=\sum _i{c}_i{\left(u_i-f_i\right)}^2+\lambda \mathrm{\Omega }\left(u,g\right)$$E34

Here, $f$ is the input image, $u$ is the dynamic guidance and $g$ is the static guidance. $\lambda$ is the regularization parameter and $c_i\le 0$ is the confidence level. The regularizer $\mathrm{\Omega }\left(u,g\right)$ can be defined as [27].

$$\mathrm{\Omega }\left(u,g\right)=\sum _{i,j\in N}{\varphi }_{\mu }\left(g_i-g_j\right){\phi }_v\left(u_i-u_j\right)$$E35

where

$${\phi }_v\left(x\right)=\frac{1-{\phi }_v\left(x\right)}{v}\text{and}{\varphi }_{\mu }\left(x\right)=e^{-\mu x^2}$$E36

$N$ is the neighborhood size which is $8\times 8$, while $\mu$ and $v$ are parameters controlling the smoothness level.

In order to perform image enhancement, a multi-scale decomposition based on Robust Guided Filtering similar to the multi-scale bilateral filtering is proposed in [1]. The filtering output is considered as the first approximation layer of the original image as:

$$A_1=\mathrm{SDF}\left[I\right]$$E37

Here, $I$ is the input image, $A_1$ is the first level approximation layer and SDF operator stands for Robust Guided Filtering. In order to obtain further levels of approximation layers, SDF is applied to previous approximation layer as:

$$A_l=\mathrm{SDF}\left[A_{l-1}\right]$$E38

with initial value $A_1=I$. The difference between two adjacent approximation layers give the detail layer of the corresponding level as:

$$D_l=A_l-A_{l-1}$$E39

One can obtain the original image by simply adding the detail layers to the final level approximation layer.

$$I=\sum _{j=1}^L{D}_j+A_L$$E40

SDF based enhancement firstly decomposes the input image by using (40).

After obtaining the detail layers, the details are amplified and added directly to the original image as:

$$I=\sum _{j=1}^L{\omega }_j{D}_j+A_L$$E41

The decomposition level and weights are determined by comparing different number of levels and weights. The best results for different images are applied for all images. Therefore, the decomposition level is chosen as $4$. Moreover, the weights $\left({\omega }_1,{\omega }_2,{\omega }_3,{\omega }_4\right)$ are chosen as $2$ [24].

2.8 Hybrid bilateral filtering and hazy image model method (BF-HIM)

The BF based enhancement method [1] has a good enhancement, however the color distortion is present, whereas the HIM method [2] has a good color preservation with a lower enhancement performance. Therefore, a hybrid method combining these two methods can be a good candidate to obtain a good performance for enhancement along with a good color preservation.

The hybrid method first applies the multi-scale bilateral filtering given in (24) to the input image to obtain the bilateral filtering outputs and detail layers. Since, we will add the HIM model, the decomposition level is chosen as 2. Then, the detail layers are amplified as given in (25) to obtain the prior enhancement result. The prior enhanced image is divided into non overlapping blocks. HIM method given above is applied to these blocks separately to perform a local enhancement. Finally, the enhanced blocks are combined to construct the final enhancement result.

Here, the choice of the block size is important. The lower block size is expected to have a better local enhancement result. Therefore, the block size is chosen as $3\times 3$.

3. Evaluation criteria

It is possible to determine the performance of an image enhancement method visually. However, a visual conclusion may not be objective. Therefore, in order to make objective comparisons, evaluation criteria has been developed. Here, the choice of criteria is also important. It is already known that every criterion can give an idea about one property of the resulting image. Therefore, criteria for different properties of the image should be used. Moreover, since each criterion gives an idea for a certain property of the image, all criteria should be considered together to have an overall idea of the image. The criteria presented below gives an idea for the performance of the enhancement methods, however all should be considered together and along with visual results.

3.1 Contrast gain (CG)

The first criteria to measure the performance of enhancement method is Contrast Gain (CG) [32]. This criterion focuses on the contrast improvement of the image as follows:

$$\mathrm{CG}=\frac{C\left(Y\right)}{C\left(X\right)}$$E42

where $C$ is average of the local Michelson contrast, which is calculated for 3x3 sized windows within an image and given as:

$$C=\frac{max-min}{max+min}$$E43

The higher CG value indicates that the contrast improvement is better.

3.2 Enhancement measurement (EME)

This criterion also considers the contrast improvement within the enhanced image and defined by following [22]:

$${\mathrm{EME}}_{\alpha ,k_1,k_2}\left(\phi \right)=\frac{1}{k_1{k}_2}\sum _{l=1}^{k_1}\sum _{k=1}^{k_2}\alpha {\left(\frac{l_{\max }^{k,l}\left(\phi \right)}{l_{\min }^{k,l}\left(\phi \right)+c}\right)}^{\alpha }ln\frac{l_{\max }^{k,l}\left(\phi \right)}{l_{\min }^{k,l}\left(\phi \right)+c}$$E44

Here the image $I$ is split into $k_1\times k_2$ sized blocks. $I_{\max }^{(}k,l)$ and $I_{\min }^{(}k,l)$ are the maximum and minimum values within the block, while $c$ is a small constant to avoid division by zero. ${\mathrm{EME}}_{\alpha ,k_1,k_2}\left(\phi \right)$ is called the Enhancement Measurement of Entropy with respect to transform $\phi$.

The higher EME value indicates that the contrast improvement is better.

3.3 Discrete entropy (DE)

Discrete entropy of an image can be evaluated as:

$$\mathrm{DE}=-\sum _{k=1}^{K}p\left(x_k\right)logp\left(x_k\right)$$E45

Here, $p\left(x_k\right)$ is the probability of the pixel $\left(x_k\right)$. The higher value of DE indicates that a smoother distributed histogram is obtained, which may indicate that the contrast is higher.

3.4 Absolute mean brightness error (AMBE)

Absolute Mean Brightness Error (AMBE) [18] is an error function calculated between image $X$ and image $Y$ as:

$$\text{AMBE}=\frac{1}{\mathit{MN}}\sum _{m=1}^M\sum _{n=1}^N\mid X\left(m,n\right)-Y\left(m,n\right)\mid$$E46

Here, $M$ and $N$ are the dimensions of the images and $\left(m,n\right)$ is the pixel location.

The lower AMBE value indicates that, the brightness preservation is better.

4. Experimental setup

The enhancement described above have been applied to several images. Comparisons of the methods are made both visually and quantitatively. Before applying the enhancement methods, the parameters for each method is determined.

4.1 Visual comparison

Visual comparisons are performed for different images and they are available online.¹

The first image used for comparison is a tank image taken by a digital imaging system as shown in Figure 1(a). Figure 1(b)–(i) show the enhancement results obtained by AGCWD, DWT-SVD, RHE-DCT, ACSEA, BF, HIM, SDF, and BF-HIM methods, respectively. In order to demonstrate closely, the zoomed version of the area inside the red square is given inside the green square. As seen in Figure 1(b), AGCWD method has improved the contrast, however the color preservation of the method is not good. The contrast improvement of DWT-SVD seems to be low, as seen in Figure 1(c). Even though the color preservation seems to be good, the edge information is lost as seen in the zoomed area. RHE-DCT method, shown in Figure 1(d), has a good contrast improvement, however the color preservation is not good. The edge enhancement of RHE-DCT is better than AGCWD and DWT-SVD methods. ACSEA method in Figure 1(e) demonstrates a better color preservation, however the contrast improvement is not as good as the other methods. Moreover, the edge enhancement is lower than RHE-DCT method. As seen in Figure 1(f), BF method preserves the color like as the ACSEA method and has a good edge enhancement performance. Figure 1(g) shows that HIM method has a good color preservation capability. However, the edge enhancement performance is not good compared to the BF method. SDF method, given in Figure 1(h) has a very good edge enhancement performance, but the color preservation is lower than ACSEA, HIM and BF methods. As demonstrated in Figure 1(i), the hybrid BF-HIM method preserve the colors closer to the ACSEA and BF methods and enhances the edge information better than the former methods.

Figure 1.

(a) Input image, enhancement results for (b) AGCWD (c) DWT-SVD, (d) RHE-DCT, (e) ACSEA, (f) BF, (g) HIM, (h) SDF, and (i) BF-HIM methods.

The second image used for comparison is an aerial image taken by a digital imaging system mounted on an air vehicle as shown in Figure 2(a). Figure 2(b)–(i) show the enhancement results obtained by AGCWD, DWT-SVD, RHE-DCT, ACSEA, BF, HIM, SDF, and BF-HIM methods, respectively. In order to make a closer look, a zoomed version of the area inside the red square is given in the green square.

Figure 2.

(a) Input image, enhancement results for (b) AGCWD (c) DWT-SVD, (d) RHE-DCT, (e) ACSEA, (f) BF, (g) HIM, (h) SDF, and (i) BF-HIM methods.

As seen in Figure 2(b), AGCWD method has improved the contrast, however the color preservation of the method is not good. The car within the zoom area is visible. The contrast improvement of DWT-SVD is better than AGCWD method, as seen in Figure 2(c). The color preservation is lower than AGCWD method and the visibility of the car in the zoomed area is not as good as AGCWD method. RHE-DCT method, shown in Figure 2(d), has a good contrast improvement and better color preservation than AGCWD and DWT-SVD methods. The edge enhancement of RHE-DCT is closer to the AGCWD method as seen in the zoomed area. ACSEA method in Figure 2(e) demonstrates a better color preservation, however the contrast improvement is not as good as the other methods. Moreover, the edge enhancement is lower than RHE-DCT method. As seen in Figure 2(f), BF method preserves the color like as the ACSEA method and has a better edge enhancement performance than RHE-DCT methods, as seen in the zoomed area. Figure 2(g) shows that HIM method has a good color preservation capability. However, the edge enhancement performance is not good compared to the BF method. SDF method, given in Figure 2(h) has a very good edge enhancement performance, but the color preservation is lower than ACSEA, HIM and BF methods. As demonstrated in Figure 2(i), the hybrid BF-HIM method preserve the colors closer to the HIM method. A closer look demonstrates that the edge improvement is better than the former methods.

The final image used for comparison is an aerial image of an area containing harbor and airport taken by a digital imaging system mounted on an air vehicle as shown in Figure 3(a). Figure 3(b)–(i) show the enhancement results obtained by AGCWD, DWT-SVD, RHE-DCT, ACSEA, BF, HIM, SDF, and BF-HIM methods, respectively. For a closer look, the area shown in red square is zoomed and given within the green square.

Figure 3.

(a) Input image, enhancement results for (b) AGCWD (c) DWT-SVD, (d) RHE-DCT, (e) ACSEA, (f) BF, (g) HIM, (h) SDF, and (i) BF-HIM methods.

As seen in Figure 3(b), AGCWD method has improved the contrast, however the color preservation of the method is not good. Moreover, the edge information is lost as seen in the zoomed area. The contrast improvement of DWT-SVD seems to be low, as seen in Figure 3(c). Even though the color preservation seems to be good, the edges have not been improved as seen in the zoomed area. RHE-DCT method, shown in Figure 3(d), has a good contrast improvement, and the color preservation is good. The edge enhancement of RHE-DCT is better than AGCWD and DWT-SVD methods. ACSEA method in Figure 3(e) demonstrates a good color preservation, and a fine contrast improvement. Moreover, the edge improvement seems to be better than RHE-DCT method. As seen in Figure 3(f), BF method preserves the color better than ACSEA method and has a good edge enhancement performance. Figure 3(g) shows that HIM method has a good color preservation capability. However, the edge enhancement performance is not good enough compared to the BF method. SDF method, given in Figure 3(h) has a very good edge enhancement performance, but the color preservation is lower than ACSEA, HIM and BF methods. As demonstrated in Figure 3(i), the hybrid BF-HIM method preserve the colors closer to the ACSEA methods and enhances the edge information better than the former methods.

For an objective visual evaluation, the profiles of the horizontal lines given in Figure 1(a), Figure 2(a) and Figure 3(a) are constructed for the enhancement methods, and the drawn profiles for the original image are given along with enhancement methods are given in Figure 4(a)–(c), respectively.

Figure 4.

Drawn profiles for input and enhanced images for (a) Figure 1, (b) Figure 2, (c) Figure 3.

According to Figure 4(a), DWT-SVD and ACSEA methods cannot follow the changes which means the details of the image are lost for these methods. BF-HIM method can follow the changes better. Moreover, BF-HIM method have increased the intensity range more compared to the other methods, which indicates that the contrast improvement is better for BF-HIM method.

According to Figure 4(b), all three methods seem to follow the pattern of the original image properly, in general. ACSEA method has lost the pattern in some parts. SDF and BF-HIM methods have followed the pattern better than ACSEA method. Moreover, BF-HIM seems to have a slightly wider range than SDF method as well.

According to Figure 4(c), all three methods seem to follow the pattern of the original image properly. AGCWD method have increased the intensity values in general, which results in a brighter region. By this way, the contrast improvement is not good enough. Similarly, HIM method have decreased the intensity values, which results in a darker region. BF-HIM method has improved the contrast better than the other methods.

Therefore, according to the visual comparisons, the higher the detail level is within the image, the better the results for methods like AGCWD and RHE-DCT are, as expected, since both methods use histogram modification.

It can also be concluded that histogram modification methods like AGCWD and RHE-DCT methods have a good performance, if the resolution is low (Figure 1) or/and the input image contains higher-scale edge information (Figure 2), while transform domain methods are generally better for high resolution images or/and images containing small-scale details. Also, transform domain methods seem to have a solid performance for high-scale details.

In addition to this, considering all aspects of the resulting images, in terms of color preservation, contrast improvement, and edge enhancement, the BF, and hybrid BF-HIM methods seem to have better results. Moreover, hybrid BF-HIM method seems to be the best method when looking at all three aspects.

4.2 Quantitative comparison

In order to perform an objective comparison, the criteria aforementioned are evaluated for enhancement results obtained by the methods, the visual results of which are given in Figures 1–3. The quantitative results are provided in Tables 1–4 where the best results are emphasized in bold. The first criterion used for comparison is the Contrast Gain (CG). Table 1 shows the CG values obtained for AGCWD, DWT-SVD, RHE-DCT, ACSEA, BF, HIM, SDF, and BF-HIM methods.

Method	AGCWD	DWT-SVD	RHE-DCT	ACSEA	BF	HIM	SDF	BF-HIM
Figure 1	1.0600	0.7468	1.6442	1.4369	1.9099	1.3171	2.2560	2.0457
Figure 2	1.5302	1.1635	2.9214	1.6665	1.8910	1.6529	2.0120	2.0517
Figure 3	1.0555	0.5505	1.3110	1.7592	1.9522	1.1654	2.0217	2.0789

Table 1.

CG values obtained for the enhancement methods.

According to Table 1, for Figure 1, the best score is obtained by SDF followed by BF-HIM method. For Figure 2, the best score is obtained by RHE-DCT method followed by BF-HIM method. For Figure 3 is achieved by hybrid BF-HIM method, followed by SDF method. Therefore, it is possible to say that RHE-DCT method has a better contrast gain for images containing high-scale details like Figure 3.

The second criterion used for comparison is the Enhancement Measurement (EME). Table 2 shows the EME values obtained for AGCWD, DWT-SVD, RHE-DCT, ACSEA, BF, HIM, SDF, and BF-HIM methods.

Method	AGCWD	DWT-SVD	RHE-DCT	ACSEA	BF	HIM	SDF	BF-HIM
Figure 1	1.47	1.91	327.05	4.11	329.11	6.00	384.70	407.07
Figure 2	1.29	7.92	5.86	7.44	1.81	1.66	4.94	6.95
Figure 3	2.59	5.01	4.41	1.35	6.91	2.00	7.98	9.29

Table 2.

EME values (${10}^4$) obtained for the enhancement methods.

According to Table 2, for Figure 1 the best EME scores are obtained by BF-HIM method, followed by SDF method. For Figure 2, the best score is obtained by DWT-SVD method followed by ACSEA method. For Figure 3, the best EME scores are obtained by BF-HIM method for followed by SDF method. Therefore, it is possible to say that DWT-SVD method has a better enhancement performance for images containing high-scale details, likeas in Figure 2.

The third criterion used for comparison is the Discrete Entropy (DE). Table 3 shows the DE values obtained for AGCWD, DWT-SVD, RHE-DCT, ACSEA, BF, HIM, SDF, and BF-HIM methods.

Method	AGCWD	DWT-SVD	RHE-DCT	ACSEA	BF	HIM	SDF	BF-HIM
Figure 1	6.5601	6.8245	7.8527	7.3600	7.3778	6.4493	7.6648	7.6723
Figure 2	6.6672	6.8799	7.6940	7.0889	7.1340	6.7327	7.4061	7.2010
Figure 3	5.9251	5.7998	6.3719	6.4604	6.4508	5.9944	6.6397	6.6682

Table 3.

DE values obtained for the enhancement methods.

According to Table 3, for Figure 1, the best score is obtained by RHE-DCT method followed by BF-HIM method. For Figure 2, the best score id obtained by RHE-DCT method followed by SDF method. As it is seen in DE values, the higher the scale of detail is within the images, the higher is the performance of RHE-DCT method. BF-HIM method has better DE values for Figure 3, and has a close score to RHE-DCT method for Figures 1 and 2.

The fourth criterion used for comparison is the Absolute Mean Brightness Error (AMBE). Table 4 shows the AMBE values obtained for AGCWD, DWT-SVD, RHE-DCT, ACSEA, BF, HIM, SDF, and BF-HIM methods.

Method	AGCWD	DWT-SVD	RHE-DCT	ACSEA	BF	HIM	SDF	BF-HIM
Figure 1	0.2798	0.0657	0.1871	0.0149	0.0652	0.1014	0.0692	0.0459
Figure 2	0.0904	0.1083	0.1145	0.1503	0.0364	0.1265	0.0355	0.0254
Figure 3	0.1121	0.0331	0.0811	0.0629	0.0713	0.0651	0.0697	0.0326

Table 4.

AMBE values obtained for the enhancement methods.

According to Table 4, for Figure 1, the best score is obtained by ACSEA method followed by BF-HIM method. For Figure 2, the best score is obtained by BF-HIM method followed by BF method. For Figure 3, the best value is obtained by BF-HIM method followed by DWT-SVD method. Since, AMBE is the error between the original image and enhanced image, the smaller value of AMBE indicates better color preservation. This criterion does not give an idea about enhancement performance.

As a result, even though the visual comparison may give the observer an idea about the enhancement performance, quantitative comparison has to be made to obtain a more objective conclusion. Here, the choice of the quantitative criterion is also important. As it is known, each criteria indicates different aspects for the resulting images. For instance, CG gives an idea about the contrast improvement, while AMBE is about the color preservation. If the aim is to compare the overall performance for the methods aforementioned, all criteria should be considered all together. Thus, the quantitative comparisons, as well as the visual comparisons demonstrate that the hybrid methods combining different methods like BF-HIM result in better enhanced images.

5. Conclusion

The use of image enhancement methods which improve the contrast and edge information of the image is vital for remote sensing applications. In this work, different remote sensing image enhancement methods based on histogram modification techniques (HE, AGCWD) and transform domain methods (DWT-SVD, ACSEA, RHE-DCT, BF, HIM, and SDF) have been reviewed. The resulting images have been compared visually and quantitatively. For quantitative comparison, several image quality criteria have been used. The resolution and the detail scales of the image affects the performance of the enhancement methods. For instance, the detail scales of the input image affect the performance of RHE-DCT and AGCWD methods deeply. Since both methods are histogram modification methods, even though RHE-DCT also uses a transformation, it can be concluded that histogram modification based methods are better if there are higher-scale details within the image or if the image has a lower resolution. The transform domain methods have a better performance for the images with low-scale details, but also the results of these methods are very solid compared to the histogram based methods for high-scale details, as well.

Another contribution of this work is to introduce a hybrid method, which combines the bilateral filtering with hazy image model. The visual and quantitative results demonstrate that using hybrid methods have a superior performance to the methods applied separately. Therefore, future research on remote sensing image enhancement should focus on hybrid methods.