RMSE evaluating performances with different values of SNR, with three constructed graphs and optimal regularized parameters. The values of threshold and
Abstract
This chapter presents graph construction for hyperspectral data and associated unmixing methods based on graph regularization. Graph is a ubiquitous mathematical tool for modeling relations between objects under study. In the context of hyperspectral image analysis, constructing graphs can be useful to relate pixels in order to perform corporative analysis instead of analyzing each pixel individually. In this chapter, we review fundamental elements of graphs and present different ways to construct graphs in both spatial and spectral senses for hyperspectral images. By incorporating a graph regularization, we then formulate a general hyperspectral unmixing problem that can be important for applications such as remote sensing and environment monitoring. Alternating direction method of multipliers (ADMM) is also presented as a generic tool for solving the formulated unmixing problems. Experiments validate the proposed scheme with both synthetic data and real remote sensing data.
Keywords
- hyperspectral imaging
- graph construction
- spectral unmixing
- graph regularization
- spectral-spatial correlation
1. Introduction
Hyperspectral imaging analysis has found a wide range of applications including agricultural monitoring, environment detection, meteorological information forecast, medical examination, and camouflage tests [1]. In a hyperspectral image, pixels are typically mixtures of several pure material components due to the limitation of spatial resolution and intimate interactions among materials. Spectral unmixing is thus one of the most important tasks in hyperspectral data analysis, aiming to separate the observed pixel spectra into a collection of constituent spectra, or spectral signatures, called endmembers and to estimate fractions associated with each component called abundances. Spectral unmixing provides a comprehensive and quantitative mapping of the elementary materials that are present in the acquired data, and it is widely used for many applications, such as determining the constitutions of geological mixtures and performing a classification of crops and vegetation.
Most spectral unmixing approaches are designed based on pre-assumed mixture models that describe in an analytical way how the endmembers are combined to mixed spectra measured by the sensor [2]. The linear mixing model (LMM) is the most widely used one, and it assumes that the mixing occurs at a macroscopic scale [3]. A measured spectrum is the linear combinations of the endmembers, weighted by the fractional abundances. To be physically interpretable, LMM is usually performed with two physical constraints, abundance nonnegative constraint (ANC) and abundance sum-to-one constraint (ASC). Multiple scattering effects and intimate interactions in real environment require using nonlinear mixture models. Such models include intimate mixture model [4], bilinear model [5], linear-quadratic mixing model (LQM) [6], polynomial post-nonlinear mixing model (PPNM) [7], to cite a few. However, due to the simplicity and interpretability of the analysis results, LMM-based unmixing strategies are mostly used in practice [2]. A number of unmixing algorithms are proposed, including long-standing geometrical and statistical approaches and the recently introduced sparse regression-based unmixing algorithms [8, 9, 10, 11].
Considering inherent spatial-spectral duality exists in hyperspectral data, regularized unmixing algorithms have been proposed in recent years to make use of spectral information and spatial contextual information to enhance the unmixing performance. For instance, in [8], authors introduce a total variation (TV) regularizer to promote spatial consistency of estimated abundances. In [12], the quadratic Laplacian regularization is introduced based on graph representation. In [13], authors present a spatial spectral coherence regularization that relates abundance estimation of a pixel to that of its neighboring pixels with spectral similarities. In [14], authors perform the unmixing with low-rank spatial regularization within fixed-size square windows.
However, it is necessary to establish a frame for these various ways of regularization via a proper mathematical tool. A graph is a ubiquitous structure that describes the connection relationship of a set of vertices. Graph theory is actively used in fields such as biochemistry (genomics), electrical engineering (communication networks and coding theory), computer science (algorithms and computation), and operations research (scheduling) [15]. In addition, several works apply graph theoretical techniques to hyperspectral images, including methods for dimensionality reduction [16], anomaly detection [17], and classification [18]. In the context of hyperspectral data unmixing, a graph can be used to model relations of spatial and spectral information of hyperspectral pixels. In this chapter, we will present a variety of ways to construct a graph for hyperspectral unmixing and formulate the associated unmixing problem with solutions given by the alternating direction method of multipliers (ADMM) strategy.
The remainder of the chapter is organized as follows: Section 2 introduces graph theory and graph construction methods in the context of hyperspectral unmixing. Section 3 formulates the sparse linear unmixing problem based on graph regularization. In Section 4, the solution to the formulated problem is derived via the ADMM algorithm. Section 5 reports the experiment’s results. Finally, Section 6 concludes this chapter.
2. Graph construction
2.1. Introduction to graphs
We firstly review some fundamental elements of a graph. A graph is a general data structure described by
In a simple setting, if two vertices are connected by an edge, the weight is set to 1, otherwise the weight is 0. The following part introduces some other ways to measure the similarity among vertices, in other words, to define the weights. We can use either adjacency matrices or incidence matrices to describe graphs depending on the type of operations to be performed. Elements of the matrix
In addition, a degree matrix for a graph is a diagonal matrix
2.2. Graph construction for hyperspectral images
In this part, we elaborate the ways to construct graphs in the context of hyperspectral image analysis. The performance of spectral unmixing is closely tied to the graph construction of images, and in the hyperspectral remote sensing literature, there are a number of techniques. In [19], authors summarize a survey of spectral graph construction techniques and discuss advantages and disadvantages of these techniques. Generally, each pixel can be viewed as a vertex (or node), and each vertex is associated with a continuous spectrum. A set of edges can be set and assigned with weights in different senses as presented here below.
2.2.1. Four-neighbor graph
A common and straightforward construction is to consider the four-neighbor graph, where every vertex (i.e., every pixel) is connected to four nearest spatially adjacent neighboring pixels, as illustrated in Figure 2.
2.2.2. Threshold-compared graph
Another alternative to construct a graph is to calculate all pairwise distances and an edge is placed if the distance between two vertices is less than a user-predefined threshold. The distance in the hyperspectral image can be measured using the spectral distance or spatial distance. For instance,
2.2.3. K-nearest neighbor graph
Constructing a graph with
2.2.4. Spatial-spectral graph
As pixels in a hyperspectral image possess spatial locations and spectral signatures, it can be beneficial to construct a graph by incorporating both spatial and spectral information. For instance, a graph can be constructed with local four neighborhood pixels and by considering spectral similarity among pixels, as described in Figure 3.
2.2.5. Weighted graph
Above methods construct unweighted graphs with only connection indications among pixels. Several other methods further impose weights on each edge. For instance, spectral similarity measured by the Gaussian kernel can be used to define weights:
where
Weights can also be calculated by considering both spatial and spectral information. For instance, [21] proposes to define weights by:
where
where
There are also some other methods to construct graphs adapted to hyperspectral images, such as adaptive nearest neighbor graphs, density-weighted
3. Graph-based regularization in unmixing
With a constructed graph at hand to model the relation of pixels, in this section, we present the way to perform a sparse unmixing with the graph regularization.
3.1. Sparse unmixing
Consider the linear mixing model:
where λ is the regularized parameter.
In this chapter, we formulate the problem without ASC constraint because of using the
3.2. Graph regularization for sparse unmixing
Since a graph relates the pixels of image via spatial and spectral relations, we can regularize the unmixing problem with pixel relations defined by the graph. Let
where
This graph regularization term Eq. (7) is based on the assumption that if two vertices are connected by an edge, then the abundances of the two vertices are similar. This term measures the differences between all pairs of abundances weighted by their degrees of similarity in the graph. The graph regularization then promotes piecewise constant transitions of estimates among the related pixels. Parameter
Problem Eq. (6) is equivalently expressed as:
If we use a spatial four-neighborhood graph in this unmixing problem with the weights being simply set to 1 and 0, it can generally be identical with the SUnSAL-TV algorithm [8]. The right term can promote piecewise constant transitions in the fractional abundance among neighborhood pixels and achieve spatial consistency of estimated abundances.
Instead of considering promoting the similarities among estimated abundances, an alternative way is to promote the similarities of reconstructed spectra among the connected pixels. In [24], authors propose a nonlocal TV regularization, with the regularization term given as:
This can also be written with incidence matrix
4. Solution to the formulated problem
We propose to solve the formulated unmixing problem Eq. (9) via the ADMM algorithm. In this section, we first briefly review the ADMM algorithm and then apply it to our unmixing problem.
4.1. Introduction of ADMM
ADMM is an algorithm that is intended to blend the decomposability of dual ascent with the superior convergence properties of the method of multipliers. The algorithm solves problems in the form [25]:
with variables
The first step is to write the augmented Lagrangian of problem Eq. (12):
ADMM suggests achieving the optimum via the following iterations:
where
4.2. Solutions via ADMM
In order to apply the canonical ADMM algorithm to the problem (9), we introduce the auxiliary variables and transform the problem as follows:
where
where
The algorithm steps are as follows:
In step 4 of minimizing the augmented Lagrangian with respect to
Similarly, the solution of
To compute
where
The solution of
The solution of
where
5. Experiments
In this section, we illustrate the experimental results via a synthetic hyperspectral data set (denoted by Data 1) and a real hyperspectral data set (denoted by Data 2) with various ways of graph construction.
5.1. Experiments with simulated data sets
In this part, the synthetic data consists of
The quality of unmixing results for the simulated data can be measured by comparing the estimated and actual abundances using the root mean square error (RMSE),
where
We define the graph based on the simulated data using three methods: the four-neighbor graph, the threshold-compared graph and the spectral-spatial graph respectively.
In the experiment, the threshold-compared undirected graph is constructed as follows:
where all pairs of spectral distance are compared with a user-defined threshold. Meanwhile, the spectral-spatial graph is constructed by considering four neighbors of spatial location and
From this table, we can see that the performance of the proposed algorithm with a threshold-compared graph is better than the others. Although the second graph in Table 1 combines the spectral and spatial information, using spatial relation is not always a good way to connect pixels because it is possible that two adjacent pixels may have significantly different spectral features. Figure 4 shows the true abundance maps and the abundances estimated by the proposed algorithm associated with the three constructed graphs. We observe that the second row of the square regions is better conserved with the proposed algorithm using the threshold-compared graph.
15 dB | 20 dB | 30 dB | |
---|---|---|---|
Four-neighbor graph | 0.0246 μ = 0.005,λ = 0.05 | 0.0184 μ = 0.005,λ = 0.05 | 0.0051 μ = 5 × 10^(−4),λ = 0.01 |
Spectral-spatial combined graph | 0.0085 k = 25; μ = 5 × 10^(−4),λ = 0.01 | 0.0052 k = 25 μ = 0.005,λ = 0.01 | 0.0021 k = 25 μ = 5 × 10^(−4),λ = 0.005 |
Threshold-compared graph | 0.0025 threshold = 9 μ = 5 × 10^(−4),λ = 0.1 | 0.0015 threshold = 3 μ = 5 × 10^(−4),λ = 0.5 | 0.0009 threshold = 0.25 μ = 5 × 10^(−4),λ = 0.1 |
5.2. Experiments with AVIRIS data
We also tested algorithms with a real hyperspectral image. The image is captured on the Cuprite mining district by AVIRIS. A sub-image of
6. Conclusion
In this chapter, we propose to use graph as a mathematical tool to relate pixels in hyperspectral data. We the present a variety of methods of constructing a graph according to spatial information and spectral information embedded in an image. A sparse unmixing problem is then formulated with the graph regularization to enhance the estimation performance. An ADMM-based algorithm is then presented to solve the formulated problem. In the experiments, we compare the unmixing performance of the presented unmixing algorithm with different graphs, using a synthetic hyperspectral data and a real data. Future works include evaluating the unmixing performance with weighted graphs.
Acknowledgments
This work was supported in part by National Natural Science Foundation of China under grant 61671382 and in part by Natural Science Foundation of Shenzhen under grant JCYJ2017030155315873.
References
- 1.
Keshava N, Mustard JF. Spectral unmixing. IEEE Signal Processing Maganize. 2002; 19 (1):44-57 - 2.
Dobigeon N, Tourneret J-Y, Richard C, Bermudez JCM, McLaughlin S, Hero AO. Nonlinear unmixing of hyperspectral images: Models and algorithms. IEEE Signal Processing Magazine. 2014; 31 (1):82-94 - 3.
Singer RB, McCord TB. Mars-large scale mixing of bright and dark surface materials and implications for analysis of spectral reflectance. In: Lunar and Planetary Science Conference Proceedings; 1979. p. 1835-1848 - 4.
Hapke B. Bidirectional reflectance spectroscopy: 1. Theory. Journal of Geophysical Research: Solid Earth. 1981; 86 :3039-3054 - 5.
Halimi A, Altmann Y, Dobigeon N, Tourneret J-Y. Nonlinear unmixing of hyperspectral images using a generalized bilinear model. IEEE Transactions on Geoscience and Remote Sensing. 2011; 49 (11):4153-4162 - 6.
Meganem I, Deliot P, Briottet X, Deville Y, Hosseini S. Linear--quadratic mixing model for reflectances in urban environments. IEEE Transactions on Geoscience and Remote Sensing. 2014; 52 (1):544-558 - 7.
Altmann Y, Dobigeon N, Tourneret J-Y. Nonlinearity detection in hyperspectral images using a polynomial post-nonlinear mixing model. IEEE Transactions on Image Processing. 2013; 22 (4):1267-1276 - 8.
Iordache M-D, Bioucas-Dias JM, Plaza A. Total variation spatial regularization for sparse hyperspectral unmixing. IEEE Transactions on Geoscience and Remote Sensing. 2012; 50 (11):4484-4502 - 9.
Heinz DC, Chang C-I. Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery. IEEE transactions on geoscience and remote sensing. 2001; 39 (3):529-545 - 10.
Dobigeon N, Tourneret J-Y, Chang C-I. Semi-supervised linear spectral unmixing using a hierarchical Bayesian model for hyperspectral imagery. IEEE Transactions on Signal Processing. 2008; 56 (7):2684-2695 - 11.
Chang C-I, Wu C-C, Liu W, Ouyang Y-C. A new growing method for simplex-based endmember extraction algorithm. IEEE Transactions on Geoscience and Remote Sensing. 2006; 44 (10):2804-2819 - 12.
Ammanouil R, Ferrari A, Richard C. A graph laplacian regularization for hyperspectral data unmixing. In: Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference; 2015. p. 1637-1641 - 13.
Castrodad A, Xing Z, Greer JB, Bosch E, Carin L, Sapiro G. Learning discriminative sparse representations for modeling, source separation, and mapping of hyperspectral imagery. IEEE Transactions on Geoscience and Remote Sensing. 2011; 49 (11):4263-4281 - 14.
Qu Q, Nasrabadi NM, Tran TD. Abundance estimation for bilinear mixture models via joint sparse and low-rank representation. IEEE Transactions on Geoscience and Remote Sensing. 2014; 52 (7):4404-4423 - 15.
Pirzada S. Applications of graph theory. Journal of the Korean Society for Industrial and. Applied Mathematics. 2007; 7 (1) - 16.
Bachmann CM, Ainsworth TL, Fusina RA. Exploiting manifold geometry in hyperspectral imagery. IEEE transactions on Geoscience and Remote Sensing. 2005; 43 (3):441-454 - 17.
Basener B, Ientilucci EJ, Messinger DW. Anomaly detection using topology. Proceedings of SPIE. 2007 - 18.
Camps-Valls G, Bandos TV, Zhou D. Semi-supervised graph-based hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing. 2007; 45 (10):3044-3054 - 19.
Stevens JR, Resmini RG, Messinger DW. Spectral-density-based graph construction techniques for Hyperspectral image analysis. IEEE Transactions on Geoscience and Remote Sensing. 2017; 55 (10):5966-5983 - 20.
Zelnik-Manor L, Perona P. Self-tuning spectral clustering. In: Advances in Neural Information Processing Systems. 2005. pp. 1601-1608 - 21.
Shi J, Malik J. Normalized cuts and image segmentation. IEEE Transactions on pattern analysis and machine intelligence. 2000; 22 (8):888-905 - 22.
Gillis DB, Bowles JH. Hyperspectral image segmentation using spatial-spectral graphs. Proc. SPIE. 2012 - 23.
Bioucas-Dias JM. Figueiredo MAT. Alternating direction algorithms for constrained sparse regression: Application to hyperspectral unmixing. In: Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), 2010 2nd Workshop; 2010. p. 1-4 - 24.
Ammanouil R, Ferrari A, Richard C. Hyperspectral data unmixing with graph-based regularization. Proc. IEEE GRSS Workshop Hyperspectral Image Signal. 2015:1-4 - 25.
Boyd S, Parikh N, Chu E, Peleato B, Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning. 2011; 3 (1):1-122 - 26.
Chen J, Richard C, Honeine P. Nonlinear unmixing of hyperspectral data based on a linear-mixture/nonlinear-fluctuation model. IEEE Transactions on Signal Processing. 2013; 61 (2):480-492