RMSE evaluating performances with different values of SNR, with three constructed graphs and optimal regularized parameters. The values of threshold and *k* are also shown.

## Abstract

This chapter presents graph construction for hyperspectral data and associated unmixing methods based on graph regularization. Graph is a ubiquitous mathematical tool for modeling relations between objects under study. In the context of hyperspectral image analysis, constructing graphs can be useful to relate pixels in order to perform corporative analysis instead of analyzing each pixel individually. In this chapter, we review fundamental elements of graphs and present different ways to construct graphs in both spatial and spectral senses for hyperspectral images. By incorporating a graph regularization, we then formulate a general hyperspectral unmixing problem that can be important for applications such as remote sensing and environment monitoring. Alternating direction method of multipliers (ADMM) is also presented as a generic tool for solving the formulated unmixing problems. Experiments validate the proposed scheme with both synthetic data and real remote sensing data.

### Keywords

- hyperspectral imaging
- graph construction
- spectral unmixing
- graph regularization
- spectral-spatial correlation

## 1. Introduction

Hyperspectral imaging analysis has found a wide range of applications including agricultural monitoring, environment detection, meteorological information forecast, medical examination, and camouflage tests [1]. In a hyperspectral image, pixels are typically mixtures of several pure material components due to the limitation of spatial resolution and intimate interactions among materials. Spectral unmixing is thus one of the most important tasks in hyperspectral data analysis, aiming to separate the observed pixel spectra into a collection of constituent spectra, or spectral signatures, called endmembers and to estimate fractions associated with each component called abundances. Spectral unmixing provides a comprehensive and quantitative mapping of the elementary materials that are present in the acquired data, and it is widely used for many applications, such as determining the constitutions of geological mixtures and performing a classification of crops and vegetation.

Most spectral unmixing approaches are designed based on pre-assumed mixture models that describe in an analytical way how the endmembers are combined to mixed spectra measured by the sensor [2]. The linear mixing model (LMM) is the most widely used one, and it assumes that the mixing occurs at a macroscopic scale [3]. A measured spectrum is the linear combinations of the endmembers, weighted by the fractional abundances. To be physically interpretable, LMM is usually performed with two physical constraints, abundance nonnegative constraint (ANC) and abundance sum-to-one constraint (ASC). Multiple scattering effects and intimate interactions in real environment require using nonlinear mixture models. Such models include intimate mixture model [4], bilinear model [5], linear-quadratic mixing model (LQM) [6], polynomial post-nonlinear mixing model (PPNM) [7], to cite a few. However, due to the simplicity and interpretability of the analysis results, LMM-based unmixing strategies are mostly used in practice [2]. A number of unmixing algorithms are proposed, including long-standing geometrical and statistical approaches and the recently introduced sparse regression-based unmixing algorithms [8, 9, 10, 11].

Considering inherent spatial-spectral duality exists in hyperspectral data, regularized unmixing algorithms have been proposed in recent years to make use of spectral information and spatial contextual information to enhance the unmixing performance. For instance, in [8], authors introduce a total variation (TV) regularizer to promote spatial consistency of estimated abundances. In [12], the quadratic Laplacian regularization is introduced based on graph representation. In [13], authors present a spatial spectral coherence regularization that relates abundance estimation of a pixel to that of its neighboring pixels with spectral similarities. In [14], authors perform the unmixing with low-rank spatial regularization within fixed-size square windows.

However, it is necessary to establish a frame for these various ways of regularization via a proper mathematical tool. A graph is a ubiquitous structure that describes the connection relationship of a set of vertices. Graph theory is actively used in fields such as biochemistry (genomics), electrical engineering (communication networks and coding theory), computer science (algorithms and computation), and operations research (scheduling) [15]. In addition, several works apply graph theoretical techniques to hyperspectral images, including methods for dimensionality reduction [16], anomaly detection [17], and classification [18]. In the context of hyperspectral data unmixing, a graph can be used to model relations of spatial and spectral information of hyperspectral pixels. In this chapter, we will present a variety of ways to construct a graph for hyperspectral unmixing and formulate the associated unmixing problem with solutions given by the alternating direction method of multipliers (ADMM) strategy.

The remainder of the chapter is organized as follows: Section 2 introduces graph theory and graph construction methods in the context of hyperspectral unmixing. Section 3 formulates the sparse linear unmixing problem based on graph regularization. In Section 4, the solution to the formulated problem is derived via the ADMM algorithm. Section 5 reports the experiment’s results. Finally, Section 6 concludes this chapter.

## 2. Graph construction

### 2.1. Introduction to graphs

We firstly review some fundamental elements of a graph. A graph is a general data structure described by

In a simple setting, if two vertices are connected by an edge, the weight is set to 1, otherwise the weight is 0. The following part introduces some other ways to measure the similarity among vertices, in other words, to define the weights. We can use either adjacency matrices or incidence matrices to describe graphs depending on the type of operations to be performed. Elements of the matrix *i* to vertex *j* and zero when there is no edge. If the graph is undirected, the adjacency matrix is symmetric. Incidence matrices show the relationship between vertices and edges. An undirected graph can have two kinds of incidence matrices: unoriented and oriented matrices. An oriented incidence matrix in the undirected graph can be denoted by

In addition, a degree matrix for a graph is a diagonal matrix *n* is the number of vertices and

### 2.2. Graph construction for hyperspectral images

In this part, we elaborate the ways to construct graphs in the context of hyperspectral image analysis. The performance of spectral unmixing is closely tied to the graph construction of images, and in the hyperspectral remote sensing literature, there are a number of techniques. In [19], authors summarize a survey of spectral graph construction techniques and discuss advantages and disadvantages of these techniques. Generally, each pixel can be viewed as a vertex (or node), and each vertex is associated with a continuous spectrum. A set of edges can be set and assigned with weights in different senses as presented here below.

#### 2.2.1. Four-neighbor graph

A common and straightforward construction is to consider the four-neighbor graph, where every vertex (i.e., every pixel) is connected to four nearest spatially adjacent neighboring pixels, as illustrated in Figure 2.

#### 2.2.2. Threshold-compared graph

Another alternative to construct a graph is to calculate all pairwise distances and an edge is placed if the distance between two vertices is less than a user-predefined threshold. The distance in the hyperspectral image can be measured using the spectral distance or spatial distance. For instance,

#### 2.2.3. K-nearest neighbor graph

Constructing a graph with *k*-nearest neighbors (k-NN) is a popular method. In this case, an edge is set between two vertices if vertex *k*-NN of vertex *k*-nearest neighbors. Consequently, the graph is a directed graph. It is worth noting that constructing such a graph requires calculating all pairwise distances and ordering these values on each vertex, and these operations lead to high computational costs.

#### 2.2.4. Spatial-spectral graph

As pixels in a hyperspectral image possess spatial locations and spectral signatures, it can be beneficial to construct a graph by incorporating both spatial and spectral information. For instance, a graph can be constructed with local four neighborhood pixels and by considering spectral similarity among pixels, as described in Figure 3.

#### 2.2.5. Weighted graph

Above methods construct unweighted graphs with only connection indications among pixels. Several other methods further impose weights on each edge. For instance, spectral similarity measured by the Gaussian kernel can be used to define weights:

where

Weights can also be calculated by considering both spatial and spectral information. For instance, [21] proposes to define weights by:

where

where

There are also some other methods to construct graphs adapted to hyperspectral images, such as adaptive nearest neighbor graphs, density-weighted *k*-NN graphs, and shared nearest-neighbor graphs [19].

## 3. Graph-based regularization in unmixing

With a constructed graph at hand to model the relation of pixels, in this section, we present the way to perform a sparse unmixing with the graph regularization.

### 3.1. Sparse unmixing

Consider the linear mixing model: *L* spectral bands, *R* pure spectral signatures, and

where λ is the regularized parameter.

In this chapter, we formulate the problem without ASC constraint because of using the

### 3.2. Graph regularization for sparse unmixing

Since a graph relates the pixels of image via spatial and spectral relations, we can regularize the unmixing problem with pixel relations defined by the graph. Let *L* spectral bands and

where

This graph regularization term Eq. (7) is based on the assumption that if two vertices are connected by an edge, then the abundances of the two vertices are similar. This term measures the differences between all pairs of abundances weighted by their degrees of similarity in the graph. The graph regularization then promotes piecewise constant transitions of estimates among the related pixels. Parameter

Problem Eq. (6) is equivalently expressed as:

If we use a spatial four-neighborhood graph in this unmixing problem with the weights being simply set to 1 and 0, it can generally be identical with the SUnSAL-TV algorithm [8]. The right term can promote piecewise constant transitions in the fractional abundance among neighborhood pixels and achieve spatial consistency of estimated abundances.

Instead of considering promoting the similarities among estimated abundances, an alternative way is to promote the similarities of reconstructed spectra among the connected pixels. In [24], authors propose a nonlocal TV regularization, with the regularization term given as:

This can also be written with incidence matrix **B** as:

## 4. Solution to the formulated problem

We propose to solve the formulated unmixing problem Eq. (9) via the ADMM algorithm. In this section, we first briefly review the ADMM algorithm and then apply it to our unmixing problem.

### 4.1. Introduction of ADMM

ADMM is an algorithm that is intended to blend the decomposability of dual ascent with the superior convergence properties of the method of multipliers. The algorithm solves problems in the form [25]:

with variables _{,} where _{,} _{.}

The first step is to write the augmented Lagrangian of problem Eq. (12):

ADMM suggests achieving the optimum via the following iterations:

where

### 4.2. Solutions via ADMM

In order to apply the canonical ADMM algorithm to the problem (9), we introduce the auxiliary variables and transform the problem as follows:

where

where

The algorithm steps are as follows:

**Step 1**. Input the observed pixels

**Step 2.** Initialization:

**Step 3.** Repeat:

**Step 4.**

**Step 5**.

**Step 6.**

**Step 7.**

**Step 8.**

**Step 9.** Update the Lagrangian multipliers:

**Step 10.** Until stopping criterion is satisfied.

In step 4 of minimizing the augmented Lagrangian with respect to

Similarly, the solution of

To compute

where _{,} respectively.

The solution of

The solution of

where _{,} respectively.

## 5. Experiments

In this section, we illustrate the experimental results via a synthetic hyperspectral data set (denoted by Data 1) and a real hyperspectral data set (denoted by Data 2) with various ways of graph construction.

### 5.1. Experiments with simulated data sets

In this part, the synthetic data consists of

The quality of unmixing results for the simulated data can be measured by comparing the estimated and actual abundances using the root mean square error (RMSE),

where *i*th pixel,

We define the graph based on the simulated data using three methods: the four-neighbor graph, the threshold-compared graph and the spectral-spatial graph respectively.

In the experiment, the threshold-compared undirected graph is constructed as follows:

where all pairs of spectral distance are compared with a user-defined threshold. Meanwhile, the spectral-spatial graph is constructed by considering four neighbors of spatial location and *k*-nearest neighbors of spectral distance.

From this table, we can see that the performance of the proposed algorithm with a threshold-compared graph is better than the others. Although the second graph in Table 1 combines the spectral and spatial information, using spatial relation is not always a good way to connect pixels because it is possible that two adjacent pixels may have significantly different spectral features. Figure 4 shows the true abundance maps and the abundances estimated by the proposed algorithm associated with the three constructed graphs. We observe that the second row of the square regions is better conserved with the proposed algorithm using the threshold-compared graph.

15 dB | 20 dB | 30 dB | |
---|---|---|---|

Four-neighbor graph | 0.0246 μ = 0.005,λ = 0.05 | 0.0184 μ = 0.005,λ = 0.05 | 0.0051 μ = 5 × 10^(−4),λ = 0.01 |

Spectral-spatial combined graph | 0.0085 k = 25; μ = 5 × 10^(−4),λ = 0.01 | 0.0052 k = 25 μ = 0.005,λ = 0.01 | 0.0021 k = 25 μ = 5 × 10^(−4),λ = 0.005 |

Threshold-compared graph | 0.0025 threshold = 9 μ = 5 × 10^(−4),λ = 0.1 | 0.0015 threshold = 3 μ = 5 × 10^(−4),λ = 0.5 | 0.0009 threshold = 0.25 μ = 5 × 10^(−4),λ = 0.1 |

### 5.2. Experiments with AVIRIS data

We also tested algorithms with a real hyperspectral image. The image is captured on the Cuprite mining district by AVIRIS. A sub-image of

## 6. Conclusion

In this chapter, we propose to use graph as a mathematical tool to relate pixels in hyperspectral data. We the present a variety of methods of constructing a graph according to spatial information and spectral information embedded in an image. A sparse unmixing problem is then formulated with the graph regularization to enhance the estimation performance. An ADMM-based algorithm is then presented to solve the formulated problem. In the experiments, we compare the unmixing performance of the presented unmixing algorithm with different graphs, using a synthetic hyperspectral data and a real data. Future works include evaluating the unmixing performance with weighted graphs.

## Acknowledgments

This work was supported in part by National Natural Science Foundation of China under grant 61671382 and in part by Natural Science Foundation of Shenzhen under grant JCYJ2017030155315873.