Introductory Chapter: Some Preliminary Aspects of Inverse Problem

1. Introduction

Physical research in science can be divided into two groups. The first is that when by complete description of a physical system, we can predict the outcome of some measurements. This problem is called the modelization problem or the forward problem. The second group of research consists of using the actual result of some observations to infer the values of the parameters that characterize the system. It is the inverse problem, which starts with the causes and then calculates the effects. The importance of inverse problems is that they tell us about physical parameters that we cannot directly observe.

2. Primary equations of inverse problem

The inverse problem is that one wants to determine the model parameters $p$ that produce the observed data or measurements $d$. $\mathbf{F}$ stays for some measurement operator that maps parameters in a functional space $\mathfrak{P}$, typically a Banach or Hilbert space, to the space of data $\mathfrak{D}$, typically another Banach or Hilbert space.

$$d=\mathbf{F}p\text{for}p\in \mathfrak{P}\text{and}d\in \mathfrak{D}.$$E1

Solving the inverse problem amounts to finding point(s) $p\in \mathfrak{P}$ from knowledge of the data $d\in \mathfrak{D}$ such that Eq. (1) (or its approximation) holds. In the case of a measurement, operator is linear and there is a finite number of parameters, Eq. (1) can be written as a linear system, where $\mathbf{F}$ is the matrix that characterizes the measurement operator, and $\mathfrak{P}$ and $d\in \mathfrak{D}$ are corresponding vector spaces. Such inverse problem is called linear.

Inverse problems may be difficult to solve for at least two different reasons:

1. Different values of the model parameters may be not consistent with the data;

2. Discovering the values of the model parameters may require the exploration of a huge parameter space.

If it is acquired enough data to uniquely reconstruct the parameters, then the measurement operator can be injective, which means

$$\mathbf{F}\left(p_1\right)=\mathbf{F}\left(p_2\right)\Rightarrow p_1=p_2\text{for}\mathrm{all}{p}_1,p_2\in \mathfrak{P}.$$E2

When $\mathbf{F}$ is injective, one can construct an inversion operator ${\mathbf{F}}^{-1}$ mapping the range of $\mathbf{F}$ to a uniquely defined element $\mathfrak{P}$. In the case of a linear inverse problem, ${\mathbf{F}}^{-1}$ is an inverse matrix. Further, the main features of the inverse operator are characterized by stability estimates that quantify how errors in the available measurements translate into errors in the reconstructions. It can be expressed as follows:

$$\parallel p_1-p_2{\parallel }_{\mathfrak{P}}\le \alpha \parallel \mathbf{F}\left(p_1\right)-\mathbf{F}\left(p_2\right){\parallel }_{\mathfrak{D}}.$$E3

Where $\alpha :{\mathrm{ℝ}}_{+}\to {\mathrm{ℝ}}_{+}$ stay for an increasing function, such that $\alpha \left(0\right)=0$. This function gives an estimate of the reconstruction error $\parallel p_1-p_2{\parallel }_{\mathfrak{P}}$ based on the error in the data $\parallel \mathbf{F}\left(p_1\right)-\mathbf{F}\left(p_2\right){\parallel }_{\mathfrak{D}}$. When the reconstructed parameters are acceptable, for instance when $\alpha \left(x\right)=\mathit{Cx}$ for some constant $C$, then the inverse problem is called well-posed. When the reconstruction is contaminated by too large a noisy component, then the inverse problem is ill-posed.

Injectivity of $\mathbf{F}$ means satisfying the two conditions for a well-posed problem suggested by Jacques Hadamard [1], Existence and Uniqueness of solutions. Eq. (3) is the third Hadamard’s condition, which is Stability of the solution or solutions.

Typically, inverse problems are ill-posed. Even when we have a linear inverse problem with invertible matrix $\mathbf{F}$, it gives an ill-posed problem that can be solved by using the Moore-Penrose inverse matrix [2, 3] and least squares solutions inducted by it.

The goal of many experiments is to infer a property or attribute from data that is indirectly related to the unknown quantity. Parameter estimation problems usually satisfy the first criterion of well-posed problems, since something is responsible for the observed system response. Instead, they violate the third criterion and” almost” violate the second criterion because many different candidate solutions exist that, when substituted into the measurement model, produce very similar data. The condition of stability is often violated, because the inverse problem is represented by a mapping between metric spaces, but inverse problems are often formulated in infinite dimensional spaces. Therefore, limitations to a finite number of measurements, and the practical consideration of recovering only a finite number of unknown parameters may lead to the problems being recast in discrete form. In this case, the inverse problem is typically ill-conditioned and a regularization can be used. One of the most famous regularizations is the Tikhonov regularization [4]. The idea of Tikhonov regularization may be introduced as follows. In its simplest form, it consists in replacing the Eq. (1) with the second kind of equation

$${\mathbf{F}}^{\ast }\mathbf{F}p+\mathit{\alpha p}={\mathbf{F}}^{\ast }d$$E4

where $\alpha$ is a positive parameter. It leads to that the problem of solving Eq. (4) is well-posed.

Unlike parameter estimation, inverse problems often violate Hadamard’s first criterion since an optimal design outcome may be specified that cannot possibly be produced by the system. On the other hand, the existence of multiple designs (solutions) that produce an acceptable outcome violates the second criterion. From these, it follows inverse problems that are mathematically ill-posed due to an information deficit. In the parameter estimation case, the measurements barely provide sufficient information to specify a unique solution, and in some cases, the data could be explained by an infinite set of candidate solutions. Information from measurement data and prior information can be combined through Bayes’ equation to produce estimates for the Quantities-of-Interest (QoI). In this approach, the measurements, $d$, and the QoI, $x\in \mathfrak{P}$, are interpreted as random variables that obey probability density functions (PDFs). The PDFs are related by Bayes’ equation

$$P\left(x|d\right)=\frac{P\left(d|x\right)}{P\left(d\right)}{P}_{\mathit{pr}}\left(x\right)$$E5

where $P\left(d|x\right)$ is the likelihood of the observed data occurring for a hypothetical parameter $x$, accounting for measurement noise and model error (“likelihood PDF”), $P_{\mathit{pr}}\left(x\right)$ defines what is known before the measurement takes place about a hypothetical parameter $x$, (“prior PDF”), $P\left(x|d\right)$ is the posterior PDF, which defines what is known about $x$ from both the measurements and prior information, and $P\left(d\right)$ is the evidence, which scales the posterior so that it satisfies the law of total probabilities.

Therefore, “the most general theory is obtained by using a probabilistic point of view, where the a priori information on the model parameters is represented by a probability distribution over the” model space”. A priori probability distribution is transformed into the a posteriori probability distribution, by incorporating a physical theory (relating the model parameters to some observable parameters) and the actual result of the observations (with their uncertainties)” [5].