Data Fusion for Close‐Range Detection

Abstract

Two approaches for combining humanitarian mine detection sensors are described in parallel, one based on belief functions and the other one based on possibility theory. In a first step, different measures are extracted from the sensor data. After that, based on prior information, mass functions and possibility distributions are derived. The combination of possibility degrees, as well as of masses, is performed in two steps. The first one applies to all measures derived from one sensor. The second one combines results obtained in the first step for all sensors used. Combination operators are chosen to account for different characteristics of the sensors. Comparison of the combination equations of the two approaches is performed as well. Furthermore, selection of the decision rules is discussed for both approaches. These approaches are illustrated on a set of real mines and non‐dangerous objects and using three sensors: an infrared camera, an imaging metal detector and a ground‐penetrating radar.

1. Introduction

Multi‐sensor data fusion techniques prove to be useful for two main humanitarian mine action types: mined area reduction and close‐range antipersonnel (AP) mine detection. In this chapter, data fusion for the latter mine action type is addressed. Close‐range AP mine detection refers to detection of (sub‐)surface anomalies that may be associated with mine presence (for instance, detection of differences in temperature thanks to an infrared camera (IR) or detection of metals by a metal detector (MD)) and/or to detection of explosive materials.

Efficient modelling and fusion of extracted features can improve the reliability and quality of single‐sensor‐based processing [1, 2]. Nevertheless, taking into account that there is a wide range of conditions and scenarios between minefields (such as mine types, structure of minefield and soil types) as well as within one minefield (e.g. burial depths and angles, moisture), there is no unique single‐sensor solution, meaning that a high‐enough performance of humanitarian mine action tools can be reached only using multi‐sensor and sensor/data fusion approaches [3]. In addition, since the sensors used are, as a matter of fact, detectors of various anomalies, the classification and detection results can be improved by combining these complementary pieces of information. Last but not least, in order to take into account partial knowledge, intra‐ and inter‐minefield variability, ambiguity and uncertainty, fuzzy set or possibility theory [4] and belief functions [5] within the framework of the Dempster‐Shafer (DS) theory [6] prove to be beneficial.

The chapter is organized as follows. An analysis of modelling and of fusion of extracted features is performed. After that, two fusion approaches are presented, one of them being based on the belief function theory and the other one related to the possibility theory. These approaches are then illustrated using real data gathered within the Dutch project HOM‐2000 [7], which are acquired using three intrinsically complementary sensors: infrared camera, metal detector and ground‐penetrating radar (GPR). These results are obtained within two Belgian humanitarian demining projects: HUDEM and BEMAT. Importance of collateral information (knowledge about types of mines, mine records, etc.) is demonstrated.

2. About close‐range detection

Due to a large variety of mine types as well as of conditions in which they can be found, no single sensor applied in close‐range AP mine detection can obtain the necessarily high‐detection rate in a wide range of possible situations/scenarios. Thus, a logical way towards deriving a solution consists in using several sensors that are complementary and taking the best out of their combination. To this end, an infrared camera (IR), a ground‐penetrating radar (GPR) and an imaging metal detector (MD) present a very promising combination. In this chapter, we describe two approaches for combining these sensors, one based on the belief function theory and the other one on the possibility theory. These approaches can easily be adapted to other combinations of sensors.

An important part of the work performed in the field of fusion of dissimilar mine detection sensors is based on statistics [8, 9]. Examples of rare alternative approaches are [10] (neural networks) and [11] (fuzzy fusion of classifiers). The statistical approaches lead to good results for a particular scenario, but they ignore or just briefly mention that, once we look for more general solutions, several important problems have to be faced in this domain of application [12]. For instance, the data are variable, highly dependent on the conditions and on the context. Then, it is impossible to model every possible object (every mine or every other object that might be confused with mines). In addition, the data do not allow for a reliable statistical learning since they are not numerous enough. Finally, the data do not give precise information regarding the mine type, resulting in an ambiguity, typically between several mine types. Note that in the domain of humanitarian mine detection, a vast majority of the fusion attempts, for example, [13, 14], treat every alarm as a mine, and not as an object that could be a mine, but a false alarm as well.

In a previous work [15], a method based on the belief functions [6, 16, 17] has been proposed. In this chapter, we compare it with an alternative approach, based on the possibility theory, in order to take advantage of the flexibility in the choice of combination operators [18]. As shown in Ref. [2], this is exploited in order to account for the different characteristics of the sensors to be combined.

In this domain of application, to our knowledge, there is no work that applies the two fusion theories in parallel or that compares them. In other domains of application, some works on comparing the two theories are published, for example [19], where the qualitative possibility theory is opposed to the belief function theory and a fictitious example of assessing the value of a candidate is used as an illustration. On the contrary to that article, we use the quantitative possibility theory here.

3. Numerical information fusion using belief functions and possibility theory

3.1. Belief function fusion: overview

In the belief function theory or Dempster‐Shafer (DS) evidence theory formalism [5, 6], both uncertainty and imprecision can be represented, using belief functions and plausibility obtained from a mass function. The mass allocated to a proposition A corresponds to a part of the initial unitary amount of belief, which supports that the solution is exactly in A. It is thus defined as a function m from 2^U into [0, 1], with U being the decision space, also called full set or frame of discernment. Usually, the following constraints are imposed:

$m(\Theta )=0,$E1

${\displaystyle \sum _{A\subseteq \Theta }m(A)}=1.$E2

Not only the singletons of U but also any combination of possible propositions/decisions from the decision space can be quantified in this framework. This aspect represents one of the key advantages of the DS theory. As a matter of fact, this possibility allows for a rich and flexible modelling, which can fit to a wide range of situations, which are occurring typically in image fusion in particular. For example, the belief function theory can be successfully applied to situations that include partial or total ignorance, partial reliability, confusion between some classes (in only one or in several information sources), etc. [3, 15, 20–22].

In the DS framework, masses assigned by different sources (e.g. classifiers) are combined by the orthogonal rule of Dempster [6]:

$m_{ij}(S)={\displaystyle \sum _{\begin{array}{c}k,l\\ A_k\cap B_l=S\end{array}}{m}_i(A_k)\cdot m_j(B_l)}$E3

where S is any subset of the full set, while m_i and m_j are masses assigned by measures i and j, and their focal elements are A₁, A₂,…, A_p and B₁, B₂,…, B_q, respectively [2].

As discussed in Ref. [2], Dempster’s rule is commutative and associative, meaning that it can be applied repeatedly, until all measures are combined, and that the result does not depend on the order used in the combination. After the combination in this unnormalized form [23], the mass that is assigned to the empty set:

$m_{ij}(\varnothing )={\displaystyle \sum _{\begin{array}{c}k,l\\ A_k\cap B_l=\varnothing \end{array}}{m}_i(A_k)\cdot m_j(B_l)}$E4

can be interpreted as a measure of conflict between the sources. It can be directly taken into account in the combination as a normalization factor. It is very important to consider this value for evaluating the quality of the combination: when it is high (in the case of strong conflict), the normalized combination may not make sense and can lead to questionable decisions [24]. Several authors suggest not normalizing the combination result (e.g. [23]), which corresponds to Eq. (3).

This fusion operator has a conjunctive behaviour. This means that all imprecision on the data has to be introduced explicitly at the modelling level, in particular in the choice of the focal elements. For instance, ambiguity between two classes in one source of information has to be modelled using a disjunction of hypotheses, so that conflict with other sources can be limited and ambiguity can be possibly solved during the combination.

From a mass function, we can derive a belief function:

$\forall A\in 2^{\Theta },Bel(A)={\displaystyle \sum _{B\subseteq A,B\ne \varnothing }m(B)}$E5

as well as a plausibility function:

$\forall A\in 2^{\Theta },Pls(A)={\displaystyle \sum _{B\cap A\ne \varnothing }m(B)}.$E6

After the combination, the final decision is usually taken in favour of a simple hypothesis using one of several rules [25]: for example, the maximum of plausibility (generally over simple hypotheses), the maximum of belief, the pignistic decision rule [26], etc.

For some applications, such as humanitarian demining, it may be necessary to give more importance to some classes (e.g. mines, since they must not be missed) at the decision level. Then maximum of plausibility can be used for the classes that should not be missed and maximum of belief for the others [27].

3.2. Fuzzy and possibilistic fusion: overview

In the framework of fuzzy sets and possibility theory [4, 28], the modelling step consists in defining a membership function to each class or hypothesis in each source, or a possibility distribution over the set of hypotheses in each source. Such models explicitly represent imprecision in the information, as well as possible ambiguity between classes or decisions.

For the combination step in the fusion process, the advantages of fuzzy sets and possibilities rely on the variety of combination operators, which may deal with heterogeneous information [18]. As stated in Ref. [2], among the main operators, we find t‐norms, t‐conorms, mean operators, symmetrical sums and operators taking into account conflict between sources or reliability of the sources. We do not detail all operators in this chapter, but they can be easily found in the literature, with a synthesis in Ref. [29].

We classify these operators with respect to their behaviour (in terms of conjunctive, disjunctive and compromise [18]), the possible control of this behaviour, their properties and their decisiveness, which proved to be useful for several applications [29]. It should be noted that, unlike other data fusion theories (e.g. Bayesian or Dempster‐Shafer combination), fuzzy sets provide a great flexibility in the choice of the operator that can be adapted to any situation at hand. In particular, nothing prevents using different operators for different hypotheses or different sources of information.

An advantage of this approach is that it is able to combine heterogeneous information, which is usually the case in multi‐source fusion (as in both examples developed in the next sections), and to avoid to define a more or less arbitrary and questionable metric between pieces of information issued from these images, since each piece of information is converted in membership functions or possibility distributions over the same decision space.

Decision is usually taken from the maximum of membership or possibility values after the combination step. Constraints can be added to this decision, typically for checking for the reliability of the decision (Is the obtained value high enough?) or for the discrimination power of the fusion (Is the difference between the two highest values high enough?). Local spatial context can be used to reinforce or modify decisions [2].

4. Close‐range mine detection

4.1. Measures

From the data gathered by the sensors, a number of measures are extracted [15] and modelled using the two approaches [2]. These measures concern the following:

• the area and the shape (elongation and ellipse fitting) of the object observed using the IR sensor,

• the size of the metallic area in MD data and

• the propagation velocity (thus the type of material), the burial depth of the object observed using GPR and the ratio between object size and its scattering function.

Although the semantics are different, similar information can be modelled in both possibilistic and belief function models. The idea here is to design the possibility and mass functions as similar as possible and to concentrate on the comparison at the combination step.

The main difference relies in the modelling of ambiguity. The semantics of possibility leads to model ambiguity between two hypotheses with the same degrees of possibilities for these two hypotheses (e.g. Eqs. (7) and (12)). On the contrary, the reasoning on the power set of hypotheses in the belief function theory leads to assigning a mass to the union of these two hypotheses (e.g. Eqs. (9) and (14)).

Another distinction concerns the ignorance. It is explicitly modelled in the belief function theory, through a mass on the whole set (to guarantee the normalization of the mass function over the power set), while it is only expressed implicitly in the possibilistic model, through the absence of normalization constraint.

4.1.1. IR measures

The possibility degrees derived from elongation and ellipse‐fitting measures are represented by π_1I and π_2I, respectively [2]. Being related to shape regularity, they are defined for a regular‐shaped mine (MR), an irregular‐shaped mine (MI), a regular‐shaped non‐dangerous (i.e. friendly) object (FR) and an irregular‐shaped friendly object (FI).

In the belief function framework, the full set is: Θ = {MR, MI, FR, FI}. As elongation and ellipse fitting aim at distinguishing regular and irregular shapes, masses assigned by these two measures, m_1I and m_2I, are split between MR ∪ FR, MI ∪ FI and Θ.

Regarding elongation, we calculate r₁ as the ratio between minimum and maximum distances of bordering pixels from the centre of gravity (we work on threshold images) and r₂ as the ratio of minor and major axes obtained from second moment calculation. Using these two ratios, the following possibility degrees are derived:

${\pi }_{1I}(MR)={\pi }_{1I}(FR)=\min (r_1,r_2),$E7

${\pi }_{1I}(MI)={\pi }_{1I}(FI)=1-{\pi }_{1I}(MR).$E8

In the framework of belief functions, for this measure, masses are defined as follows:

$m_{1I}(MR\cup FR)=\min (r_1,r_2),$E9

$m_{1I}(MI\cup FI)=\left|r_1-r_2\right|,$E10

and the full set takes the rest:

$m_{1I}(\Theta )=1-\max (r_1,r_2).$E11

In the case of ellipse fitting, let A_oe is the part of object area that belongs to the fitted ellipse as well, A_o is the object area and A_e is the ellipse area. Then we define:

${\pi }_{2I}(MR)={\pi }_{2I}(FR)=\max {\displaystyle \left(0,\min {\displaystyle \left\{\frac{A_{oe}-5}{A_o},\frac{A_{oe}-5}{A_e}\right\}}\right)},$E12

${\pi }_{2I}(MI)={\pi }_{2I}(FI)=1-{\pi }_{2I}(MR).$E13

Masses for this measure are the following ones:

$m_{2I}(MR\cup FR)=\max {\displaystyle \left(0,\min {\displaystyle \left\{\frac{A_{oe}-5}{A_o},\frac{A_{oe}-5}{A_e}\right\}}\right)},$E14

$m_{2I}(MI\cup FI)=\max {\displaystyle \left\{\frac{A_e-A_{oe}}{A_e},\frac{A_o-A_{oe}}{A_o}\right\}},$E15

$m_{2I}(\Theta )=1-m_{2I}(MR\cup FR)-m_{2I}(MI\cup FI).$E16

Note that in cases where it is sure that all mines have a regular shape, the possibility degrees of MR can be reassigned to mines of any shape (M = MR∪MI) while the possibility degrees of MI can be reassigned to friendly objects of any shape (F = FR∪FI). Similarly, masses given to MR∪FR can be reassigned to M, while masses given to MI∪FI can be reassigned to F [2].

The area directly provides a degree π_3I (M) of being a mine. Namely, since the range of possible AP mine sizes is approximately known, the degree of possibility of being a mine is derived as a function of the measured size:

${\pi }_{3I}(M)=\frac{a_I}{a_I+0.1\cdot a_{I\text{min}}}\cdot exp\frac{-{{\displaystyle \left[a_I-0.5\cdot {\displaystyle \left(a_{I\text{min}}+a_{I\text{max}}\right)}\right]}}^2}{0.5\cdot {{\displaystyle \left(a_{I\text{max}}-a_{I\text{min}}\right)}}^2},$E17

where a_I is the actual object area on the IR image, while the approximate range of expectable mine areas is between a_Imin and a_Imax (for AP mines, it is reasonable to set a_Imin = 15 cm² and a_Imax = 225 cm²). On the contrary, friendly objects can be of any size, so the possibility degree is set to one whatever the value of the size:

${\pi }_{3I}(F)=1.$E18

The area/size mass assignment based on the above reasoning is given by

$m_{3I}(\Theta )=\frac{a_I}{a_I+0.1\cdot a_{I\text{min}}}\cdot exp\frac{-{{\displaystyle \left[a_I-0.5\cdot {\displaystyle \left(a_{I\text{min}}+a_{I\text{max}}\right)}\right]}}^2}{0.5\cdot {{\displaystyle \left(a_{I\text{max}}-a_{I\text{min}}\right)}}^2},$E19

$m_{3I}(FR\cup FI)=1-m_{3I}(\Theta ).$E20

4.1.2. MD measures

In reality, as explained in Ref. [2], MD data are usually saturated and data gathering resolution in the cross‐scanning direction is typically very poor, so the MD information used consists of only one measure, which is the width of the region in the scanning direction, w [cm]. As friendly objects can contain metal of any size, we define:

${\pi }_{\text{MD}}(F)=1.$E21

If there is some knowledge on the expected sizes of metal in mines (for AP mines, this range is typically between 5 and 15 cm), we can assign possibilities to mines as, for example:

${\pi }_{\text{MD}}(M)=\frac{w}{20}\cdot {\displaystyle \left[1-exp(-0.2\cdot w)\right]}\cdot exp\left(1-\frac{w}{20}\right).$E22

The corresponding mass functions are

$m_{\text{MD}}(\Theta )=\frac{w}{20}\cdot {\displaystyle \left[1-exp(-0.2\cdot w)\right]}\cdot exp{\displaystyle \left(1-\frac{w}{20}\right)},$E23

$m_{\text{MD}}(FR\cup FI)=1-m_{\text{MD}}(\Theta ).$E24

4.1.3. GPR measures

All three GPR measures provide information about mines [2].

In the of burial depth information (D), friendly objects can be found at any depth, while it is known that there is some maximum depth up to which AP mines can be expected, mainly due to their activation principles. However, due to soil perturbations, erosions, etc., mines can, by time, go deeper or shallower than the depth at which they were initially buried. In any case, they can rarely be found buried below 25 cm (D_max). Thus, for this GPR measure, possibility distributions π_1G for mines and friendly object can be modelled as follows:

${\pi }_{1G}(M)=\frac{1}{\cosh {(D/D_{\max })}^2},$E25

${\pi }_{1G}(F)=1.$E26

In terms of belief functions, the masses for this measure are

$m_{1G}(\Theta )=\frac{1}{\cosh {(D/D_{\max })}^2},$E27

$m_{1G}(FR\cup FI)=1-m_{1G}(\Theta ).$E28

Another GPR measure exploited here is the ratio between object size and its scattering function, d/k. Again, friendly objects can have any value of this measure, while for mines, there is a range of values that mines can have, and outside that range, the object is quite certainly not a mine:

${\pi }_{2G}(M)=exp{\displaystyle \left(-\frac{{{\displaystyle \left[(d/k)-m_d\right]}}^2}{2\cdot p^2}\right)},$E29

${\pi }_{2G}(F)=1,$E30

where m_d is the d/k value at which the possibility distribution reaches its maximum value (here, m_d = 700, chosen based on prior information), and p is the width of the exponential function (here, p = 400).

Similarly, the mass assignments for this measure are

$m_{2G}(\Theta )=exp{\displaystyle \left(-\frac{{{\displaystyle \left[(d/k)-m_d\right]}}^2}{2\cdot p^2}\right)},$E31

$m_{2G}(FR\cup FI)=1-m_{2G}(\Theta ).$E32

Finally, propagation velocity, v, can provide information about object identity. Here, we extract depth information on a different way than in the case of the burial depth measure [30] and we preserve the sign of the extracted depth. This information indicates whether a potential object is above the surface. If that is the case, the extracted v should be close to c = 3 × 10⁸ m/s, the propagation velocity in vacuum. Otherwise, if the sign indicates that the object is below the soil surface, the value of v should be around the values for the corresponding medium, for example, from 5.5 × 10⁷ to 1.73 × 10⁸ m/s in the case of sand:

${\pi }_{3G}(M)=exp{\displaystyle \left(-\frac{{(v-v_{\max })}^2}{2\cdot h^2}\right)},$E33

where v_max is the value of velocity which is the most typical for the medium (here, for sand, it is 0.5 × (5.5 × 10⁷ + 1.73 × 10⁸) = 1.14 × 10⁸ m/s, and for air, it is equal to c), and h is the width of the exponential function (here, h = 6 × 10⁷ m/s). Once again, friendly objects can have any value of the velocity:

${\pi }_{3G}(F)=1.$E34

The corresponding mass functions are

$m_{3G}(\Theta )=exp{\displaystyle \left(-\frac{{(v-v_{\max })}^2}{2\cdot h^2}\right)},$E35

$m_{3G}(FR\cup FI)=1-m_{3G}(\Theta ).$E36

4.2. Combination

The combination of possibility degrees, as well as of masses, is performed in two steps [2]. The first one applies to all measures derived from one sensor. The second one combines results obtained in the first step for all three sensors.

In the case of possibilities, only the combination rules related to mines are considered. The issue of combination rules for friendly objects is discussed in Section 4.4.

Let us first detail the first step for each sensor. For IR, since mines can be regular or irregular, the information about regularity on the level of each shape measure is combined using a disjunctive operator (here the max):

${\pi }_{1IM}=\max {\displaystyle \left({\pi }_{1I}(MR),{\pi }_{1I}(MI)\right)},$E37

${\pi }_{2IM}=\max {\displaystyle \left({\pi }_{2I}(MR),{\pi }_{2I}(MI)\right)}.$E38

The choice of the maximum (the smallest disjunction and idempotent operator) as a t‐conorm is related to the fact that the measures cannot be considered as completely independent from each other. Thus, there is no reason to reinforce the measures by using a larger t‐conorm, and the idempotent one is preferable in such situations. These two shape constraints should be both satisfied to have a high degree of possibility of being a mine, so they are combined in a conjunctive way (using a product). Finally, the object is possibly a mine if it has a size in the expected range or if it satisfies the shape constraint, hence the final combination for IR is

${\pi }_I(M)={\pi }_{3I}(M)+{\displaystyle \left[1-{\pi }_{3I}(M)\right]}\cdot {\pi }_{1IM}\cdot {\pi }_{2IM}.$E39

The conjunction in the second term guarantees that π_I(M) is in [0,1].

In the case of GPR, it is possible to have a mine if the object is at shallow depths and its dimensions resemble a mine and the extracted propagation velocity is appropriate for the medium. Thus, the combination of the obtained possibilities for mines is performed using a t‐norm, expressing the conjunction of all criteria. Here the product t‐norm is used:

${\pi }_G(M)={\pi }_{1G}(M)\cdot {\pi }_{2G}(M)\cdot {\pi }_{3G}(M).$E40

For MD, as there is just one measure used, there is no first combination step and the possibility degrees obtained using Eqs. (21) and (22) are directly used.

In the case of possibilities, the second combination step is performed using the algebraic sum:

$\begin{array}{lll}\pi (M)\hfill & =\hfill & {\pi }_I(M)+{\pi }_{MD}(M)+{\pi }_G(M)-{\pi }_I(M)\cdot {\pi }_{MD}(M)-{\pi }_I(M)\cdot {\pi }_G(M)-{\pi }_{MD}(M)\cdot {\pi }_G(M)\hfill \\ \hfill & \hfill & +{\pi }_I(M)\cdot {\pi }_{MD}(M)\cdot {\pi }_G(M),\hfill \end{array}$E41

leading to a strong disjunction [18, 29], as the final possibility should be high if at least one sensor provides a high possibility, reflecting the fact that it is better to assign a friendly object to the mine class than to miss a mine [2].

In the belief function framework, for IR and GPR, masses assigned by the measures of each of the two sensors are combined by Dempster’s rule in unnormalized form (Eq. (3)). A general idea for using the unnormalized form of this rule instead of more usual, the normalized form is to preserve conflict [27], i.e. mass assigned to the empty set, Eq. (4). Here, a high degree of conflict would indicate that either there are several objects and the sensors, as detectors of different physical phenomena, do not provide information on the same object, or some sources of information are not completely reliable. Our main interest is in the possibility that sensors do not refer to the same object, as the unreliability can be modelled and resolved through discounting factors [3]. After combining masses per sensor, the fusion of sensors is performed, using Eq. (3) again. If the mass of the empty set after combination of sensors is high, they should be clustered as they do not sense the same object.

4.3. Comparison of the combination equations

For IR, based on Eqs. (6)–(20) and (39), it can be shown that

$P{l}_I(M)\le {\pi }_I(M).$E42

This is in accordance with the least commitment principle used in the possibilistic model [2], as usually done in this framework.

As far as MD is concerned, there is no difference since it provides only one measure.

In the case of GPR, based on the comparison of Eqs. (25) and (27), Eqs. (29) and (31), as well as Eqs. (33) and (35), we can conclude that Eq. (40) can be rewritten as

${\pi }_G(M)=m_{1G}(\Theta )\cdot m_{2G}(\Theta )m_{3G}(\Theta ).$E43

Furthermore, the application of the Dempster’s rule (Eq. (3)) to the mass assignments of the three GPR measures results in the fused mass of the full set for this sensor:

$m_G(\Theta )=m_{1G}(\Theta )\cdot m_{2G}(\Theta )m_{3G}(\Theta )$E44

which leads to

${\pi }_G(M)=m_G(\Theta ).$E45

This means that the ignorance is modelled as a mass on Θ in the belief function framework, while it privileges the class that should not be missed (M) in the possibilistic framework (i.e. the ignorance will lead to safely decide in favour of mines).

4.4. Decision

As the final decision about the identity of the object should be left to the deminer not only because his life is in danger but also because of his experience, the fusion output is a suggested decision together with confidence degrees [2].

In the case of possibilities, the final decision is obtained by thresholding the fusion result for M and providing the corresponding possibility degree as the confidence degree. As almost all possibility degrees obtained at the fusion output are either very low or very high, the selected regions having very low values of π(M) (below 0.1) are classified as F, and the ones with very high values (above 0.7) are classified as M. Only a few regions exist at which the resulting possibility degree for M has an intermediary value and there, as mines must not be missed, the decision is M. In the following, this decision approach is referred to as dec1.

An alternative (dec2) for the final decision‐making is to derive the combination rule for F as well, compare the final values for M and F and derive an adequate decision rule. Due to operation principles of GPR and MD, the measures of these two sensors can only give information where mines are possibly not. As they are non‐informative with respect to friendly objects, it is not useful to combine their possibility degrees for F. Thus, for deriving the final combination rule for F, π (F), we can rely only on IR, that is:

$\pi (F)={\pi }_I(F).$E46

In the case of IR, since friendly objects can be regular or irregular, we apply a disjunctive operator (the max) for each of the shape constraints. In order to be cautious when deciding F, we combine the two shape constraints and the area measure using a conjunctive operator. Taking into account of Eq. (18), this reasoning results in

$\pi (F)=\max ({\pi }_{1I}(F_R),{\pi }_{1I}(F_I))\cdot \max ({\pi }_{2I}(F_R),{\pi }_{2I}(F_I)).$E47

Thus, in this alternative way to derive decisions, in regions where IR gives an alarm, the decision rule chooses M or F depending on which one of the two has a higher possibility value, given by Eqs. (41) and (58), respectively. In other regions, at which IR does not give an alarm although at least one of the two other sensors gives an alarm, the decision is based on the fusion result for M, as in dec1.

In the case of belief functions, as shown in Ref. [15], usual decision rules based on beliefs, plausibilities [6] and pignistic probabilities [26] do not give useful results because there are no focal elements containing mines alone [27]. As a consequence, these usual decision rules would always favour friendly objects [2]. The underlying reason is that the humanitarian demining sensors are anomaly detectors and not mine detectors. In such a sensitive application, no mistakes are allowed so in the case of any ambiguity, much more importance should be given to mines. Hence, in Ref. [15], guesses G(A) are defined, where A∈{M, F, Ø}:

$G(M)={\displaystyle \sum _{M\cap B\ne \varnothing }m(B)},$E48

$G(F)={\displaystyle \sum _{B\subseteq F,B\ne \varnothing }m(B)},$E49

$G\text{(}\varnothing )=m(\varnothing ).$E50

In other words, the guess value of a mine is the sum of masses of all the focal elements containing mines, regardless their shape, and the guess of a friendly object is the sum of masses of all the focal elements containing nothing else but friendly objects of any shape, meaning that the guesses are a cautious way to estimate confidence degrees.

As the output of the belief function fusion module, the three possible outputs (M, F, conflict) are provided together with the guesses, for each of the sensors and for their combination.

For GPR, the focal elements are only F and Θ, so guesses for this sensor become simply:

$G_G(M)=m_G(\Theta ),$E51

$G_G(F)=m_G(F).$E52

From Eqs. (45) and (51), we conclude that for GPR, the possibility degree of a mine is equal to the guess of a mine:

${\pi }_G(M)=G_G(M).$E53

Furthermore, Eqs. (6) and (48) show that the guess of a mine is equal to its plausibility, while Eqs. (5) and (49) show that the guess of a friendly object is equal to its belief. This means that the relation given by Eq. (42) shows, actually, that for IR:

$G_I(M)\le {\pi }_I(M).$E54

4.5. Results

The proposed approach has been applied to a set of known objects, buried in sand, leading to 36 alarmed regions in total [2]: 21 mines (M), 7 placed false alarms (PF, friendly objects) and 8 false alarms caused by clutter (FN, with no object).

The results of the possibilistic fusion are very promising, since all mines are classified correctly with the proposed approach, as can be seen in Table 1. The numbers given in the parenthesis indicate the number of regions selected in the pre‐processing step for further analysis, that is, measure extraction and classification. Regarding the combination operators, the results given in this table are based on the combination proposed in Section 4.2. (Eqs. (39)–(41)). The second fusion step is important, since a decision taken after the first step provides only 18 mines for IR, nine for MD and 13 for GPR. This illustrates the interest of combining heterogeneous sensors.

Classified correctly, possibility theory	Sensors			Fusion
	IR	MD	GPR	dec1	dec2
M (total: 21)	18 (18)	9 (9)	13 (13)	21 (21)	21 (21)
PF (total: 7)	0 (4)	0 (4)	2 (6)	1 (7)	2 (7)
FN (total: 8)	0 (1)	0 (0)	6 (7)	6 (8)	6 (8)

Table 1.

Correct classification results, possibilistic fusion.

The two decision rules, dec1 and dec2, give the same results for mines and friendly objects caused by clutter [2]. In the case of placed false alarms, two are correctly classified in the case of dec2, which is a slight improvement with respect to dec1 and the same result as for the belief function fusion, shown in Table 2. It is not surprising that the placed false alarms are not so well detected by any of the methods, since our model is designed in order to favour the detection of mines. This is also the type of results expected from deminers. Regarding correct classification of mines, the results of the possibilistic fusion are slightly better than those obtained using the belief function method (19 mines detected, Table 2). This is due to the increased flexibility at the combination level. False alarms with no objects are correctly identified by the belief function method (six out of eight), and it is the same result as for the two possibilistic decision rules. This result shows that a power of our methods is in decreasing the number of clutter‐caused false alarms without decreasing the result of mine detection, thanks to knowledge inclusion.

Classified correctly, belief functions	Sensors			Fusion
	IR	MD	GPR
M (total: 21)	10 (18)	9 (9)	13 (13)	19 (21)
PF (total: 7)	3 (4)	0 (4)	1 (6)	2 (7)
FN (total: 8)	0 (1)	0 (0)	6 (7)	6 (8)

Table 2.

Correct classification results, belief functions.

All results have been obtained with the models proposed in Section 4.1., with the same parameters. Note that although the general shapes of the possibility distributions are important and have been designed based on prior knowledge, they do not need to be estimated very precisely, and the results are robust to small changes in these functions. What is important is that the functions are not crisp (no thresholding approach is used) and that the rank is preserved (e.g. an object with a measure value outside of the usual range should have a lower possibility degree than an object with a typical measure value). Two main reasons explain the experienced robustness: (i) these possibility distributions are used to model imprecise information, so they do not have to be precise themselves and (ii) each of them is combined in the fusion process (Section 4.2.) with other pieces of information, which diminishes the importance and the influence of each of them.

Analysis regarding the robustness of the choice of the operator is also performed (within a class corresponding to the type of reasoning we want to achieve) [2]. Different operators within the same family have been tested, leading to the maximization and minimization of the possibility degrees of mines, thus being the safest and the least safe situations from the point of view of mine detection. The results obtained show that the model is robust indeed: all mines are detected in the second step, for all fusion schemes.

Differences between the results of Tables 1 and 2 can be formally explained as discussed in Section 4.3. For GPR, Eq. (53) explains why the results are the same for the two fusion approaches. In the case of IR, Eq. (54) indicates that the possibilistic approach would favour mines more than the belief function approach, which is indeed the case here.

5. Conclusion

Fusion approaches for close‐range humanitarian mine detection are presented and compared. These approaches are based on the belief functions as well as on the fuzzy/possibility theory. The differences at the combination step are mainly highlighted in this comparison. The modelling step is performed according to the semantics of each framework, but the designed functions are as similar as possible, so as to enhance the combination step. Different fusion operators are tested, depending on the information and its characteristics. An appropriate modelling of the data along with their combination in a possibilistic framework leads to a better differentiation between mines and friendly objects. The decision rule is designed to detect all mines, at the price of a few confusions with friendly objects. This is a requirement of this sensitive application domain (mines must not be missed). Still the number of false alarms remains limited in our results. The robustness of the choice of the operator is also tested, and all mines are detected for all fusion schemes. The proposed modelling is flexible enough to be easily adapted to the introduction of new pieces of information about the types of objects and their characteristics, as well as of new sensors.

The work shown in this chapter is useful in many other applications, even in quite different domains, and constitutes thus a large set of methods and tools for both research and applicative work. The developed schemes have a noticeable variety and richness and constitute a real improvement over existing tools.

Acknowledgments

The author of this chapter would like to thank Prof. Isabelle Bloch (Télécom ParisTech, Paris, France) for her long‐term help and support. The author also thanks the Belgian Ministry of Defence for its financial support and the TNO Physics and Electronics Laboratory (The Hague, the Netherlands) for the permission to work on the data gathered on their test facilities within the Dutch HOM‐2000 project. Parts of this chapter were previously published in Milisavljevic et al. [2].