Open access peer-reviewed chapter

Temporal MCDA Methods for Decision-Making in Sustainable Development Context

Written By

Anissa Frini, Sarah Benamor and Bruno Urli

Submitted: 06 May 2019 Reviewed: 28 November 2019 Published: 13 February 2020

DOI: 10.5772/intechopen.90698

From the Edited Volume

Sustainability Concept In Developing Countries

Edited by Surendra N. Kulshreshtha

Chapter metrics overview

715 Chapter Downloads

View Full Metrics

Abstract

Public decision-making problems are more and more complex in a context where decisions have to be made based concurrently on economic, social, and environmental considerations. In this context, decisions need to be evaluated in the short, medium, and long term because their planning horizons are usually of several years or even decades. A literature review on MCDA methods used in the sustainable development (SD) context shows that most MCDA methods used are static and existing research does not propose any aggregation framework for temporal assessment of actions. In the last 5 years, development of temporal MCDA has witnessed the interest of some researchers. However, the latest developments remain limited, and only a few research studies offer aggregation frameworks for multi-period settings. This paper presents two recent temporal MCDA methods that were applied in SD context. The first is MUPOM method which demonstrates how outranking methods, based on concordance-discordance principles, can be generalized to processing temporal impacts of decisions. The second, named PROMETHEE-MP, consists of a multi-period generalization of PROMETHEE under random uncertainty.

Keywords

  • multi-criteria decision aid (MCDA)
  • multi-period evaluations
  • outranking methods
  • sustainable development
  • PROMETHEE
  • MUPOM

1. Introduction

Decision-making processes today evolve in a context where sustainability is an important issue. Decisions have to be made while concurrently evaluating their economic, social, and environmental consequences. The most quoted definition of sustainable development (SD) is that of the report of Brundtland Commission [1] entitled “Our Common Future,” where sustainable development is defined as “development that meets the needs of the present without compromising the ability of future generations to meet their own needs. This definition contains two key concepts: (i) the concept of needs, in particular the essential needs of the world’s poor, to which overriding priority should be given; and (ii) the idea of limitations imposed by the state of technology and social organization on the environment’s ability to meet present and future needs.” This definition indicates the three main pillars of sustainable development, i.e., economic growth, environmental protection, and social equality. Secondly, it puts emphases on the long-term vision associated with sustainable development. In fact, decision processes should take into account not only the immediate but also the future consequences of decisions in order not to compromise future generations. In such context, decisions are generally ill-defined, the impacts of decisions are uncertain and often difficult to measure, and the acceptability of decisions is more difficult to attain. And so, the need for using structured methods and novel approaches to support sustainable decisions has emerged.

A state-of-the-art survey on sustainable decision prioritization [2] shows multi-criteria decision aid (MCDA) methods are the most popular approach to support sustainable decisions. These methods enable the simultaneous consideration of conflicting criteria as it occurs in a real-world problem under sustainability imperatives. However, although sustainable development tries to reach a balance between the evaluations of actions in the short and the long term, most articles surveyed in [2] did not investigate the long-term perspective related to sustainable development. Only very recently have some researchers proposed novel temporal MCDA methods for application in SD context. But, the state of the art remains limited, and only a few research studies offer temporal aggregation frameworks.

This paper presents two novel temporal MCDA methods that were applied in SD context. The first is MUPOM method (MUlti-criteria multi-Period Outranking Method) which demonstrates how outranking methods can be used in processing the temporal impacts of decisions. The second method is named PROMETHEE-MP and consists of a temporal generalization of PROMETHEE in a context of random uncertainty. This paper is organized as follows: Section 2 presents the previous work. Section 3 proposes a formulation for decision-making problem in SD context. Sections 4 and 5 expose the MUPOM and PROMETHEE-MP methods. Section 6 provides an illustration of these two methods on the same case study. Finally, Section 7 concludes the paper.

Advertisement

2. Previous work

Despite the importance of temporal (multi-periods) evaluation of actions for sustainable decisions, only a few articles have dealt with this aspect. Some authors consider the long-term effects as a criterion [3, 4], while others use scenario planning and predictive techniques or fuzzy modeling to deal with future unknowns [5]. In [6], the long-term effects are discounted, and in [4] they are roughly and qualitatively assessed. Very recently, some temporal extensions of MCDA methods have been developed [3, 6, 7, 8, 9, 10, 11]. In a forest management context, the long-term impacts were addressed as a specific criterion [3], and the local community was asked to evaluate it. In [6], the authors proposed a sustainable environmental management system (SEMS) where actions are ranked using ELECTRE III. The authors indicate that special care was taken in the assessment of criteria and that expected short- and long-term consequences were considered but without any explanation on how this was achieved. In [9], a multi-period multi-criteria method based on adapting TOPSIS to temporal context is proposed. But, compensation between the decision criteria on which TOPSIS rely (as scoring methods) is not appropriate for sustainability. In [10], authors generalize PROMETHEE to temporal setting. The weighted mean is applied for aggregation of the net flow scores over the periods, and then the method is compensatory. Another PROMETHEE-based model was published in [11] to assess the long-term impact of energy supply technologies. In this research work, different criteria weights were considered depending on the life cycle steps (from introduction to saturation of the market).

The literature review presented here shows a limited state of the art and an as yet largely undeveloped research area on multi-period aggregation. As discussed earlier, compensation is the main issue behind the few existing temporal proposals. We believe outranking methods are more suitable for sustainable decision problems because of their level of compensation (partial or non-compensatory), their use of thresholds, and their use of different types of data/criteria (qualitative and quantitative) without the need for normalization. To the best of our knowledge, research on developing temporal outranking methods is not well advanced. In this context, we started a research program to develop temporal outranking MCDA methods. In 2019, our research team proposed a generalization of outranking methods to temporal context and show how they can be applied for processing temporal impacts of decisions [7, 8]. Frini and Benamor [7] propose the first outranking method for multi-period (temporal) evaluations of actions called MUPOM. Based on pairwise comparisons, outranking relations and a measure of distance between preference relations, MUPOM accommodates the requirements of sustainable development discussed earlier and supports decisions that comply with the long-term vision related to SD context. Besides, existing research works dealing with MCDA methods under uncertainty are developed for static MCDA methods [12, 13, 14, 15, 16, 17, 18]. In order to develop extend MCDA under uncertainty to temporal context, we proposed in [8] a temporal generalization of PROMETHEE in a context of uncertainty. In the rest of the present paper, two generalizations of the outranking methods MUPOM and PROMETHEE-MP are presented and the results of their application on the same case study compared. The next sections expose the main results of this research program.

Advertisement

3. Problem formulation

In order to formulate the problem, let us consider a set A of N candidate actions (a1,aN), a set T of K assessment periods (P1, …, PK), a set C of M criteria C1C2CM, a set Π of M criteria weights (π1, … πM), aset(α1,,αK) of the K relative importance of periods (P1, …, PK), and gjai the evaluation of an action ai on criterion j.

The following assumptions of the model are made. (i) All evaluations are evaluated in the future with no missing evaluations. (ii) Criteria weights may change over time. (iii) Criteria, preference functions, and thresholds can vary over time. (iv) Criteria C1C2CM are assumed to be independent.

Figure 1 displays the decision matrices for multi-period multi-criteria decision problems.

Figure 1.

Decision matrices for the considered decision problems.

Advertisement

4. MUPOM: multi-criteria multi-period outranking method

MUPOM (MUlti-criteria multi-Period Outranking Method) is a three-phase temporal outranking MCDA method. In Phase 1, multi-criteria aggregation is performed in order to obtain outranking and preference relations for each period and for each pair of actions. Then in Phase 2 and for each pair of actions, a measure of distance between preference relations is used for temporal aggregation of the preference relations obtained in Phase 1. A graph showing relations between all pairs of actions illustrates the results of this aggregation. Next, in Phase 3 an exploitation procedure is used to compute the performance of each action ai. The following subsections provide details on the three phases. A full version of the mathematical details of the method is provided in [7].

Figure 2 graphs the steps of the MUPOM method.

Figure 2.

Steps of the MUPOM method [7].

4.1 Phase 1: multi-criteria aggregation

Multi-criteria aggregation relies on pairwise comparisons and concordance-discordance principles. For each pair of actions, we compute the concordance index (resp. discordance index), which evaluates the extent to which the criterion agrees (does not agree) with the assertion “action ai is at-least as good as action ak.” Then, if a majority of the criteria support this assertion and if the opposition of the other criteria—the minority—is not “too strong,” action ai is declared to be at least as good as action ak. Strong and weak outranking relations are constructed at this step. Next, the obtained outranking relations are transformed into preference relations PQIRQ1P1. Thus, for each pair of actions and for each period, we obtain either a strict preference (P), weak preference (Q), indifference (I), incomparability (R), inverse weak preference (Q1), or inverse strict preference (P1). The multi-criteria aggregation is a four-step phase [7]:

Step 1.1: For each period t and for each pair of actions aiak, compute the concordance index Ctaiak.

Step 1.2: For each period t, for each pair of actions aiak, and for each criterion j, compute the discordance index Djtaiak.

Step 1.3: Construct the relational preference systems Staiak for each pair of actions aiak and for each period t using concordance and discordance thresholds. We deduce that action ai strongly outranks ak (aiSFak) or ai weakly outranks akaiSfak.

Step 1.4: For each period t and for each pair of actions aiak, convert the obtained outranking relations to preference relation RtaiakPQIRQ1P1 where P,Q,I,R refers, respectively, to strict preference, weak preference, indifference, and incomparability. We note aiP1ak for akPai, and aiQ1ak for akQai.

4.2 Phase 2: temporal aggregation

This phase consists of aggregating the preference relations obtained for each pair of actions and at each period (results of Phase 1). This aggregation is done using a measure of distance between preorders [19]. Thus, the aggregated preference relation which minimizes the distance with the preorders at each period is obtained. The temporal aggregation phase consists of three steps [7]:

Step 2.1: For each pair of actions (ai,ak) and at each period t, compute the distance between the preference relation Rtaiak resulting from Step 1.4 and each possible preference relation HPQIRQ1P1. This distance is noted ΔHRtaiak.

Step 2.2: Aggregate the distances obtained at each period into a mean distance ΦHaiak.ΦHaiak=t=1TαtΔHRtaiak where αt is the relative importance of period t.

Step 2.3: Assign to the pair of actions aiak the preference relation H, such as:

H=H/ΦH=minHPQIRQ1P1ΦHaiak

A graph representing relations between all pairs of actions displays the results.

4.3 Phase 3: exploitation

This phase consists of computing the performance of each action ai . Performance calculation is based on the number of actions that are preferred (strictly or weakly) to ai and those that ai are preferred to (strictly or weakly). The set of “best compromise” action(s) is then deduced based on the computed performance. This set contains the actions with the highest performance and those which are incomparable to them. Details on the exploitation phase are provided in [19].

MUPOM method has important contributions. First, it proposes a generalization of outranking methods based on ELECTRE principles (concordance, discordance, and credibility indexes) to multi-period and temporal settings. Consequently, the method supports partial preferences and partial rankings and confirms that the outranking methods can be generalized to temporal context. In practical terms, MUPOM provides valuable contributions for researchers and practitioners concerned with decision-making processes under sustainability. Beyond the financial dimension, it enables integration of social and environmental impacts in the short, medium, and long term. By taking into account immediate and future consequences of actions, it guarantees decisions are not made that compromise future generations.

Advertisement

5. PROMETHEE-MP: a generalization of PROMETHEE for multi-period evaluations under uncertainty

PROMETHEE-MP is a recently developed temporal outranking method that allows aggregation of multi-periods and uncertain evaluations. It consists of three phases. Phase 1 aggregates the criteria, at each period of the horizon, based on PROMETHEE outgoing and incoming flows and Monte Carlo simulations. Binary relations are computed for each pair of actions. Phase 2 consists of aggregating the binary relations obtained over the periods using the measure of distance between preorders [19] as is done with MUPOM. Finally, in Phase 3 the performance of each action aiis computed, based on the number of actions that are preferred (strictly or weakly) to ai and those that ai are preferred to (strictly or weakly). Figure 2 presents all the steps of PROMETHEE-MP. The following subsections provide details on the three phases. A full version of the mathematical details of the method is found in [8] (Figure 3).

Figure 3.

Steps of PROMETHEE-MP [8].

5.1 Phase 1: multi-criteria aggregation and Monte Carlo simulations

In Phase 1, the criteria at each period of the horizon are aggregated. The method looks at a representation of uncertainty with probability distributions for uncertain parameters (evaluations and weights) and uses Monte Carlo simulation to generate numerical values for each uncertainty scenario. In this illustration and without loss of generality, uniform distributions using intervals are simulated for each parameter and for each period t. For each scenario of uncertainty s, we generate from the interval a specific value for evaluations and weights. Then at each period t and for each scenario s, we use the PROMETHEE method, and we compute outgoing t,s+ai and incoming flows t,sai for each action ai. As part of the model, we propose a generalization of PROMETHEE III that associates an interval to the outgoing and incoming flows for each action and deduces a partial preorder for the actions. The multi-criteria aggregation and Monte Carlo simulation phase consists of these steps [8]:

Steps 1.1 and 1.2: For each period t, we conduct a Monte Carlo simulation s. Each simulation generates, for each criterion j, a specific evaluation of action ai noted gjt,sai in the interval [gjai,gj+ai]. Also, each simulation considers a different value for criteria weights for each criterion j, noted πjt,sai.

Step 1.3: For each scenario s, action i and period t, we apply PROMETHEE and compute outgoing and incoming flows t,s+ai and t,sai.

Step 1.4: In this step, the outgoing and incoming flow distributions are defined by computing the mean t+¯ai and t¯ai and the standard deviations σt+ai and σtai.

Step 1.5: The resulting interval limits of the outgoing and incoming flows max,t+ai, min,t+ai,max,tai,min,tai are deduced.

Step 1.6: Preference relations St(ai,ak)ϵIPQR are deduced, depending on the values of max,t+ai, min,t+ai,max,tai,min,tai (see [8]).

5.2 Phase 2: temporal aggregation

Here the temporal aggregation procedure of MUPOM (Section 4.2) is used to aggregate the preference relations obtained over the periods in Step 1.6. As with the MUPOM method, the measure of distance between preorders developed in [19] is used.

5.3 Phase 3: exploitation

The temporal exploitation procedure of MUPOM (Section 4.3) is used in this phase. It computes the performance of each action ai based on the number of actions that are preferred (strictly or weakly) to ai and those that ai are preferred to (strictly or weakly).

Advertisement

6. Case study

In this section, MUPOM and PROMETHEE-MP are applied in the context of sustainable forest management. Sustainable forest management is a well-suited application context since it considers conflicting and heterogeneous criteria that should be assessed on about 150 years ahead. Actually, the selection of sustainable forest management options should arrive at a balance between biodiversity, soil and water conservation, forest productivity, socioeconomic benefits, and the population’s values and needs. Second, the impact of each decision has to be assessed long term over the period of forest regeneration (about 150 years).

Five options are for consideration: (a1) a reference option corresponding to the terms of the intervention standards regulation; (a2) a removal of protected areas for wood production; (a3) a specific plan for protecting the caribou habitat; (a4) a reforestation program; and (a5) a variable-level harvesting strategy which accelerates the harvest rate for the near periods. For evaluating these forest management options, we consider five criteria assessed every 5 years: (C1) the 5-year exploitable volume, (C2) index of caribou habitat, (C3) good habitat for moose, (C4) old forest areas, and (C5) carbon footprint. Figure 4 provides an example of the evolution over time (30 periods of 5 years) of criteria C2.

Figure 4.

Evolution over time of the criteria.

The AHP method was used to model the preferences in terms of criteria weights. A questionnaire was presented to an expert asking for pairwise comparisons between pairs of criteria and for the indifference, preference, and veto thresholds for each criterion, as well as the most appropriate criteria functions to be used with PROMETHEE. Also requested was the relative importance of periods. Tables 13 present the weights and an overview of the data used for option a3,respectively. Used weights and data for MUPOM are crisp and for PROMETHEE-MP are intervals.

CriteriaCrisp weights for MUPOMWeights intervals for PROMETHEE-MP
C15-year exploitable volume0.1443[0.137, 0.151]
C2Index of caribou habitat0.3064[0.291, 0.322]
C3Good habitat for moose0.1606[0.153, 0.169]
C4Old forest areas0.3063[0.291, 0.322]
C5Carbon footprint0.0825[0.078, 0.087]

Table 1.

Criteria weight intervals.

PeriodC1 (millions of m3)C2 (in [0, 1])C3 (thousands of hectares)C4 (thousands of hectares)C5 (tons of CO2)
P1390.591295361143,716,919
P2360.588297362145,123,580
P30190.569453262225,395,456

Table 2.

Decision matrix for option C used with MUPOM.

PeriodC1 (millions of m3)C2 (in [0, 1])C3 (thousands of hectares)C4 (thousands of hectares)C5 (tons of CO2)
P1[37.05, 40.95][0.561, 0.620][280.25, 309.75][342.95, 379.05][136,531,073; 150,902,765]
P2[34.20, 37.80][0.558, 0.617][282.15, 311.85][343.90, 380.10][137,867,401; 152,379,759]
P30[18.05, 19.05][0.540, 0.597][430.35, 475.65][248.9, 275.1][214,125,683; 236,665,229]

Table 3.

Decision matrix for option C used with PROMETHEE-MP.

To start, Phase 1 of MUPOM and PROMETHEE-MP is applied. Results are obtained in terms of binary relations (P, Q, I, R, Q1, P1) for each pair of actions and for each period. Table 4 shows the results of Phase 1 of MUPOM and PROMETHEE-MP for the pair (a1,a2). Then, in Phase 2, the results for each period are aggregated using the temporal aggregation procedure. For each pair of actions, Table 5 shows the aggregated relation which minimizes the distance between the relations obtained at each period and the set of preference relations (P, Q, I, P-1, Q-1).

PeriodPreference relation resulting from MUPOMPreference relation resulting from PROMETHEE-MPPeriodPreference relation resulting from MUPOMPreference relation resulting from PROMETHEE-MP
1P-1P-116RP-1
2P-1P-117Q-1P-1
3QP-118Q-1P-1
4QP-119Q-1P-1
5PQ-120Q-1Q-1
6PI21Q-1Q-1
7PI22Q-1Q-1
8RI23Q-1I
9RQ-124Q-1I
10QQ-125Q-1I
11RP26Q-1P-1
12RR27Q-1P-1
13QQ28Q-1P-1
14QR29Q-1I
15RQ-130Q-1P-1

Table 4.

Preference relations resulting from multi-criteria aggregation for pair (a1,a2).

PairAggregated relation with MUPOMAggregated relation with PROMETHEE-MPPairAggregated relation with MUPOMAggregated relation with PROMETHEE-MP
(a1, a2)Q-1P-1(a2, a4)RP
(a2 ,a1)QP(a4, a2)RP-1
(a1 ,a3)PP(a2, a5)RP-1
(a3 ,a1)P-1P-1(a5, a2)RP
(a1 ,a4)RQ-1(a3, a4)Q-1P-1
(a4 ,a1)RQ(a4, a3)QP
(a1 ,a5)RP-1(a3, a5)RP-1
(a5 ,a1)RP(a5,a3)RP
(a2 ,a3)QP(a4, a5)RP-1
(a3, a2)Q-1P-1(a5,a4)RP

Table 5.

Preference relations resulting from temporal aggregation for each pair of actions.

A graph representing relations between all pairs of actions illustrates the results. Phase 3 consists of exploiting the graph (Figures 5 and 6) and determining which action performs better. Results of MUPOM show {a2,a5} are the best compromise solutions, whereas PROMETHEE-MP shows {a5} as the only best compromise solution.

Figure 5.

Exploitation graph with MUPOM.

Figure 6.

Exploitation graph with PROMETHEE-MP.

Results show that in a deterministic context and without considering uncertainty, the two options a2 and a5 are both of best compromise and incomparable. However, when considering uncertainty on the evaluations and weights, only a5 is then of best compromise. It should first be noted that by modeling uncertainty on the evaluation and weights, as done with PROMETHEE-MP, the result is more robust because it takes into account the variability of evaluation over the intervals. However, comparison of results given by the two methods needs to take into account that they are not based on the same foundations. MUPOM uses concordance-discordance principles as ELECTRE methods do, while PROMETHEE-MP uses outgoing and incoming flows as PROMETHHE methods do.

In future research, it will be important to validate the findings of the two models by comparing the obtained results with those given by a panel of expert in forest management. A Delphi procedure could be applied in order to get the opinion of experts on the results. A level of 70% of agreement between experts will be considered. This validation process will confirm the quality of the results given by the method.

Besides, for stronger interpretation of results, future work will focus on applying the proposed methods on different horizons. For instance, in our case study, we can apply MUPOM and PROMETHEE-MP on the short-term horizon (aggregation of evaluations of the first 20 years), the medium term (aggregation of evaluations of year 20 to year 50), and finally the long term (aggregation of evaluations of year 50 to year 150). By doing so, we can compare the different results depending on the horizon and limit the effect of the aggregation. Results will show if the best compromised option on the whole horizon will differ or not from to the best compromised options in the short, medium, and long term.

Advertisement

7. Conclusion

This paper presents the main results of a recent research program on developing temporal outranking MCDA methods. It presents two generalizations of outranking methods to temporal context to show how outranking methods can be of use in processing the temporal impacts of decisions. The state of the art in this research area still remains limited, and such a proposal is valuable to support sustainable decision-making processes. This paper exposes two recent temporal outranking methods and displays the results of their application in SD context. The MUPOM method demonstrates how outranking methods and, more specifically, the ELECTRE concordance-discordance principles can be of use in processing temporal impacts of decisions. PROMETHEE-MP consists of a multi-period generalization of PROMETHEE under random uncertainty using Monte Carlo simulations. Their application on the same case study shows their applicability.

Advertisement

Funding

This research was funded by “Fonds de recherche du Québec – Societé et Culture.”

References

  1. 1. Brundtland GH. Report of the World Commission on Environment and Development: Our Common Future. Oxford: Oxford University Press; 1987
  2. 2. Kandakoglu A, Frini A, Benamor S. Multi-criteria decision making for sustainable development: A systematic review. Journal of Multi-Criteria Decision Analysis. 2019;26:202-251
  3. 3. Balana BB, Mathijs E, Muys B. Assessing the sustainability of forest management: An application of multi-criteria decision analysis to community forests in northern Ethiopia. Journal of Environmental Management. 2010;91:1294-1304. DOI: 10.1016/j.jenvman.2010.02.005
  4. 4. Betrie GD, Sadiq R, Morin KA, Tesfamariam S. Selection of remedial alternatives for mine sites: A multicriteria decision analysis approach. Journal of Environmental Management. 2013;119:36-46. DOI: 10.1016/j.jenvman.2013.01.024
  5. 5. Scholten L, Schuwirth N, Reichert P, Lienert J. Tackling uncertainty in multi-criteria decision analysis – An application to water supply infrastructure planning. European Journal of Operational Research. 2015;242:243-260. DOI: 10.1016/j.ejor.2014.09.044
  6. 6. Khalili NR, Duecker S. Application of multi-criteria decision analysis in design of sustainable environmental management system framework. Journal of Cleaner Production. 2013;47:188-198. DOI: 10.1016/j.jclepro.2012.10.044
  7. 7. Frini A, Benamor S. MUPOM: A multi-criteria multi-period outranking method for decision-making in sustainable development context. Environmental Impact Assessment Review. 2019;76:10-25. DOI: 10.1016/j.eiar.2018.11.002
  8. 8. Urli B, Frini A, Benamor S. PROMETHEE-MP: A generalization of PROMETHEE for multi-period evaluations under uncertainty. International Journal of Multi-Criteria Decision Making. 2019;8(1):13-37. DOI: 10.1504/IJMCDM.2019.10019420
  9. 9. Frini A, Benamor S. Making decisions in a sustainable development context: A state-of-the-art survey and proposal of a multi-period single synthesizing criterion approach. Computational Economics. 2017;52(2):341-385. DOI: 10.1007/s10614-017-9677-5
  10. 10. Benamar I, De Smet Y. An extension of PROMETHEE II to temporal evaluations. International Journal of Multi-criteria Decision Making. 2018;3-4:7. DOI: 10.1504/IJMCDM.2018.094371
  11. 11. Oberschmidt J, Geldermann J, Ludwig J, Schmehl M. Modified PROMETHEE approach for assessing energy technologies. International Journal of Energy Sector Management. 2010;4:183-212. DOI: 10.1108/17506221080000394
  12. 12. Corrente S, Figueira JR, Greco S. The SMAA-PROMETHEE method. European Journal of Operational Research. 2014;239(2):514-522. DOI: 10.1016/j.ejor.2014.05.026
  13. 13. Khalili-Damghani K, sadi-Nezhad S. A hybrid fuzzy multiple criteria group decision making approach for sustainable project selection. Applied Soft Computing. 2013a;13:339-352. DOI: 10.1016/j.asoc.2012.07.030
  14. 14. Khalili-Damghani K, sadi-Nezhad S. A decision support system for fuzzy multi-objective multi-period sustainable project selection. Computers and Industrial Engineering. 2013b;64:1045-1060. DOI: 10.1016/j.cie.2013.01.016
  15. 15. Baudry G, Macharis C, Vallée T. Range-based multi-actor multi-criteria analysis: A combined method of multi-actor multi-criteria analysis and Monte Carlo simulation to support participatory decision making under uncertainty. European Journal of Operational Research. 2018;264(1):257-269. DOI: 10.1016/j.ejor.2017.06.036
  16. 16. Van Der Kleij C, Hulscher S, Louters T. Comparing uncertain alternatives for a possible airport island location in the North Sea. Ocean and Coastal Management. 2003;46:1031-1047. DOI: 10.1016/j.ocecoaman.2003.09.001
  17. 17. Mirakyan A, De Guio R. Modelling and uncertainties in integrated energy planning. Renewable and Sustainable Energy Reviews. 2015;46:62-69. DOI: 10.1016/j.rser.2015.02.028
  18. 18. Ascough JC, Maier HR, Ravalico JK, Strudley MW. Future research challenges for incorporation of uncertainty in environmental and ecological decision-making. Ecological Modelling. 2008;219:383-399. DOI: 10.1016/j.ecolmodel.2008.07.015
  19. 19. Benamor S, Martel J-M. A new distance measure including the weak preference relation: Application to the multiple criteria aggregation procedure for mixed evaluation. European Journal of Operational Research. 2014;237:1165-1169. DOI: 10.1016/j.ejor.2014.03.036

Written By

Anissa Frini, Sarah Benamor and Bruno Urli

Submitted: 06 May 2019 Reviewed: 28 November 2019 Published: 13 February 2020