Priorities for Projects X, Y, and Z (normal synthesis).
Abstract
Resource allocation (RA) and supplier selection (SS) are two major decision problems regarding supply chain management (SCM). A supply chain manager may solve these problems by considering a single criterion, for instance, costs, customer satisfaction, or delivery time. Applying analytic hierarchy process (AHP), the supply chain manager may combine such criteria to enhance a compromised solution. This chapter presents AHP applications to solve two real SCM problems faced by Brazilian companies: one problem regarding the RA in the automotive industry and another one to SS in a chemical corporation.
Keywords
- analytic hierarchy process
- ranking reversal
- resource allocation
- supplier selection
- supply chain management
1. Introduction
Supply chain is the “global network used to deliver products and services from raw materials to end customers through an engineered flow of information, physical distribution, and cash” [1]. Therefore, many decision problems make up the supply chain management (SCM). Resource allocation (RA) and supplier selection (SS) are two major decision problems regarding SCM. A supply chain manager may solve these problems by considering a single criterion, for instance, costs, customer satisfaction, or delivery time. Applying Analytic Hierarchy Process (AHP), the supply chain manager may combine such criteria to enhance a compromised solution.
Foundations of AHP came back from the 1970s [2]. Originally, AHP consisted in hierarchy structuring, relative measurement (pairwise comparisons between criteria and between alternatives), and distributive synthesis (priorities are normalized, i.e., they sum equal to one).
One great advance in AHP practice is the “absolute measurement”, also known as “ratings” [3]. In absolute measurement, alternatives are compared with standard levels, instead of pairwise compared. Relative measurement has been most applied than ratings, even with software packages including ratings [4, 5]. Since in relative measurement alternatives must be pairwise compared, their number must be less than or equal to nine, that is, “seven, plus or minus two” [6]. In absolute measurement there is no bound for the set of alternatives. Another advantage from using ratings is the opportunity to avoid biases. With alternatives being compared with each other, two by two (relative measurement), some historical trends could be kept in mind. Comparing alternatives with a standard (absolute measurement) seems to provide a less partial or unbiased measurement.
Another advance for original AHP comes with the “ideal synthesis” [7]. With ideal synthesis, priorities are not normally distributed. That is, the sum of priority vectors components will not be equal to one. In this mode, the highest priority regarding each criterion will be equal to one. Normalizing priorities creates a dependency among priorities. However, if an old alternative is deleted or if a new one is inserted normalized priorities can lead to illegitimate changes in the rank of alternatives, known as rank reversal (RR). RR was firstly associated with AHP in preliminary studies by Professor Valerie Belton at University of Cambridge [8].
This chapter presents AHP applications to solve SCM problems. Two real cases from Brazilian companies are presented, one case regarding to RA in the automotive industry and another one to SS in a chemical corporation. In both cases, AHP was applied with absolute measurement. However, in the first case, the normal synthesis was adopted; in the second case, ideal synthesis was applied. The first conclusion from these cases is that for RA, normal synthesis maybe proper than ideal synthesis; conversely, for SS, ideal synthesis maybe more indicated.
2. Ranking reversal and synthesis mode
To illustrate the concept of RR, let us consider a decision of project selection by a company. The decision criteria are Benefits (B), Opportunities (O) and Risks (R). After pairwise comparisons, the priorities for the criteria (B, O and R) are, respectively, 73%, 19%, and 8%. Project X will be the selected one, due its highest overall priority (Table 1).
Project | Benefits (73%) | Opportunities (19%) | Risks (8%) | Overall |
---|---|---|---|---|
X | 0.540 | 0.185 | 0.149 | 0.442 |
Y | 0.348 | 0.659 | 0.691 | 0.434 |
Z | 0.112 | 0.156 | 0.160 | 0.124 |
Let us now consider that the supplier responsible for Project Z unexpectedly discontinues its operations. So, this alternative must be deleted of the decision. Then, surprisingly, overall priority of Project Y becomes higher than Project X’s (Table 2).
Project | Benefits (73%) | Opportunities (19%) | Risks (8%) | Overall |
---|---|---|---|---|
X | 0.600 | 0.250 | 0.167 | 0.499 |
Y | 0.400 | 0.75 | 0.833 | 0.501 |
With the same set of criteria and alternatives, but with ideal synthesis, overall priority of Project X will be higher than Project Y’s, considering Project Z (Table 3), or not (Table 4).
Project | Benefits (73%) | Opportunities (19%) | Risks (8%) | Overall |
---|---|---|---|---|
X | 1 | 0.281 | 0.215 | 0.801 |
Y | 0.643 | 1 | 1 | 0.740 |
Z | 0.207 | 0.237 | 0.232 | 0.215 |
Project | Benefits (73%) | Opportunities (19%) | Risks (8%) | Overall |
---|---|---|---|---|
X | 1 | 0.281 | 0.215 | 0.801 |
Y | 0.643 | 1 | 1 | 0.740 |
As presented in Tables 1–4, when one alternative is pulled out from the decision, ideal synthesis preserves ranks; conversely, normal synthesis reverses ranks. RR also may occur when new alternatives are inserted, or even when the set of criteria is changed. Matter of fact, absolute measurement and ideal synthesis always preserve ranks [9]. However, it is important to note that RR can be legitimate. That is, RR has already occurred. Two examples of real world decisions with RR are the United States Presidential Election, in 2000, and the Election of Host City of the 2016 Summer Olympics, in 2009.
The United States Presidential Election, in 2000, was quite controversial. The main contesters were Al Gore, George W. Bush, and Ralph Nader (Table 5).
Contester | Party | Electoral votes | Popular votes |
---|---|---|---|
Al Gore | Democratic | 266 | 48.9% |
George W. Bush | Republican | 271 | 47.9% |
Ralph Nader | Green | 0 | 2.7% |
Bush won with 271 electoral votes against 266 votes for Gore. It was the only fourth time, in 54 presidential elections, that the electoral vote winner failed to win also by popular vote. However, this is not an RR situation, because the set of alternatives was unchanged. An RR could have happened if Nader had quit. That is, most of Nader’s popular votes could go to Gore, in a case of Nader deletion from Table 5. For this reason, Nader was accused of spoil the Gore presidency [10].
During the 121st International Olympic Committee Session, Rio de Janeiro was selected as the host city of the 2016 Summer Olympics. Chicago, Madrid, and Tokyo were the other applicant cities (Table 6). On the first round, Madrid had more votes, Rio was the second, Tokyo was the third, and Chicago, with fewer votes, was eliminated. From the second round, Rio had more votes, then, an RR occurred, and, for the first time, South America will host the Summer Olympics.
City | Round 1 | Round 2 | Round 3 |
---|---|---|---|
Chicago | 19.1% | ||
Madrid | 29.8% | 30.5% | 32.6% |
Rio de Janeiro | 27.7% | 48.4% | 67.4% |
Tokyo | 23.4% | 21.1% |
Depending on the type of measurement and synthesis, ranks can be preserved or reversed with AHP. Nevertheless, the main discussion is the legitimacy of RR for the decision. For instance, RR may be avoided for a president election. After all, it is not only a pair of persons (the nominee for president and the running mate) who are being elected. With the candidate, also his ideas, political orientation (conservationist or reformist, etc.), and a whole party is being selected for a four-year term. For this reason, in countries like Brazil, a two round election is adopted for presidential elections.
In another instance, for the selection of the host city for a major event, RR can be acceptable. At first, it may sound strange: X is preferred among X, Y and Z, but Y is preferred between X and Y. What happened? Who preferred Z also preferred Y than Z. In this case, RR will be legitimate. Since, AHP is a method that allows RR, its application than will be proper than other methods which not allow RR.
In Sections 3 and 4, two SCM problems are presented. For the first one, RR will not be a problem: the normal synthesis is adopted. In the second case, RR must be avoided: the ideal synthesis is adopted.
3. AHP applied to SCM in an automobile plant
Supply chain is a network of supplier-customer companies connected by information and production flows, among other flows. For instance, a supply chain for an automobile production, beyond the car maker, or car assembler, may include auto parts manufacturers (Tiers 1 and 2 Suppliers), raw material providers, logistics providers, car dealers, and, as end customer, the car owner (Figure 1).
Let us consider a Brazilian manufacturer of vehicle frames, also known as chassis. This manufacturer has four plants (two in Brazil, one in Mexico, and a new one in Argentina) to closer supply its customers. These plants act as Tier 1 Suppliers for major car assemblers. In all these plants, the multiple source policy is adopted. That is, purchased items are provided eventually by more than one supplier. This policy has the main advantages of supplier competition and operational flexibility [11].
Blanking is the main process in the chassis production. Blank dies are specialized tools purchased in large lots to attend annual demands. The decision making is decentralized: buyers from each plant select suppliers for local requirements. Due to multiple supplier policy, a lot is often distributed in more than one supplier. Then, this is a resource allocation (RA) problem.
Suppliers are usually selected for a lot considering two main attributes: payment conditions and supplier loading. Usually, suppliers offering best payment conditions are selected. However, if a supplier has too many orders, then this supplier will be sidestepped. The distribution of a lot among suppliers is all done by a single buyer.
In one of the Brazilian plants, the Production Manager decides to apply the AHP, considering his team expertise to a new purchase of blank dies. The production management team, composed of three engineers, including the Production Manager, listed twelve more attributes desired for a supplier:
Capability, that is, supplier’s know-how.
Certification of the management system, according to international standard.
Quality, based on engineering tolerance.
Reliability, based on expected lifetime.
Services, post-sale technical support provided by suppliers.
Post-sales costs, differently charged by suppliers for post-sales support.
Flexibility, based on the supplier’s skills to change product specifications or lot size in orders.
Reaction, that is, supplier’s speed to incorporate these changes.
Sub-suppliers, that is, suppliers of the supplier.
Price truthful (does the die worth the charged price?).
Risk of the no-access supplier, on time, when a need for support emerges.
Historical delays from previous deliveries.
The fourteen attributes (the new twelve plus the old two) were grouped based on the Benefits-Opportunities-Costs-Risks (BOCR) model [12], resulting in a hierarchy for the set of attributes (Figure 2).
The Production Manager decided that each major aspect of BOCR should equally contribute for the decision. Therefore, the overall priorities for Benefits, Opportunities, Costs, and Risks were set in 25% each. Every member of the production management team made pairwise comparisons between attributes inside the aspects (Table 7).
Attribute | A1 | A2 | A3 | A4 | A5 |
---|---|---|---|---|---|
Capability (A1) | 1 | 7 | 1 | 1 | 3 |
Certification (A2) | 1/7 | 1 | 1/7 | 1/7 | 1/7 |
Quality (A3) | 1 | 7 | 1 | 1 | 1 |
Reliability (A4) | 1 | 7 | 1 | 1 | 3 |
Services (A5) | 1 | 1/7 | 1 | 1/3 | 1 |
Comparisons made by every engineer were individually aggregated, by geometrical mean, since they are willing their preferences for the same organization [13]. This procedure results in an aggregated comparisons matrix (Table 8).
Attribute | A1 | A2 | A3 | A4 | A5 | Priority |
---|---|---|---|---|---|---|
Capability (A1) | 1 | (392)1/3 | (6)1/3 | (1/6)1/3 | (6)1/3 | 26% |
Certification (A2) | (1/392)1/3 | 1 | (1/280)1/3 | (1/567)1/3 | (315)1/3 | 3% |
Quality (A3) | (1/6)1/3 | (280)1/3 | 1 | (1/15)1/3 | (6/5)1/3 | 17% |
Reliability (A4) | (6)1/3 | (567)1/3 | (15)1/3 | 1 | (8/3) 1/3 | 35% |
Services (A5) | (1/6)1/3 | (315)1/3 | (5/6) 1/3 | (3/8) 1/3 | 1 | 19% |
The local priorities for all attributes can be obtained normalizing the right eigenvector for the aggregated comparison matrices. The overall priorities for the attributes are obtained weighting local priorities by 25% (Table 9).
Attribute | Overall priority |
---|---|
Capability (A1) | 7% |
Certification (A2) | 1% |
Quality (A3) | 4% |
Reliability (A4) | 8% |
Services (A5) | 5% |
Flexibility (A6) | 6% |
Reaction (A7) | 13% |
Sub-suppliers (A8) | 6% |
Payment conditions (A9) | 17% |
Price truthfulness (A10) | 4% |
Post-sale costs (A11) | 4% |
No-access (A12) | 2% |
Historical delays (A13) | 5% |
Supplying load (A14) | 18% |
The production management team prioritized six potential suppliers with ratings, that is, relative measurement, by consensus. They prioritized the suppliers rating them in a 0–1 linear scale (Table 10). For Certification (A2), the priority value was binary: 1, when the supplier had a certified management system, or 0, otherwise.
Supplier | A1 | A2 | A3 | A4 | A5 | A6 | A7 | A8 | A9 | A10 | A11 | A12 | A13 | A14 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
S1 | 0.8 | 1 | 0.7 | 0.8 | 0.8 | 0.8 | 0.9 | 0.9 | 0.6 | 0.6 | 0.7 | 0.8 | 0.9 | 0.9 |
S2 | 0.7 | 0 | 0.8 | 0.7 | 0.8 | 0.9 | 0.7 | 0.8 | 0.7 | 0.8 | 0.8 | 0.8 | 0.7 | 0.8 |
S3 | 1 | 1 | 0.9 | 0.9 | 0.9 | 0.7 | 1 | 1 | 0.5 | 0.5 | 0.7 | 0.7 | 0.7 | 0.9 |
S4 | 0.8 | 0 | 0.9 | 0.9 | 0.8 | 0.9 | 0.8 | 0.8 | 0.8 | 0.9 | 0.8 | 0.9 | 0.9 | 0.8 |
S5 | 0.7 | 1 | 0.8 | 0.85 | 0.9 | 0.8 | 0.6 | 0.8 | 0.3 | 0.3 | 0.8 | 0.9 | 0.8 | 0.7 |
S6 | 0.5 | 0 | 0.6 | 0.6 | 0.6 | 0.5 | 0.2 | 0.6 | 1 | 1 | 0.5 | 0.2 | 0.5 | 0.5 |
S5 was the most expensive supplier (A9 = 0.3) and S6 was the cheapest one (A9 = 1). However, S6 will be not be selected, since it was relatively overloaded (A14 = 0.5). Usually, a buyer makes this decision. Sometimes, buyers make unpredictable decisions for production management.
Then, Production Manager decides to go on with AHP application. Next step is normalizing priorities for each attribute. Then, normalized priorities need to be weighted by criteria priorities (Table 9). This way, a decision matrix brings local priorities for each aspect (BOCR) and overall priorities (Table 11).
Supplier | Benefits (25%) | Opportunities (25%) | Costs (25%) | Risks (25%) | Overall |
---|---|---|---|---|---|
S1 | 17.4% | 19.7% | 15.4% | 19.6% | 18.0% |
S2 | 15.1% | 17.3% | 18.3% | 17.1% | 17.0% |
S3 | 20.4% | 20.9% | 13.3% | 18.5% | 18.3% |
S4 | 17.4% | 18.5% | 20.4% | 18.2% | 18.6% |
S5 | 17.9% | 15.5% | 9.4% | 16.2% | 14.7% |
S6 | 11.7% | 8.0% | 23.2% | 10.4% | 13.3% |
Production Manager decided to delete Suppliers 5 and 6 for the next supplying of blank dies. Then, the lot will be equally divided in four suppliers (S1–S4). The overall priorities for suppliers will change from (18.0%, 17.0%, 18.3%, 18.6%, 14.7%, 13.3%) to (25%, 25%, 25%, 25%). This is not a case of ranking reversal. If S5 and S6 were deleted in Tables 10 and 11, overall priorities will be (25.0%, 23.7%, 25.3%, 26.0%). Therefore, AHP application supported the Production Manager decision.
Production Manager, finally, communicates the decision to procurement management. The new role for buyers will be only to contact suppliers and purchase the blank dies. After all, production will be the user of the blank dies in their processes, not the buyers. No contest emerged, and the company was confident that select suppliers not only based in costs and risks, but also considering benefits and opportunities would be a better decision.
4. AHP applied to SCM by a chemical corporation
Outsourcing is an increased business practice. Instead of directly providing to customers, suppliers may contract third-party companies to perform non-core activities. Functions often outsourced include accounting, human resources, and marketing [14]. Logistics of finish goods to resellers is 100% outsourced by a chemical corporation in Brazil. That is, Third-Party Logistics (3PL) providers transport all items produced in the Brazilian plants to distribution centers. Land transport is the dominant mode of transport, and there are three 3PL providers. When a plant requires a new transportation, from the corporate office, located in Sao Paulo City, one of four buyers selects a 3PL provider. There is no structured method to select 3PL providers. Then, the Supply Chain Manager decided to apply AHP to select a 3PL for a new transportation order.
Desired attributes for a 3PL provider include [15, 16]:
Costs.
Financial performance.
Geographical location.
Information technology.
Operations performance.
Quality.
Responsiveness.
Risks.
Workforce.
The SCM team grouped 27 attributes, relevant for 3PL of the chemical corporation, in a BOCR model (Figure 3).The fourteen attributes regarding to Benefits were grouped as structural benefits or technological benefits.
As in the previous case of blank dies supplier selection, the chemical corporation’s SCM team decided that each major aspect of BOCR should equally contribute for the decision. Then, the overall priorities for Benefits, Opportunities, Costs, and Risks are also 25% each. Every member of SCM, including the Supply Chain Manager, compared the attributes inside the aspects. They input their judgments in an AHP web-based software. This software enables comparison values with two decimal points, as 2.33 (Table 12).
Costs | A34 | A35 | A36 |
---|---|---|---|
Performance extras (A34) | 1 | 2.33 | 0.70 |
Price (A35) | 1/2.33 | 1 | 0.40 |
Technical extras (A36) | 1/0.70 | 1/0.40 | 1 |
Comparisons made by every SCM member were individually aggregated, by geometrical mean, resulting in aggregated comparison matrices, which resulted in the local priorities for the attributes (Table 13).
Costs | A34 | A35 | A36 | Local priority |
---|---|---|---|---|
Performance extras (A34) | 1 | (0.122)1/5 | (0.233)1/5 | 25.9% |
Price (A35) | (1/0.122)1/5 | 1 | (2.238)1/5 | 39.8% |
Technical extras (A36) | (1/0.233)1/5 | (1/2.238)1/5 | 1 | 34.3% |
The local priorities for all attributes can be obtained normalizing the right eigenvector for the aggregated comparisons matrices. The overall priorities for the attributes A29–A41 are obtained weighting local priorities by 25%; for A15–A28, local priorities must be weighted by 12.5%, since local priorities for structural benefits and technological benefits are 50% each (Table 14).
Attribute | Overall priority |
---|---|
Chemicals expertise (A15) | 1.0% |
Financial health (A16) | 1.9% |
Managerial skills (A17) | 1.6% |
Qualified workforce (A18) | 1.4% |
Responsiveness (A19) | 1.7% |
Continuous improvement (A20) | 1.7% |
Location (A21) | 1.2% |
Operations flexibility (A22) | 1.2% |
Quality conformity (A23) | 0.8% |
Communication systems (A24) | 2.7% |
Customers complaint (A25) | 2.7% |
Electronic data interchange (A26) | 2.1% |
Information technology (A27) | 2.1% |
Quality of facilities (A28) | 2.9% |
Delivery reliability (A29) | 5.8% |
Delivery time (A30) | 5.4% |
Key performance indicators (A31) | 3.6% |
Projected investments (A32) | 3.9% |
Value added (A33) | 6.3% |
Performance extras (A34) | 6.5% |
Price (A35) | 9.9% |
Technical extras (A36) | 8.6% |
Information sharing (A37) | 4.3% |
Long-term relations (A38) | 4.9% |
Market share (A39) | 5.1% |
Order cancellation (A40) | 6.3% |
Services portfolio (A41) | 4.4% |
In this case, AHP must be applied with absolute measurement and ideal synthesis, since RR is not desired. The rating of 3PL providers will be based on a five-level scale from Poor to Excellent performance (Table 15).
Level | L1 | L2 | L3 | L4 | L5 | L6 | Priority |
---|---|---|---|---|---|---|---|
Excellent (L1) | 1 | 2 | 3 | 5 | 7 | 9 | 1 |
Between excellent and very good (L2) | 1/2 | 1 | 3 | 4 | 7 | 8 | 0.75 |
Very good (L3) | 1/3 | 1/3 | 1 | 5 | 6 | 8 | 0.49 |
Between very good and good (L4) | 1/5 | 1/4 | 1/5 | 1 | 6 | 8 | 0.25 |
Good (L5) | 1/7 | 1/7 | 1/6 | 1/6 | 1 | 3 | 0.10 |
Poor (L6) | 1/9 | 1/8 | 1/8 | 1/8 | 1/3 | 1 | 0.06 |
By consensus, the SCM team rated three 3PL providers, for every attribute (Table 16).
3PL provider | A15 | A16 | A17 | A18 | A19 | A20 | A21 | A22 | A23 | A24 | A25 | A26 | A27 | A28 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
S7 | L1 | L2 | L4 | L2 | L1 | L3 | L2 | L2 | L1 | L4 | L5 | L2 | L2 | L4 |
S8 | L1 | L4 | L1 | L3 | L3 | L2 | L3 | L3 | L2 | L3 | L3 | L3 | L3 | L5 |
S9 | L1 | L2 | L2 | L3 | L4 | L4 | L1 | L4 | L3 | L2 | L3 | L3 | L1 | L6 |
Local priorities regarding every criterion are obtained with the priorities of levels L1–L6. Overall priorities for 3PL providers (Table 17) are obtained weighting local priorities by the priorities of attributes.
3PL provider | Benefits | Opportunities | Costs | Risks | Overall |
---|---|---|---|---|---|
S7 | 0.55 | 0.75 | 0.25 | 0.54 | 0.52 |
S8 | 0.51 | 0.49 | 0.56 | 0.7 | 0.57 |
S9 | 0.55 | 0.42 | 0.53 | 0.65 | 0.54 |
Overall priorities for 3PL providers indicate medium performances for the suppliers. SP8, SP9 and SP7 are almost tied with respective overall priorities equal to 0.57, 0.54 and 0.52. However, these priorities were obtained with balanced priorities for the major aspects of BOCR model. The current Brazilian scenario, with economic recession, suggests unbalances favoring Costs and Risks. Then, after sensitivity analysis (Figure 4), SP8, SP9 and SP7 were prioritized in this order. That is, when available SP8 will be chosen as 3PL provider; SP7 will provide 3PL only when both SP8 and SP9 were not available.
The SCM team validated the results. The experts considered the AHP based model as applicable in practice. Besides the results are considered adequate, the steps of AHP were perceived as a better practice for supplier selection to logistics.
5. Conclusions
This chapter presents two applications of AHP to SCM. In both applications, absolute measurement was adopted instead of the original relative measurement. That is, in both applications the set of criteria was larger than usual AHP applications. With absolute measurement pairwise comparisons between suppliers are not necessary. This way, the effort for the AHP application is reduced. However, this is not the main reason for absolute measurement, in both cases. In the case of RA by an automobile plant, there was feeling that the decision must be impartial. That is, pairwise comparing Supplier S1 with S2 may invoke past deficiencies already overcome.
Another reason for absolute measurement, only for the SS by a chemical corporation, is the avoiding of RR. That is, for a problem of SS, RR seemed not justifiable. If Supplier S8 is preferred than S7 and S9, it must be preferred without S7 or S9 in the decision, and even if other Suppliers S10, S11, S12, …, all worst then S7 were inserted in the decision. Concluding: there is no space for RR in SS. For that reason, AHP must be applied to SS with absolute measurement and ideal synthesis.
On the other hand, adding new suppliers in an RA problem may change previous priorities of older suppliers. This way, RR can be legitimate for this problem. Then normal synthesis, and also, relative measurement, seems to be proper procedures for AHP application to RA.
It is important to state that this chapter presents some highlights based on only two cases. Obviously, more examples and real cases must be studied to generalize our conclusions. Perhaps the main contribution of this work is presenting that there are different ways to solve a problem. The way a problem is solved defines a decision, which may be irreversible, as the selection of a supplier to a specific order.
Acknowledgments
This research was financially supported by Sao Paulo Research Foundation (FAPESP) Grant No. 2013/03525-7
References
- 1.
Blackstone JJH, editor. APICS Dictionary. 14th ed. Chicago: APICS; 2013. - 2.
Saaty TL. A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology. 1977; 15 (3):234–281. DOI: 10.1016/0022-2496(77)90033-5 - 3.
Saaty TL. Absolute and relative measurement with the AHP. The most livable cities in the United States. Socio-Economic Planning Sciences. 1986; 20 (6):327–331. DOI: 10.1016/0038-0121(86)90043-1 - 4.
Creative Decisions Foundation. Super Decisions Software for Decision Making [Internet]. 2015. Available from: http://www.superdecisions.com [Accessed: 2016-01-06] - 5.
Incorporated Expert Choice. Comparison suite 5.40 [Internet]. 2015. Available from: http://comparion.expertchoice.com [Accessed: 2016-01-06] - 6.
Saaty TL, Ozdemir MS. Why the magic number seven plus or minus two. Mathematical and Computer Modelling. 2003; 38 (3–4):232–244. DOI: 10.1016/S0895-7177(03)90083-5 - 7.
Millet I, Saaty TL. On the relativity of relative measures – accommodating both rank preservation and rank reversals in the AHP. European Journal of Operational Research. 2000; 121 (1):205–212. DOI: 10.1016/S0377-2217(99)00040-5 - 8.
Belton V, Gear T. On a short-coming of Saaty’s method of analytic hierarchies. Omega. 1983; 11 (3):228–230. DOI: 10.1016/0305-0483(83)90047-6 - 9.
Saaty TL, Vargas LG, Whitaker R. Addressing with brevity criticism of the analytic hierarchy process. International Journal of the Analytic Hierarchy Process. 2009; 1 (1):121–134. DOI: 10.13033/ijahp.v1i2.53 - 10.
Herron MC, Lewis JB. Did Ralph Nader spoil Al Gore’s presidential bid? A ballot-level study of green and reform party voters in the 2000 presidential election. Quarterly Journal of Political Science. 2007; 2 (3):205–226. DOI: 10.1561/100.00005039 - 11.
Minner S. Multiple-supplier inventory models in supply chain management: a review. International Journal of Production Economics. 2003; 81–82 :265–279. DOI: 10.1016/S0925-5273(02)00288-8 - 12.
Wijnmalen DJD. Analysis of benefits, opportunities, costs, and risks (BOCR) with the AHP–ANP: a critical validation. Mathematical and Computer Modelling. 2007; 46 (7–8):892–905. DOI: 10.1016/j.mcm.2007.03.020 - 13.
Forman E, Peniwati K. Aggregating individual judgments and priorities with the analytic hierarchy process. European Journal of Operational Research. 1998; 108 (1):165–169. DOI: 10.1016/S0377-2217(97)00244-0 - 14.
Isıklar G, Alptekin E, Buyukozkanb G. Application of a hybrid intelligent decision support model in logistics outsourcing. Computers and Operations Research. 2007; 34 (12):3701–3714. DOI: 10.1016/j.cor.2006.01.011 - 15.
Jayaram J, Tan KC. Supply chain integration with third-party logistics providers. International Journal of Production Economics. 2010; 125 (2):262–271. DOI: 10.1016/j.ijpe.2010.02.014 - 16.
Qureshi MN, Kumar D, Kumar P. An integrated model to identify and classify the key criteria and their role in the assessment of 3PL services providers. Asia Pacific Journal of Marketing and Logistics. 2008; 20 (2):227–249. DOI: 10.1108/13555850810864579