Open access peer-reviewed chapter

Identification of Environmental Criteria for Selecting a Logistics Service Provider: A Step Forward towards Green Supply Chain Management

Written By

Patricija Bajec and Danijela Tuljak-Suban

Submitted: October 19th, 2015 Reviewed: February 16th, 2016 Published: June 30th, 2016

DOI: 10.5772/62533

Chapter metrics overview

2,519 Chapter Downloads

View Full Metrics


Green environmental performance increases the competitiveness of the supply chain. However, the greening of the supply chain depends on the manufacturer who drives the green initiative, as well as on all the members of the supply chain who take part in the process. The manufacturer’s attention has been largely focused on the environmental performance of the supplier and retailer, whereas logistics service providers have been somehow neglected. It is, in fact, the case is that logistics service providers have begun to play a critical role in supply chain management and could therefore significantly improve environmental sustainability. They have already undertaken a green initiative that unfortunately has rarely, if at all, been required by the manufacturer. The lack of requirements for logistics providers hinders the progress of a green initiative. To take a step forward towards green supply chain management, this chapter aims to introduce all the necessary criteria for the selection of a logistics service provider (LP), with an emphasis on environmental criteria. The environmental selection criteria, with all related subcriteria, were achieved on the basis of a systematic literature review. It has been found that buyers of logistics services still strive to minimize costs, expect quality logistics services, a well-positioned LP, all the while overlooking environmental issues. The most frequently applied environmental selection criteria are value-added reverse logistics services, followed by environmental expenditures, pollutants released, energy consumption, clean materials and energy use. The findings presented here are useful particularly for researchers, as issues regarding sustainable LP selection and its limitations are highlighted, related to selection criteria identification. These findings may be of less use to managers. However, future phases of this study, richer for the evaluation of logistics experts, will be much more applicable to buyers and providers of logistics services.


  • environmental logistics
  • environmental selection criteria
  • reverse logistics service provider
  • green supply chain management
  • logistics outsourcing

1. Introduction

Green supply chain management has recently been of increased interest mainly because of increased environmental awareness and significantly, economic motivation (corporate image, higher profits, marketing exposure) [1], which affect the competitiveness and differentiation [2] of the supply chain. Bacallan [3] even argued that some companies have become more competitive because of better environmental performance. An environmentally friendly supply chain depends on the eco-efficiency and synergy of all partners in the supply chain [4], from supplier to manufacturer, distributor and retailer to the customer. Manufacturers are one of the main drivers of the green initiative and play a crucial role in increasing the environmental performance of the supply chain [5]. A greening of the supply chain, therefore, depends on the requirements of the manufacturers who have consequently devoted considerable attention to green purchasing over the past decades [610]. Green product design has also been frequently studied in the literature by many researchers [1, 11], as well as green operations related to product manufacturing. The only activity that has been somewhat neglected in this respect is logistics. Most of the literature published on this topic has been directed towards transport, namely green transportation planning, use of intermodal solutions, information and communication technologies for managing green transport, etc. [1214]. The number of publications on this topic is not surprising as transportation is one of the most significant polluters of the environment. Much more surprising is the fact that external providers of logistics services are rarely the object of research. LPs actually provide transport and other logistics services for companies (so-called logistics outsourcing).

Logistics outsourcing is not a new phenomenon. It has become a necessity and not just a fad. Companies use logistics outsourcing for three reasons. The first reason is the importance of logistics for supply chain efficiency [15], second is the inability of companies to provide the high level of logistics services that is currently required and expected and the third reason is the difficulties inherent in focusing on the core business when also taking care of logistics services. Logistics outsourcing enables a higher quality of logistics services, lower flexibility and a higher level of differentiation. In addition, companies can focus on their core business, which is crucial to survive and contend in a global and competitive environment.

Accordingly, LPs are significant partners in the supply chain and who bear the same degree of responsibility for protecting the environment as other supply chain partners (suppliers, manufacturers, distributors, retailers, etc.). According to Cooper et al. [16], LPs have a very significant impact on the overall sustainability of the supply chain. They should, therefore, be subject to the green requirements of manufacturers. A brief literature review revealed the following, contradictory facts. First, the studies on buying logistics services (which are, in fact, quite limited) show that there is a lack of green demand on the part of companies buying logistics services [11, 17, 18]. LPs are still selected based on a “traditional” selection criteria, namely price, quality, provider status, while environmental criteria have received minimum attention [19]. Second, the logistics industry itself is focused on environmental initiatives. “Important commitments to environmental sustainability improvements have been made by LPs during the past several years” [14]. And third, green logistics depends on the buyer’s requirements and expectations of LPs. In summary, it would seem that the first and quite important step towards eco-efficiency has already been taken by LPs. Nevertheless, further actions towards sustainability are needed by the buyers of logistics services as well. The next challenge would, therefore, be to integrate environmental requirements into the manufacturer/logistics provider partnership, starting with the identification of relevant environmental criteria.

There is extensive literature that covers selection criteria in logistics outsourcing. Studies mainly focus on general selection criteria such as costs, service level and the status of the logistics service provider, while the environmental consideration is more or less neglected. Few studies address environmental criteria and we assert that special attention must be paid to this as well because of its important role in achieving a competitive business. The main objective of the chapter is, therefore, to introduce all the criteria for the selection of logistics service providers, focusing, in particular, on environmental criteria, which will be achieved by a systematic literature review. The research questions are as follows: (1) How common are articles on the selection of reverse LPs? How often are environmental issues taken into account when selecting a LP? (2) Which are the most commonly used environmental criteria? (3) Are they well-defined? (4) Do they cover all environmental aspects?

After the introduction, the rest of the chapter will be organized as follows: first, the logistics outsourcing selection process will be introduced. In the second part, the methodology of the literature review will be presented. Review results will be reported and critically analysed in the third section, followed by the conclusion.


2. Evaluation and selection of outsourcing provider

When a company identifies the need to outsource one or more services, the next step is to evaluate and select a suitable provider to meet its requirements. Outsourcing provider selection is one of the most important steps of the outsourcing process [20]. It is composed of a few different phases from determining selection criteria, making a list of possible providers and evaluating all possible providers to making the final selection of the most appropriate partner [21, 22]. The selection phase has a strong influence on the business performance of both outsourcing partners. Its success depends on the company’s knowledge of outsourcing methodology, appropriate communication between the buyer and outsourcing provider, a comparable request for proposal [23], the selection of appropriate criteria as well as the right decision-making method or methods [24].

Selection criteria (sometimes called attributes) are actually the requirements of the buyer from the outsourcing provider, covering all skills, knowledge, staff requirements, infrastructure and suprastructure requirements, etc. They represent a base for evaluating and selecting the best outsourcing provider [25]. Each criterion and its weight are absolutely dependent upon the buyer’s needs, the strategy and also the external environment [2628]. There is no general formula suitable for all cases. Criteria can also be further divided into various subcriteria.

Additionally, there are also many decision-making methods important for determining the optimal solution and their suitability changes from case to case. An individual approach is, therefore, required in each instance of outsourcing implementation. Even if the company chooses the correct decision-making method but has imprecise and inadequate criteria, the selection process will not be successful. Outsourcing will consequently not meet the expectations of either partner or could even be doomed to failure.

Criteria have been a frequently discussed topic in the relevant literature. Different authors use very different criteria and subcriteria to select a provider. However, the two most frequently mentioned selection criteria are cost savings and service quality, followed by operational capability and provider status. Environmental criteria appear only in recent studies. There have been periodic changes detected on the importance of criteria [26]. From 1994 to 1999, quality was the most important criteria. By 2003, price took first place [29]. Selection criteria could be classified into five main groups, namely (1) operational capability, (2) service level, (3) costs, (4) provider status and (5) environmental capability. Operational capability could be further divided into six subgroups (breadth of services, value-added services, technical and technological capability, optimization capability and information technology), service level into seven subgroups (quality, flexibility, responsiveness, network coverage, reliability, information exchange and risk management), costs into two subgroups (fixed price and variable price), provider status into six subgroups (culture recognition, financial stability, degree of reputation, experience, staff quality and empathy) and last, environmental capability into three subgroups (pollutants released, energy consumption, clean material and energy used, environmental expenditures, etc.).


3. Research methodology

After determining the need for a review, the first step is to set the research target and discuss some general facts related to the evaluation and selection process – which actually represents the first phase of a systematic literature review approach – Planning the review, and the next step is to explain its other two phases, namely Implementation of the review, representing the second phase and Reporting the results, representing the last phase [30].

The implementation phase began with a systematic search in the leading electronic databases, namely, Elsevier, Emerald, Taylor & Francis, Springer Wiley Online Library, and Inderscience Publisher, covering comprehensive journals that very frequently publish studies from the field of logistics, sustainable logistics, logistics outsourcing, supply chain management, etc. (e.g., The Journal of Multi-Criteria Decision Analysis, International Journal of Physical Distribution & Logistics Management, International Journal of Logistics Research and Applications, Supply Chain Management: An International Journal, International Journal of Logistics Research and Application: A Leading Journal of Supply Chain and Management, International Journal of Shipping and Transport Logistics, International Journal of Logistics Research and Applications and Operations and Supply Chain Management, International Journal of Production Research, Expert Systems with Applications, Journal of Applied Sciences, International Journal of Fuzzy Systems, Journal of Manufacturing Technology Management, Journal of Modelling and Management, Management Decision, Journal of Intelligent Manufacturing, The International Journal of Management Science, Journal of Applied Research and Technology and International Advances in Economic Research). Non-peer-reviewed and non-international publications were also included. Moreover, searches for conference proceedings, master’s theses, books and book chapters as well as personal requests were also conducted. Seventy-six percent of the articles related to the selection of LPs were published in journals, 13% in conference proceedings, 4% in book chapters or books and 4% in master’s theses (Figure 1).

Figure 1.

Distribution of publications over publication type.

The review was not limited by time frame for two reasons. Firstly, we wanted to demonstrate how the focus on selection criteria has increased over time and secondly, we wished to see which significant changes have occurred over the years. The search therefore covers articles published from 1999 to 2015. We have limited our review to studies written in English only.

Figure 2 presents a distribution of the articles over time. The oldest paper was published in 1999. The distribution is more peaked than a Gaussian distribution, since kurtosis is positive (0.47) and asymmetrical with a long tail to the left (lower values), since skew is negative (−0.78). The value of mean and median is 2010 and the value of mode is 2013. The highest growth, therefore, appeared between 2010 and 2015, when 63 articles were published. Moreover, 2013 is the year with the maximum number (17) of publications.

Figure 2.

Distribution of publications over time.

Defining the search terms signified the end of the implementation phase and started the process of selecting articles for the systematic review. The selection of articles consisted of two steps. First, we searched for the keywords “selection process” AND “selection criteria” AND “logistics outsourcing” and “logistics service provider” in electronic databases (e.g. Elsevier, Emerald, Taylor & Francis, Springer Wiley Online Library, Inderscience Publisher). This process resulted in 25 articles; three of them were excluded following our exclusion criteria. To find more so-called “grey publications” [31], we first searched Google Scholar and Web of Science, which added another 6 publications to the pool (N=31). Second, we traced references from the pool of publications which resulted in a further 15 publications. In summary, the first step generated 46 publications. As the defined search terms did not bring the desired number of publications, we decided to take a new, second step. The whole above written process was repeated, using additional search terms, namely “decision making methods” AND “selection of third party logistics providers” OR “selection of LP” AND “full name of method, acronym of method”. The reason for these keywords stems from the fact that the selection criteria are very often listed in the articles which are related to the decision-making method and not directly to selection criteria. This additional search was indeed very successful, if not time-consuming. Searching electronic databases led to an additional 33 papers; a manual search on Google Scholar and Web of Science added an additional 7 papers and tracing references gave us an additional 9 papers. Ultimately, our search resulted in 95 articles which were further analysed in the third and last phase, Reporting the result, presented in the next section.


4. Descriptive analysis

Each article from our pool of publications was analysed with respect to the selection criteria, in particular, to the environmental criteria, their classification, frequency of use, distribution over time interval and the limitations of these criteria.

4.1. Distribution of articles on selection of reverse LPs

The LP selection process has received a great deal of attention from researchers since 1994, when the first paper was published. Ninety-five publications directly or indirectly related to selection criteria were identified. The reason for such a surge in popularity is globalization that (1) increases the need to establish a more robust LP evaluation and selection process [21] and also (2) increases service requirements (selection criteria) because of a higher demand for logistics outsourcing, and consequently the complexity of the selection process. New approaches that handle multiple, and in most cases also conflicting, criteria have emerged as a result of this need.

In contrast, publications on the selection of LPs able to manage reverse logistics flow are rare. Only 11 of the 95 articles were published on this topic [3242]. Three were written by the same author and co-author. All the articles mentioned have been published in recent years, from 2008 to 2015, which clearly indicates the growing importance of green logistics.

4.2. Selection criteria applied for the logistics industry, with an emphasis on environmental criteria

LP selection is a multi-criteria problem that takes into consideration many qualitative and quantitative, tangible and intangible [43] criteria that are often in conflict with one another [23]. Some of these criteria are client-based and depend on specific customer requirements, and others are general and applicable in all cases [26, 43].

Our investigation revealed (1) the significantly diverse classification of selection criteria and subcriteria; (2) that some authors use a very detailed and reasonable classification of criteria and subcriteria, while the others are very unsystematic and in some cases superficial; (3) that the number of criteria vary from six to more than twenty and (4) that environmental criteria are very rarely used. The above-mentioned confusion caused us some difficulty in determining the proper classification of criteria and subcriteria. Nevertheless, we believe that we were able to devise a very useful classification of criteria and subcriteria for the logistics industry. We have managed to capture the largest possible number of criteria in a systematic manner that is neither too limited nor overly detailed. Criteria were, therefore, arranged in five groups, further divided into several subgroups. Figure 3 presents a summary of all frequently cited evaluation criteria and their subcriteria, followed by a detailed explanation of each.

Figure 3.

Various criteria and subcriteria for an LP selection.

4.2.1. Operational capability

Transport services

Transport management includes fleet management, technical and technological capabilities [42] and freight forwarding [41]. It refers to accurate or mistaken deliveries [44], frequency of transport services, transportation costs, etc.

Distribution services

Distribution management refers to the delivery to the final customer and is characterized by order cycle time, fill rate, response to customer inquiry, number of returns, back orders, etc. [44].

Warehouse/storage services

Warehouse management is extremely important in the case of reverse logistics as many activities (reassembly, repackaging, etc.) are executed at the warehouse [41]. Warehouse management reverse logistics includes inventory management, infrastructure and suprastructure ownership, customer order processing, warehouse efficiency [44], inventory carrying costs [44], inventory turnover [44], cross docking possibility [42] and more.

The size and quality of the infrastructure (warehouses, distribution centres, etc.) and suprastructure (fleet, manipulation technology, etc.) increase delivery performance, which has a positive impact on final customer satisfaction as well as the competitiveness of the buyer looking to outsource [45, 46].

Value-added services and Innovation capability

Value-added services as well as innovation capability play a very important role. They somehow improve the buyer’s services, which is absolutely necessary in this very competitive environment. It should, therefore, be used as selection criteria; however, our literature review revealed that this has not yet been put into practice [26, 28, 41, 42, 4749].

Breadth of services

The range of services provided is currently a highly desirable characteristic, since enterprises look for a single sourcing provider capable of offering the widest range of services (transportation, warehousing, inventory management, freight forwarding, etc.) and not just a single activity [50].

Information technology (IT) capability

IT capability became one of the key selection criteria in the LP selection process. IT ability represents management related to information systems that control and support logistics functions [34, 41, 42] having a significant positive impact on quality, operational performance [45], inventory level, lead time, etc. It also reduces the bullwhip effect and consequently the cost [51, 52].

IT ability can be increased using a variety of sophisticated software such as Enterprise Resource Planning (ERP), EDI networking, simulation software, Transport Management System (TMS), Warehouse Management System (WMS), Customer and Supplier Relationship (CRM, SRM), among others. Hardware such as servers, networking, internet/intranet connections, radio frequency devices, bar code printers and scanners are an essential component of IT capability [50].

4.2.2. Service level

Service level is a key selection criterion as well. Because of its positive impact on reputation and consequently market share, the level of service provided plays a very important role on the strength of the supply chain [26]. It integrates several subcriteria, which are described as follows.

Reliability (accurate time, accurate place, accurate quality and quantity)

Reliability refers to accuracy and providing consistent quality [51, 53]. Delivery reliability, therefore, consists of elements such as accurate time in providing logistics services, accuracy in delivering to the correct location, accurate quality and quantity of shipments delivered, and documents [54]. Time reliability describes the “number of shipments which respect the agreed upon time-frame” [55, 56]. Place reliability is characterized by the number of deliveries that conform to the agreed delivery location. Quality reliability describes the capability of LPs to minimize any damage [25, 57], shipment losses or other problems that affect the customer [46, 55]. The quality of the services also has a positive impact on end-customer satisfaction [45].

A systematic review of the literature showed that in determining the selection criteria, customers often choose specific sub-attributes such as accurate time and not the whole attribute (delivery reliability). It, therefore, makes sense to split delivery reliability criteria into individual subcriteria, namely, accurate time, accurate place, accurate quality and quantity. Subcriteria associated with discrepancies, lost or damaged goods [35] and loss rate [58] belong to the reliability-related criteria.

Flexibility and responsiveness

Flexibility in delivery and performance is characterized by the ability to meet the customer’s changing needs including specific and non-routine business requirements or personalized customer needs [53, 59, 60]. Responsiveness is the ability of a LP to react quickly to unexpected events and emergencies. This may include urgent deliveries because of a sudden rise in product demand [27, 45].

Flexibility-related criteria as well as responsiveness-related criteria encourage and enhance reliability-related criteria [26]. Flexibility-related criteria are, therefore, often treated as reliability-related criteria [25]. We believe that this is not recommended as they have an important impact on the service level. If they are not required, competitiveness may be threatened. It is, therefore, our position that flexibility-related and responsiveness-related criteria should be included in the selection process.

Customer satisfaction

Customer satisfaction is a rarely used criterion [24, 34, 35, 38, 6163]. It covers effective communication, service improvement [25, 34, 47], cost savings, profitability [61], optimization capabilities [47], the ability to meet or exceed promises [64], successful relationships, etc. [34].

Network coverage

Network or geographical coverage, market size and market share belong to the same criteria, which indicate that the countries in which LPs are present with offices, branches, warehouses, etc. [65]. This criterion is important for two reasons; first, buyers of logistics services look for a single logistics provider able to cover as many logistics services in the larger area as possible [53] and second, international LPs can help the buyer expand their business globally [52] and also save money on product distribution and marketing [46]. A global presence-related attribute, used by many authors [47, 49, 66, 67], could therefore be integrated into this attribute as well. Network coverage-related attributes are an indication of the financial stability of the LP [24], level of customer satisfaction, reputation [46] and create better access to the market [60].

Information exchange

Information sharing ability relates to providing high quality information between the buyer and the LP. It indicates the maturity of both parties and consequently ensures healthy business practices [53]. Such information sharing requires a high degree of confidence. Information sharing as well as confidence is the foundation of long-term cooperation and the continuous improvement of logistics services [24, 60].

Risk management

“Risk management is the capability of the provider to address any unforeseen problem. It is needed to ensure the continuity of the services” [24, 46].

Service level-related criteria with all subcriteria (accurate quality, quantity, time and place, as well as flexibility, responsiveness, reliability, order fill rate, inventory turnover, space utilization, document accuracy, system security, confidentiality of sensitive data and the other subcriteria) as well as performance-monitoring capability (identifying key performance indicators – KPI), implementing Six-Sigma, ISO standards, fault diagnosis capability, statistical data reporting and many more [24, 45, 46, 50, 65] have also been integrated in just one criteria, referred to as performance criteria by many authors [24, 66, 68]. Nevertheless, we believe that this concept covers an excessive number of quite different elements that require individual identification. Merging all these elements under the umbrella of a single criterion is simply not appropriate.

Performance management

Performance monitoring capability refers to key performance indicators (KPI) identification as well as quantitative and qualitative performance evaluation. The aim of such a measurement is the evaluation of LP quality and identification of any problems [24, 46].

4.2.3. Price/Costs

The price of the service, sometimes referred to as service costs, is the total costs of logistics outsourcing [24, 46, 51, 60, 69], consisting of transportation costs, warehousing costs, freight forwarding, packaging costs, value-added costs, etc. [52]. Because of its role in ensuring a competitive advantage, price is the most commonly used criterion; however, it is no longer the most important [50].

Cost-related criteria could cover only the basic price or it could be divided into basic and variable prices including discounts. This further division demonstrates the degree of LP flexibility [25] and should therefore be treated as flexibility in billing and payment-related subcriteria. This subcriterion that increases consideration between the LP and the buyer [24, 46, 51, 60] often appears in the literature. Credit terms, error-free bills [37], payment accuracy, payment method and payment speed and open-book accounting [53] are part of this attribute as well [37, 63].

4.2.4. Provider status

LP status, often referred to as strategic evaluation status, is the foundation of many strategic subcriteria, namely, culture comparability, reputation, financial stability, quality of personnel and empathy.

Culture recognition

Compatibility with the user’s culture and traditions refers to the similar values, size, comparable culture, business processes, technology capability, etc. of the LP and the buyer. It is the basis for working together and achieving common goals [24, 45, 51]. Moreover, it is one of the key elements for establishing a long-term and strategic relationship [26, 45, 50, 68].

Financial stability

Financial stability, often indicated as being an essential requirement for logistics providers [45], shows that any given LP has a sound financial position that ensures continuity of services, regular upgrading of infrastructure and suprastructure, the ability to respond to an ever-changing environment [53, 60] and a low risk relationship as well [45].

Financial stability-related criteria could be determined by total annual revenue [52], indicated by balance sheets and income statement, which demonstrate liquidity, operating, profitability and other financial performance indicators [27, 65].

Provider reputation

A provider’s reputation reflects the image of the LP in the market [53] (how good it is at outsourcing, its position in the industry). Jharkharia and Shankar [24] in addition to Çakir [46] consider this attribute to be of vital importance, especially in the initial evaluation of the providers. Qureshi et al. [45] have put forward that reputation encourages the buyer of logistics service to begin a long-term relationship.

Provider experience

A provider’s experience demonstrates that a LP has the knowledge of the specifics and critical issues [53] of some industries or certain products. LPs with this characteristic have a significant advantage over others [24]. This criterion ensures a smooth transition to logistics outsourcing and a lower degree of risk associated with unsuccessful outsourcing.

Staff quality

Quality of personnel or staff quality represents the specialized, extensive knowledge of the workforce [50], which consequently ensures a higher level of service [45] and enhances the responsiveness and flexibility of the supply chain [53]. Quality management promotes a long-term relationship and boosts information sharing and trust [24, 45, 46].


Empathy is the term used to encompass employee courtesy [54], customer respect and care, the goodwill of the LP [56], a readiness to provide dedicated services [53], the ability to understand the buyer’s needs [53], the possibility of establishing a long-term strategic relationship [24, 25, 45, 48, 53, 68] based on trust, risk and benefits sharing [40, 49, 53, 54, 62, 70], conflict resolution [61], controlling the opportunistic behaviour of providers [46], etc.

The willingness to use logistics manpower-related subcriteria [46, 71], which refers to the willingness of the LP to hire those employees who would otherwise remain unemployed, could also be classified in this group [24, 64].

Employee satisfaction

Employee satisfaction level was a rarely used criterion [7274]. It refers to human resource management [75, 76], human relationships and human resource policy [24, 46, 47, 49, 60, 71, 77]. It affects the quality and smooth execution of logistics services.

4.2.5. Environmental sustainability

Environmental sustainability is quite a broad term. It is best divided into a further two subcriteria:

  • so-called green logistics and

  • reverse logistics services (Figure 4)

Green logistics is characterized by the level of pollutant emissions or the pollutants released, energy consumption [58], the use of clean materials and energy as well as environmental expenditures [35, 36, 78].

Pollutant emissions or pollutants released refer to emissions generated from one of the logistics services. Energy consumption refers to all energy used for one part of logistics services. The use of clean materials and energy relates to the use of renewable energy resources (sun, wind, etc.) and fuels (fossil fuels, etc.). Environmental expenditures describe the costs ensued to ensure an eco-efficient logistics processes (costs for fleet optimizations, eco-efficient vehicles, new network design, eco-efficient standard implementation, staff training, etc.) [40, 79].

Figure 4.

Environmental criteria and subcriteria for an LP selection.

Reverse logistics services related to criteria includes some criteria that are general in nature and useful for broader purposes (not only for the purpose of sustainable development) as well as some more specific criteria.

General criteria relate mainly to transportation services (reverse transportation costs [33, 40], redelivery costs, etc. [40]), distribution services (number of returns, number of back orders [44]), warehouse/storage services (infrastructure ownership, inventory carrying costs [44]), reverse logistics costs (inspection, transportation, inventory, and packaging costs) [33, 42, 49, 80, 81].

Specific criteria refer mainly to value-added reverse logistics services and waste disposal capabilities.

Value-added reverse logistics services include the reassembly (deconstruction of the product for reuse or recycling) [33, 41, 42], repackaging (products are repacked before being resold) [41], remanufacturing of a product before being resold, refurbishment or renovation of products (making minor repairs on products) [34, 42, 82] and recycling [34]. Recycling capability criteria refer to recycling costs [33, 40], recycling plant capacity and ownership [33, 36, 39, 83], satisfying the quality specifications for recycling [33], recycling equipment [38], advanced technology level of recycling and disassembling, speed of the recycling centre [40], etc.

Waste disposal activities include the screening, collecting, sorting, packaging and storage of waste [32, 34, 40, 41].

Despite the complexity of the environmental sustainability-related criteria, some authors do not find it reasonable to use subcriteria [28, 35, 49, 79, 84, 85]. We assert that by not using subcriteria, they lose critical data on the competencies of LP and are not aware of what the exact environmental competence of the LP actually is. We, therefore, consider the division into subcategories both reasonable and necessary.

The most commonly used environmental criteria are value-added services, used by 11 authors, followed by environmental expenditures, identified in 9 articles. Return order process capability was detected in six cases, reverse logistics cost in five cases, pollutants released and energy consumption in four cases and clean material and energy was used by just two authors [Figure 5].

The most commonly used criteria of them all (not just environmental criteria) is price, used in 99 cases, followed by information technology application, used in 61 cases (Figure 6). Accurate delivery time was used by 60 authors, accurate quality and quantity by 51 authors, technical and technological ownership by 49 authors, staff quality by 47 authors, flexibility in operations by 46 authors, network coverage by 45 authors and empathy and degree of reputation by 42 authors. All the other criteria were used less frequently, namely experience and culture recognition in 39 cases, reliability in 36 cases, information exchange ability in 35 cases and accurate delivery place in 31 cases. Environmental-related criteria are the least selected criteria, used in less than 12 cases.

Figure 5.

The frequency of the use of individual environmental criteria.

Figure 6.

The frequency of the use of all (not just) environmental criteria.


5. Critical analysis of the environmental criteria in terms of quality and quantity

The buyers of logistics services mainly focus their attention on traditional logistical services (the physical flows of products from the manufacturer to the consumer) [86], when defining criteria for selecting an LP. Environmental logistics services are only very rarely applied. Regarding the literature review, environmental criteria could be divided into two main groups. The first is green logistics activities-related criteria and the second is reverse logistics activities-related criteria. The most frequently mentioned green logistics activities were pollutants released, energy consumption, clean materials and energy use and environmental expenditures. Value-added reverse logistics services and reverse logistics costs are the most frequently used reverse logistics criteria. Value-added reverse logistics services are further divided into the following activities: reassembly, remanufacturing of product before being resold, refurbishment or renovation of products, recycling capability, recycling equipment, advanced technology level of recycling and disassembling, speed of recycling centre, etc. Waste disposal activities included screening, collecting, sorting, packaging, storages of wastes.

To make a critical analysis, in recent years there were few published articles on the topic of the framework of reverse logistics services [8789]. Environmental logistics activities are, according the published articles, very different in content and aim. Some are focused on resource reduction, some on maximizing reuse, some on recycling and others on disposal activities [90]. It would, therefore, be best to classify reverse logistics services into three main groups, namely reverse logistics processes, waste management and green logistics activities.

Reverse logistics processes refers to reverse physical flows, directed from the final consumer to the manufacturer. They are further divided into four main groups, namely (1) collection; (2) integrated function, which includes inspection, selection and sorting process; (3) direct recovery and (4) redistribution [87]. Collection is the process of bringing products to the place of recovery where they are first inspected. They are then selected and finally sorted as to the type of recovery. The types of recovery are reuse, resale, redistribution and reprocessing, further divided into repair (process used for products), refurbishing (process used for modules), remanufacturing (process used for components), recycling (process used for material), etc. [87, 88].

Green logistics refers to providing environmentally friendly methods of traditional or forward logistics services, directed from the manufacturer to the final consumer [87]. It covers packaging reduction, recycling, remanufacturing, reusable packaging, air and noise emission reduction, environmental impact of mode selection [91], measuring the environmental impact and energy reduction [92].

There are differing opinions on the need to distinguish between green and reverse logistics. Some authors believe that reverse logistics should be included in green logistics [92]; others claim that they only share some common activities and are otherwise very different in nature [91]. We believe they are different enough to be treated as independent selection criteria.

Waste management refers to the management of waste generated during production, distribution, usage of final products, etc. It includes prevention, collecting, recycling or recovery, any treatment of waste, the optimum final disposal of wastes and improved monitoring [93].

There were no significant differences between the results of a systematic review on the selection criteria and the brief review of the literature on the framework of green and reverse logistics. There were two detected differences that should be considered. First, the classification of waste disposal activities as separate criteria, not integrated within value-added reverse logistics services, and second, adding measuring the environmental impact as subcriteria of green-related criteria. Moreover, further distribution of value-added reverse logistics activities-related criteria into subcriteria is also recommended. All three suggestions will most certainly improve the quality of selection criteria. Nevertheless, additional evaluation and examination of the validity of selection criteria on the part of experts from the logistics industry who have knowledge and expertise is also required.


6. Conclusion

This review has taken an extensive look at criteria and subcriteria for LP selection, focusing in particular on environmental criteria. Ninety-five studies on LP selection have been systematically analysed to identify the most frequently used general and environmental criteria, which were critically analysed further in terms of quality and quantity.

The systematic literature review revealed how rare publications on this topic actually are as only 11 papers were found. The situation was quite similar when the use of environmental selection criteria were analysed. The frequency of the use of environmental selection criteria was found to be far behind that of cost-related criteria, which held first place and followed by service level and LP status.

Criteria that focused on environmental issues were found to be surprisingly very well defined, covering the whole range of environmental logistics activities, namely green logistics activities, reverse logistics activities and waste disposal activities. All three groups were further divided into subcriteria. The additional evaluation of the defined selection criteria by the experts from different fields will significantly improve the quality and quantity of criteria. We find this survey very helpful for decision makers, logistics managers as well as researchers. It is our contention that decision makers and logistics managers could apply the results of this paper in practice, while researchers may find the results useful for further studies. We believe that a step forward towards green logistics management has been made.


  1. 1. Srivastava SK. Green supply‐chain management: A state‐of‐the‐art literature review. International Journal of Management Reviews 2007;9[1]:53-80.
  2. 2. Colicchia C, Strozzi F. Supply chain risk management: A new methodology for a systematic literature review. Supply Chain Management: An International Journal. 2012;17[4]:403-418.
  3. 3. Bacallan JJ. Greening the supply chain. Business and Environment. 2000;6[5]:11-12.
  4. 4. Rao P, Holt D. Do green supply chains lead to competitiveness and economic performance? International Journal of Operations & Production Management. 2005;25[9]:898-916.
  5. 5. Evangelistia P, Huge-Brodin M, Isaksson K, Sweeney E. The Impact of 3PL’s Green Initiatives on the Purchasing of Transport and Logistics Services: an Exploratory Study. Vision 20/20 - Preparing today for tomorrow’s challenges. In: Proceedings of the 20th International Purchasing and Supply Education and Research Association (IPSERA), Conference, Maastricht University; April 2011; Maastricht
  6. 6. Zhu Q, Geng Y. Integrating environmental issues into supplier selection and management. Greener Management International. 2001;2001[35]:26-40.
  7. 7. Markley MJ, Davis L. Exploring future competitive advantage through sustainable supply chains. International Journal of Physical Distribution & Logistics Management. 2007;37[9]:763-774.
  8. 8. Blome C, Hollos D, Paulraj A. Green procurement and green supplier development: Antecedents and effects on supplier performance. International Journal of Production Research 2014;52[1]:32-49.
  9. 9. Klassen RD, Vachon S. Collaboration and evaluation in the supply chain: The impact on plant‐level environmental investment. Production and Operations Management 2003;12[3]:336-352.
  10. 10. Lindgreen A, Swaen V, Maon F, Walker H, Brammer S. Sustainable procurement in the United Kingdom public sector. Supply Chain Management: An International Journal. 2009;14[2]:128-137.
  11. 11. Walker H, Di Sisto L, McBain D. Drivers and barriers to environmental supply chain management practices: Lessons from the public and private sectors. Journal of Purchasing and Supply Management 2008;14[1]:69-85.
  12. 12. McKinnon A, Cullinane S, Browne M, Whiteing A. Environmental sustainability. Green logistics: improving the environmental sustainability of logistics. 1st ed. London: Kogan Page Limited; 2010.
  13. 13. Eglese R, Black D. Optimizing the routing of vehicles. In: Green Logistics: Improving the Environmental Sustainability of Logistics. KoganPage; 2010:215-228.
  14. 14. Lieb KJ, Lieb RC. Environmental sustainability in the third-party logistics [3PL] industry. International Journal of Physical Distribution & Logistics Management. 2010;40[7]:524-533.
  15. 15. Fadavi A, Khanghah AS, Asli MN. A hybrid model for supplier selection in outsourcing: Evidence from Shima Film Company in Iran. Research Journal of Applied Sciences, Engineering and Technology 2013;5[12]:3298-3305.
  16. 16. Cooper J, Black I, Peters M. Creating the sustainable supply chain: Modelling the key relationships. In: Banister D, editors. Transport Policy and the Environment. 1st ed. London: E & FN Spon; 1998. p.176-203.
  17. 17. Björklund M. Influence from the business environment on environmental purchasing—Drivers and hinders of purchasing green transportation services. Journal of Purchasing and Supply Management 2011;17[1]:11-22.
  18. 18. Halldórsson Á, Kovács G, Wolf C, Seuring S. Environmental impacts as buying criteria for third party logistical services. International Journal of Physical Distribution & Logistics Management. 2010;40[1/2]:84-102.
  19. 19. Selviaridis K, Spring M. Third party logistics: a literature review and research agenda. The International Journal of Logistics Management. 2007;18[1]:125-150.
  20. 20. Zhu Z, Hsu K, Lillie J. Outsourcing – a strategic move: The process and the ingredients for success. Management Decision 2001;39[5]:373-378.
  21. 21. Keramydas C, Tsolakis N, Xanthopoulos A, Aidonis D. Selection and evaluation of 3PL providers: A conceptual decision-making framework. In: Folinas D, editors. Outsourcing Management for Supply Chain Operations and Logistics Service.1st ed. United States of America: IGI Global; 2012. p. 280.
  22. 22. Aghazadeh S-M. How to choose an effective third party logistics provider. Management Research News. 2003;26[7]:50-58.
  23. 23. Soh S. A decision model for evaluating third-party logistics providers using fuzzy analytic hierarchy process. African Journal of Business Management. 2010;4[3]:339-349.
  24. 24. Jharkharia S, Shankar R. Selection of logistics service provider: An analytic network process [ANP] approach. Omega 2007;35[3]:274-289.
  25. 25. Lehmusvaara A, Tuominen M, Korpela J. An integrated approach for truck carrier selection. International Journal of Logistics: Research and Applications. 1999;2[1]:5-20.
  26. 26. Bansal A, Kumar P. 3PL selection using hybrid model of AHP-PROMETHEE. International Journal of Services and Operations Management. 2013;14[3]:373-397.
  27. 27. Yayla AY, Oztekin A, Gumus AT, Gunasekaran A. A hybrid data analytic methodology for 3PL transportation provider evaluation using fuzzy multi-criteria decision making. International Journal of Production Research.2015;53[20]:6097-6113.
  28. 28. Bansal A, Kumar P, Issar S, editors. 3PL selection: A multi-criteria decision making approach. Industrial Engineering and Engineering Management [IEEM], 2013 IEEE International Conference on; 2013 10-13 December 2013.
  29. 29. Aguezzoul A. The third party logistics selection: A review of literature. In: Proceedings of the International Logistics and Supply Chain Congress; November 2007; Istanbul. Turkey: 2007. p. 7 <hal-00366527>
  30. 30. Tranfield DR, Denyer D, Smart P. Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management 2003;14:207-222.
  31. 31. Briner RB, Denyer D. Systematic review and evidence synthesis as a practice and scholarship tool. Handbook of Evidence-Based Management: Companies, Classrooms and Research. 2012:112-129.
  32. 32. Choudhary D, Madaan J, Narain R. An integrated decision model for selection of third party recovery facilitator [3PRF] for product recovery operations. Journal of Operations and Supply Chain Management. 2014;7[2]:37-58.
  33. 33. Sasikumar P, Haq AN. Integration of closed loop distribution supply chain network and 3PRLP selection for the case of battery recycling. International Journal of Production Research 2010;49[11]:3363-3385.
  34. 34. Zareinejad M, Javanmard H. Evaluation and selection of a third-party reverse logistics provider using ANP and IFG-MCDM methodology. Life Science Journal. 2013;10[6s]:350-355.
  35. 35. Efendigil T, Önüt S, Kongar E. A holistic approach for selecting a third-party reverse logistics provider in the presence of vagueness. Computers & Industrial Engineering 2008;54[2]:269-287.
  36. 36. Saen RF. A new look at selecting third-party reverse logistics providers. Information Technologies, Methods, and Techniques of Supply Chain Management. 2012:246.
  37. 37. Farzipoor Saen R. A mathematical model for selecting third-party reverse logistics providers. International Journal of Procurement Management. 2009;2[2]:180-190.
  38. 38. Momeni E, Azadi M, Saen RF. Measuring the efficiency of third party reverse logistics provider in supply chain by multi objective additive network DEA model. International Journal of Shipping and Transport Logistics. 2015;7[1]:21-41.
  39. 39. Saen R. A new model for selecting third-party reverse logistics providers in the presence of multiple dual-role factors. International Journal of Advanced Manufacturing Technology 2010;46[1-4]:405-410.
  40. 40. Zhang R, Zhang H, Liu B. Selection of reverse-logistics servicer for electronic products with fuzzy comprehensive evaluation method. Grey Systems: Theory and Application. 2012;2[2]:207-216.
  41. 41. Cheng Y-H, Lee F. Outsourcing reverse logistics of high-tech manufacturing firms by using a systematic decision-making approach: TFT-LCD sector in Taiwan. Industrial Marketing Management 2010;39[7]:1111-1119.
  42. 42. Khodaverdi R, Hashemi SH. A grey-based decision-making approach for selecting a reverse logistics provider in a closed loop supply chain. International Journal of Management and Decision Making 2015;14[1]:32-43.
  43. 43. Aguezzoul A. Third-party logistics selection problem: A literature review on criteria and methods. Omega 2014;49[0]:69-78.
  44. 44. Cooper O, Tadikamalla P, Shang J. Selection of a third‐party logistics provider: Capturing the interaction and influence of performance metrics with the analytical network process. Journal of Multi-Criteria Decision Analysis 2012;19[3-4]:115-128.
  45. 45. Qureshi M, Kumar D, Kumar P. An integrated model to identify and classify the key criteria and their role in the assessment of 3PL services providers. Asia Pacific Journal of Marketing and Logistics. 2008;20[2]:227-249.
  46. 46. Çakir E. Logistics outsourcing and selection of third party logistics service provider [3PL] via fuzzy AHP. Master Thesis from Institute of Science, Industrial Engineering, TC Bahceshir University, Istanbul; 2009.
  47. 47. Liu H-T, Wang W-K. An integrated fuzzy approach for provider evaluation and selection in third-party logistics. Expert Systems with Applications. 2009;36[3, Part 1]:4387-4398.
  48. 48. Yan J, Chaudhry PE, Chaudhry SS. A model of a decision support system based on case‐based reasoning for third‐party logistics evaluation. Expert Systems 2003;20[4]:196-207.
  49. 49. Sahu NK, Datta S, Mahapatra SS. Decision making for selecting 3PL service provider using three parameter interval grey numbers. International Journal of Logistics Systems and Management. 2013;14[3]:261-297.
  50. 50. Bottani E, Rizzi A. A fuzzy TOPSIS methodology to support outsourcing of logistics services. Supply Chain Management: An International Journal. 2006;11[4]:294-308.
  51. 51. Kumar P. An integrated model of AHP and TOPSIS for 3PL evaluation. Asia Pacific Business Review. 2008;4[3]:14-21.
  52. 52. Singh R, Shankar R, Kumar P, Singh RK. A fuzzy AHP and TOPSIS methodology to evaluate 3PL in a supply chain. Journal of Modelling in Management. 2012;7[3]:287-303.
  53. 53. Thakkar J, Deshmukh S, Gupta A, Shankar R: Selection of third-party logistics [3PL]: A hybrid approach using interpretive structural modeling [ISM] and analytic network process [ANP]. Supply Chain Forum: An International Journal. 2005; 6[01]:38-46.
  54. 54. Bourlakis M, Melewar T, Banomyong R, Supatn N. Selecting logistics providers in Thailand: A shippers’ perspective. European Journal of Marketing 2011;45[3]:419-437.
  55. 55. Aloini D, Dulmin R, Mininno V. A hybrid fuzzy-PROMETHEE method for logistics service selection: Design of a decision support tool. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems. 2010;18[04]:345-369.
  56. 56. Niestrój K. The conformity assessment of lead logistics provider and third party logistics preferences using analytic hierarchy process method [AHP]. Journal of Economics & Management/University of Economics in Katowice. 2013;[12]:61-74.
  57. 57. Choy KL, Chow HK, Tan K, Chan C-K, Mok EC, Wang Q. Leveraging the supply chain flexibility of third party logistics–Hybrid knowledge-based system approach. Expert Systems with Applications. 2008;35[4]:1998-2016.
  58. 58. Zhang H, Zhang G, Zhou B. Research on selection of the third-party logistics service providers. Integration and Innovation Orient to E-Society Volume 1. Springer; 2007. 211-221.
  59. 59. Jianguo X, Zhong Q. The research on service quality influencing factors of the logistics enterprises based on gray relative analysis. Journal of Advanced Management Science.2013;1[2]:236-240.
  60. 60. Rajesh R, Pugazhendhi S, Muralidharan C, Ganesh K. Development of a composite model for selection of third party logistics service provider. International Journal of Electronic Customer Relationship Management. 2009;3[4]:375-401.
  61. 61. Pang B, Bai S. An integrated fuzzy synthetic evaluation approach for supplier selection based on analytic network process. Journal of Intelligent Manufacturing 2013;24[1]:163-174.
  62. 62. Li D-F, Wan S-P. Fuzzy heterogeneous multiattribute decision making method for outsourcing provider selection. Expert Systems with Applications. 2014;41[6]:3047-3059.
  63. 63. Wang Y, Gao X, Yang L. Logistics outsourcing and selecting of logistics service provider of the e-commerce companies: A fuzzy TOPSIS approach. 2014.
  64. 64. Kannan G, Pokharel S, Kumar PS. A hybrid approach using ISM and fuzzy TOPSIS for the selection of reverse logistics provider. Resources, Conservation and Recycling 2009;54[1]:28-36.
  65. 65. Çatay B, Göl H. Logistics outsourcing and 3PL selection: A case study in an automotive supply chain. Supply Chain Management: An International Journal. 2007.
  66. 66. Zhang H, Li X, Liu W. An AHP/DEA methodology for 3PL vendor selection in 4PL. Computer Supported Cooperative Work in Design II. Springer; 2006. 646-655.
  67. 67. Singh Bhatti R, Kumar P, Kumar D. Analytical modeling of third party service provider selection in lead logistics provider environments. Journal of Modelling in Management. 2010;5[3]:275-286.
  68. 68. Işıklar G, Alptekin E, Büyüközkan G. Application of a hybrid intelligent decision support model in logistics outsourcing. Computers & Operations Research 2007;34[12]:3701-3714.
  69. 69. Akman G, Baynal K. Logistics service provider Selection through an Integrated Fuzzy Multicriteria Decision Making Approach. Journal of Industrial Engineering.2014; 2014[article ID 794918]: 1-16. DOI:10.1155/2014/794918
  70. 70. Zhi-hong Y, Qiang L, editors. The grey comprehensive evaluation model of a third-party reverse ligistics provider selection. Artificial Intelligence, 2009 JCAI’09 International Joint Conference on; 2009: IEEE.
  71. 71. Li C, Sun Y, Du Y, editors. Selection of 3PL service suppliers using a fuzzy analytic network process. Control and Decision Conference, 2008 CCDC 2008 Chinese; 2008: IEEE.
  72. 72. Leina Z, Tiejun P, Guoqing Y, editors. The process integration evaluation method of the fourth party logistics using fuzzy theory. Management of e-Commerce and e-Government [ICMeCG], 2010 Fourth International Conference on; 2010: IEEE.
  73. 73. Hsu C-C, Liou JJH, Chuang Y-C. Integrating DANP and modified grey relation theory for the selection of an outsourcing provider. Expert Systems with Applications. 2013;40[6]:2297-2304.
  74. 74. Xianlong G, Yujie G. Research on functional logistics provider selection based on QFD modeling. Journal of Applied Sciences 2013;13[17]:3563-3568.
  75. 75. Huang J-D, Hu M, Wee H-M. Evaluation of lead logistics provider using the SMART Process: A case study in a Tiwan automotive industry. Operations and Supply Chain Management. 2013;6[1]:9.
  76. 76. Göl H, Çatay B. Third‐party logistics provider selection: Insights from a Turkish automotive company. Supply Chain Management: An International Journal. 2007;12[6]:379-384.
  77. 77. S-h So, Kim J, Cheong K, Cho G. Evaluating the service quality of third-party logistics service providers using the analytic hierarchy process. JISTEM-Journal of Information Systems and Technology Management 2006;3[3]:261-270.
  78. 78. Gupta R, Sachdeva A, Bhardwaj A. Selection of logistic service provider using fuzzy PROMETHEE for a cement industry. Journal of Manufacturing Technology Management 2012;23[7]:899-921.
  79. 79. Yang I-z. A 3PL provider selection model for companies adopting a pro-active environmental strategy in the context of green supply chain management [Master]. National Central University; 2010.
  80. 80. Govindan K, Grigore MC, Kannan D, editors. Ranking of third party logistics provider using fuzzy Electre II. Computers and Industrial Engineering [CIE], 40th International Conference on; 2010; Japan; 2010: IEEE.
  81. 81. Kannan G, Murugesan P, Senthil P, Noorul Haq A. Multicriteria group decision making for the third party reverse logistics service provider in the supply chain model using fuzzy TOPSIS for transportation services. International Journal of Services, Technology and Management 2009;11[2]:162-181.
  82. 82. Chiang Ziping, Tzeng G-H. A third party logistics provider for the best selection in fuzzy dynamic decision environments. International Journal of Fuzzy Systems. 2009;11[1]:9.
  83. 83. Saen RF. A new model for ranking 3PL providers. Australian Journal of Basic and Applied Sciences 2010;4[8]:3762-3769.
  84. 84. Gupta R, Sachdeva A, Bhardwaj A. A framework for the selection of logistic service provider using duzzy delphi and fuzzy topcis. In: Ao S-I, Amouzegar M, Rieger BB, editors. Intelligent Automation and Systems Engineering. Lecture Notes in Electrical Engineering. 103. New York: Springer; 2011. 189-202.
  85. 85. Mehri S, Roghanian E, Khodadadzadeh T. A methodology for outsourcing resources in reverse logistics using fuzzy TOPSIS and fuzzy linear programming. Uncertain Supply Chain Management. 2013;1[2]:107-114.
  86. 86. González-Benito J, González-Benito Ó. The role of stakeholder pressure and managerial values in the implementation of environmental logistics practices. International Journal of Production Research 2006;44[7]:1353-1373.
  87. 87. De Brito MP, Dekker R. A framework for reverse logistics: Reverse logistics. Springer; 2004. 3-27.
  88. 88. Yusuf I: Reverse logistics: an empirical study for operational framework. Proceedings of the Pakistan Academy of Sciences. 2013;50[3]:201-210.
  89. 89. Pokharel S, Mutha A. Perspectives in reverse logistics: A review. Resources, Conservation and Recycling 2009;53[4]:175-182.
  90. 90. Carter CR, Ellram LM. Reverse logistics: A review of the literature and framework for future investigation. Journal of Business Logistics. 1998;19[1]:85.
  91. 91. Rogers DS, Tibben‐Lembke R. An examination of reverse logistics practices. Journal of Business Logistics 2001;22[2]:129-148.
  92. 92. Sbihi A, Eglese R. Combinatorial optimization and green logistics. 4OR. 2007;5[2]:99-116.
  93. 93. Hester R E, Harrison R M, editors. Environmental and health impact of solid waste management activities. 1st ed. Cambridge: The Royal Society of Chemistry; 2002. p. 171-182. DOI:10.1039/9781847550767

Written By

Patricija Bajec and Danijela Tuljak-Suban

Submitted: October 19th, 2015 Reviewed: February 16th, 2016 Published: June 30th, 2016