Application of Data Envelopment Analysis (DEA) in the Selection of Sustainable Suppliers: A Review and Bibliometric Analysis

Katerina Fotova Čiković

Ivana Martinčević

2 and

Joško Lozić

Department of Logistics and Sustainable Mobility, University North, Trg dr. Žarka Dolinara 1, 48000 Koprivnica, Croatia

Submission received: 14 April 2022 / Revised: 26 May 2022 / Accepted: 26 May 2022 / Published: 30 May 2022

Abstract

The supplier selection process is a strategic decision-making process that influences the company’s sustainability. Lately, the sustainability concept has been highlighted as an organization’s source of success and profitability. Therefore, the selection of a sustainable supplier has become an imperative for organizations and is the focus of this manuscript. Suppliers are key stakeholders in the supply chain, and their proper selection is a key factor in a successful and sustainable supply chain. For this reason, it is crucial to determine how and which methods are mostly used by companies when choosing sustainable suppliers with the aim of examining whether the Data Envelopment Analysis (DEA) contributes to the same. This article is the first to present a comprehensive bibliometric analysis of 87 articles dealing with the application of DEA in the sustainable supplier selection in the period 2010–2022, with the application of the keywords “Data Envelopment Analysis”, “Supplier”, and “Sustainable” in Scopus and Web of Science databases. The main goal of this manuscript is to explore the applications of DEA in a sustainable supplier selection and to provide an analysis and visualization of bibliometric data to reveal the annual trends of published articles in this area, the top contributing journals, the most cited papers, the most contributing authors, citations, affiliations, and countries’ analysis, and an in-depth keyword visualization analysis. The findings of this study provide valuable insights and emphasize the ever-growing trend toward the selection of sustainable partners and suppliers in business using DEA methodology. Notably, this work shows the applicability and efficacy of DEA in specialized areas of supply chain management and should contribute to the construction of an overview of the existing literature on DEA studies regarding the process of selection of sustainable suppliers in supply chain management as well as stimulate the interest in the topic. This article gives an overview of a research field that is actually insufficiently explored through the scientific literature and presents a wide area and guidelines for future work.

1. Introduction

Purchasing and supply chain management have a key role in creating a company’s competitive edge. Moreover, purchasing represents “a strategic role in supply chain management for a firm and is the driver of competitive advantage” [1]. Therefore, the selection of suppliers in any business is one of the vital decision-making processes that have a major impact on the business’ viability and sustainability. “Supplier selection is an important decision-making problem which involves many quantitative and qualitative factors incorporating vagueness and imprecision” [2]. The importance of a supplier selection whose actions affect the performance and productivity of the supply chain is a process that requires quality assessment, evaluation, and, ultimately, the selection of suppliers that will contribute to an increase in the overall efficiency of the supply chain, reduced procurement costs, as well as an increase in market competitiveness [3].

However, the changes in the societies, as well as the increasing importance of corporative social responsibility, have imposed even greater importance on the selection and cooperation with sustainable, eco, and green suppliers.

The concept of sustainable development, which is connected to the selection of sustainable suppliers, emphasizes the need and necessity of the company to change the current business, values, rules, attitudes at all levels, activities, and business processes. Economic development and growth and the process of globalization and internationalization of society and markets cannot be stopped, but it is necessary to think in the direction of ensuring a healthy, sustainable, and green society, and a quality economy that will have less harmful effects on the environment. Therefore, it is no wonder that the selection of sustainable suppliers that ensures sustainable development is one of the key elements needed to formulate and implement development policies in the world. The key factor for a successful supply chain is the selection of sustainable suppliers but also the selection of the best sustainable strategy in the supply chain. Application of the DEA model in the process of selecting sustainable suppliers and the best sustainable strategy in the supply chain contributes to increasing the effectiveness and efficiency of the entire supply chain. It also contributes to the sustainability concept in the supply chain design [4].

This encapsulates the reason why DEA methodology has been of interest for researchers in studies regarding the process of selection of sustainable suppliers. DEA is the leading non-parametric approach for measuring the relative efficiency and benchmarking of peer decision-making units (DMUs). Ever since its introduction in 1978 by the seminal paper of Charnes, Cooper, and Rhodes [5,6], it has been widely recognized as one of the most effective methodologies for measuring efficiency and performance. Moreover, there are some specific areas in which DEA has been most applied [7]. More on the DEA methodology is laid out in Section 2.

To research and review novelties and benefits in the field of DEA and its application in the selection of sustainable suppliers, numerous pieces of world literature were researched. In this manuscript, an extensive bibliometric-based survey was conducted to summarize past findings in a research field—in this case, findings on DEA and its application in the selection of sustainable suppliers. Papers within the Scopus and Web of Science (WoS) databases were used to gather information on the research topic, which shows that the field of DEA has been researched by a large number of authors but is still insufficient in the field of its application in the selection of sustainable suppliers. This research and literature gap were the main motivation behind this research. The results of searching Scopus and Web of Science databases are presented in order to demonstrate the scarcity of this research topic, which resulted in 87 scientific papers dealing with DEA and its application in the selection of sustainable suppliers. The number of papers that investigate the issue of selection of sustainable suppliers with DEA methodology increased substantially from 2017 to 2022. Most of the research papers were from Iran, China, and the USA. In the research approach, the focus is only on papers in the field of DEA that are linked to the selection of sustainable suppliers.

Therefore, the main goal of the present work is to perform an extensive literature review with bibliometric analysis of studies integrating DEA in a sustainable supplier selection in an attempt to answer the main research question: what are the current avenues of research for such studies? The results obtained are further complemented with a thematic analysis of real-world applications. There are not many studies that explore the application of DEA in the selection of sustainable suppliers and its importance and contribution to strengthening sustainable supply chain management, so this research contributes to strengthening the awareness of organizations about its application. Application of DEA is relatively new in a process of selection of sustainable suppliers so it needs to be strongly and systematically developed further especially with its impact on development of sustainable supply chains. By analyzing the relevant scientific sources, it can be concluded that a strategic approach to the application and use of DEA in the selection of sustainable suppliers can provide numerous advantages and benefits. Based on the review and analysis of previous world research and based on the analysis of the application of the DEA methodology, several additional research questions are asked: (1) How does the use of the DEA methodology by companies affect the supply chain, and is this presented through researched literature? (2) Does the application of the DEA methodology by companies affect the quality and better selection of suppliers, i.e., sustainable suppliers, and is this presented through researched literature? (3) Is there a connection between the application of the DEA methodology when companies choose suppliers (they do not care about sustainability) and when they choose sustainable suppliers, and is this presented through researched literature? These questions are posed as additional research questions to the main research question to find out whether the DEA methodology affects and is a useful tool to select sustainable suppliers recognized by companies that can ultimately affect the sustainable supply chain presented through bibliographic review of the literature. Moreover, this study reveals the annual trends of published articles in this area, the top contributing journals, the most cited papers, the most contributing authors, citations, affiliations and countries’ analysis, and an in-depth keyword visualization analysis.

The main contributions to this bibliometric review can be summarized as follows: (a) A bibliometric-based survey on the DEA applications for selection of sustainable suppliers is conducted; (b) Clarivate WoS and Scopus databases are explored for collecting the relevant studies; (c) Interesting statistical information from the DEA literature regarding DEA applications in the process of sustainable suppliers’ selection is extracted; (d) The main real-world applications and case studies of DEA are categorized; (e) The main research directions and gaps are suggested for future studies that employ DEA in the sustainable supplier selection process.

The remainder of this paper is structured as follows: In Section 2, a theoretical framework regarding the sustainable suppliers and the DEA methodology is given. Section 3 presents the results from DEA articles in the selection of sustainable suppliers, including the research approach, publication years, document types, keywords analysis, the authors and journals analysis, affiliations analysis, citations analysis, and the co-authorship analysis based on countries. Section 4 opens up a discussion and presents the implications of this research and introduces the future trends for DEA in the selection of sustainable suppliers. The last, fifth section concludes the findings from the bibliometric literature review.

2. Theoretical Framework

2.1. Sustainable/Green Suppliers

Supply chains enable and represent the flow of goods, services, and information without which it is impossible to imagine the normal functioning of the market. They include and are part of the primary, secondary, and tertiary sectors and cover the entire process and flow of production [8]. Supply chains create added value in the business process, which is why it is necessary to manage the supply chain in order to create added value constantly. The supply chain management process includes an “integrated process that includes planning and managing all stakeholder selection activities, procurement of materials, transformation of materials into the final product, as well as related logistics activities within the entire chain” [9] (p. 187). In order to examine the efficiency of the supply chain, the “Game-theoretical Design Technique” approach was tested. The design approach allows easier access to business decisions within the supply chain when the supply chain is exposed to some uncertain parameters and ensures agile, cooperative, and resource-efficient design of multi-stage supply chains [10].

The supply chain is evolving rapidly, and while this development follows the technology development, it also intends to ensure sustainable development. Many authors equate sustainable development with economic and social growth. Sustainable development is the aspiration of society to achieve sustainable economic growth to the extent that it will meet the needs of present and future generations. It is necessary to preserve production capacity for the long term while achieving social goals such as increasing real income per capita, improving hygiene and nutrition, educational achievement, access to resources, equitable distribution of wealth, and increasing freedom [11,12,13,14]. The key goal of any organization is to ensure sustainability and create sustainable business models. Such an approach and business model rely primarily on the selection of sustainable and green suppliers that will ensure the path to sustainable development. Supply chains and their management and governance are a complex system that strives to minimize costs and maximize the level of service, and its focus today should primarily be on the selection of sustainable (green) suppliers.

Sustainable (green) suppliers improve and assist sustainable business models in the field of supply chain management. With an emphasis on sustainability and environmental care, the selection of green suppliers should be a central component and goal in supply chain management [15]. Suppliers and supplier relationship management create several benefits for the company—from creating long-term and loyal partners to creating greater visibility through better communication with them. Choosing a reliable business partner is the task of the management of companies that want to achieve long-term competitiveness and sustainability in the market. Sustainable (green) supplier selection and assessment are the most significant and complex challenges for supply chain managers [16,17]. Supplier selection is “intrinsically related to the Multi-Criteria Decision Making (MCDM) problem” [18]. Moreover, the process of selecting a supplier represents a “key competence in the sourcing function” [19]. Today, everything is intensified through environmental (green) factors that have, i.e., should have an impact on the selection of reliable and green suppliers. The question of how to identify and select sustainable suppliers can be answered through a system of weights that determine environmental factors as important decision factors when selecting suppliers by changing the DEA methodology [20]. One of the problems that arises in the business environment today is the selection of suppliers, primarily the selection of sustainable suppliers. Quality and proper selection of suppliers enable and facilitate the cross-efficiency fuzzy DEA technique. This method evaluates the efficiency of the supply chain and all its members (primarily in this case—suppliers) with the correct ranking which then allows the correct selection of suppliers and later reflects on the overall efficiency, costs, and delivery time within the supply chain [21].

Choosing sustainable suppliers means choosing business partners with the most beneficial monetary value on the one hand and the least harmful impact on society and the environment on the other [22]. Assessing the sustainability and performance of the supplier and the supply chain itself requires a selection of optimal tools and methods for the selection process, where DEA is an important tool for measuring the performance of sustainable suppliers and sustainable supply chains [23]. That the DEA methodology and the evaluation index method are an objective and quantitative tool for evaluating and selecting sustainable (green) suppliers is a fact confirmed by numerous researchers through their empirical research [23,24,25,26,27,28,29,30].

However, considering the importance of suppliers in the strategy framework of supply chains, it is rather surprising that “the sourcing function has not been subject to more focused research on the development of adequate decision support tools” [19].

2.2. Data Envelopment Analysis (DEA)

The DEA is a linear mathematical programming technique that is used for the evaluation of the performance (i.e., the relative efficiencies) of a group of complex entities referred to as Decision-Making Units (DMUs) [31]. DEA is one of the most widely applied non-parametric methodologies ever since its introduction in 1978 by Charnes, Cooper, and Rhodes (1978) [5,6] that has grown into a powerful mathematical and linear programming technique. Namely, according to [7], there are five areas in which DEA has been most applied, and these are Agriculture, Banking, Supply Chain, Transportation, and Public Policy. Research conducted in the field of supply chain related to planning and resource management in intermodal terminals proves that the application of the DEA model allows planning decisions under conflicting requirements (the DEA method provides data on the number, capacity, and allocation of resources to address increasing flows) [32]. Moreover, the positive implication and real case study regarding the usage of the DEA method in the supply chain is used “to determine the efficiency of the rescheduled timetable (in terms of reduction of delays and maximization of robustness) and to rank alternatives according to their efficiency values [33] (p. 256).

DEA is a non-statistical and non-parametric approach that “makes no assumptions regarding the distribution of inefficiencies or the functional form of the production function (although it does impose some technical restrictions such as monotonicity and convexity)” [34], which represents one of its main advantages over parametric methodologies. It is a “data-oriented“ method that converts multiple inputs to multiple outputs when evaluating peer units—DMUs [35]. DEA focuses on the extreme observations, which is its main distinction from the parametric methodologies that “focus on average tendencies and deviations from it”. Moreover, DEA can employ multiple inputs and outputs, while parametric methodologies can only employ one output, which represents one of their biggest limitations [36]. This makes DEA “an excellent data-oriented efficiency analysis method” when using multiple inputs and outputs and “a useful performance evaluation and decision-making tool” [37].

DEA identifies the relative efficient DMUs in the observed sample that shape the efficiency frontier that measures the inefficiency of inputs or outputs of the other DMUs in the sample by comparing them and benchmarking with the relative efficient DMUs. Thus, DEA is also an econometric frontier method. The DMUs that lie on the frontier have the best relative efficiency, whereas the ones that are inefficient lie below the efficiency frontier [38]. Moreover, the results from the DEA vary from 0 to 1 (or 0 to 100%), with 1 being relatively efficient and a result below 1 being relatively inefficient. Thus, this approach enables a simple comparison of the DMUs in the sample [39,40].

Two basic DEA models are named in honor of their founders (CCR—Charnes, Cooper, and Rhodes and BCC—Banker, Charnes, and Cooper [5,6]). The CCR model employs a constant return to scale (CRS) assumption, i.e., “the output variables increase proportionally with input variables” [41], whereas the BCC DEA model employs a variable returns-to-scale (VRS) assumption, assuming that the proportional change in inputs does not necessarily lead to a proportional change in the outputs. The CCR model is graphically represented as a straight line, whereas the BCC model is represented by a convex hull [42].

However, it should be kept in mind that DEA is a methodology that explores the relative (and not absolute) efficiency within the analyzed sample of peer decision-making units [43]. The limitations and downsides of DEA are often scholarly researched, but most of the authors agree that “its advantages outweigh its limitations” and DEA should be considered “a significant diagnostic tool” [31].

In this bibliographic review, the applications of DEA in the process of supplier selection were taken into consideration. Moreover, this paper could represent an incentive to other scholars and researchers to employ the DEA methodology in the decision-making processes in any industry due to its advantages, easy implementation, and insights it provides.

3. Results on DEA Articles in the Selection of Sustainable Suppliers

3.1. Research Approach—A Survey and Bibliometric Analysis of DEA Literature Regarding the Selection of Sustainable Suppliers

In this research, the bibliometric analysis was conducted as a way of summarizing past findings in a research field (in this case, findings on the selection of sustainable suppliers using DEA). A systematic and comprehensive search was carried out in different databases, including the Web of Science (WoS) and Scopus databases. Relevant databases for this research were identified, and the focus was on the peer-review journals that are cited in Scopus and WoS (SSCI and SCI papers). Table 1 and Table 2 present search strategies in WoS (SSCI and SCI) and Scopus (1990–2022 for the WoS database and 1980–2022 for the Scopus database). In the mentioned periods, the papers appear by searching the mentioned databases according to the search strategy.

Through the first part of the search, WoS and Scopus were checked using keywords: “Data Envelopment Analysis” + “Supplier” + “Sustainable”. This approach resulted in 813 hits (795 in Scopus and 18 in WoS). The aforementioned keywords are searched in “Title, Abstract, and Keywords” of documents belonging to Scopus and WoS databases.

The selection process of the papers for the bibliometric analysis is presented in Figure 1. According to Figure 1, the keywords “Data Envelopment Analysis + Supplier + Sustainable” were searched in “Title, Abstract, and Keywords” of documents belonging to WoS and Scopus databases. However, after merging all papers, 14 papers that were found in both databases were excluded. Therefore, 799 papers remained for the analysis. After reviewing the abstracts and keywords of all 799 papers, the papers that did not report the description of Data Envelopment Analysis + Supplier + Sustainable were eliminated. The paper was considered relevant if it fit the criterion of specifically covering the area of using DEA in the selection of sustainable suppliers (Figure 1). Finally, 87 publications remained after applying this exclusion criterion, and these 87 represent the basis for further analysis, including journal papers (JP), conference papers (CP), and review (RW) found in the DEA literature regarding the selection of sustainable suppliers using the DEA approach (2003–2022 regarding the Scopus database and 2017–2022 regarding the WoS database according to the search strategy).

3.2. Publication Years, Document Types, and Keywords Analysis

In the first step of the bibliometric analysis, the DEA literature regarding the selection of sustainable suppliers using the DEA approach is analyzed based on publication frequency during time horizon, document types, and the number of authors. The statistics based on basic information from the DEA literature regarding the selection of sustainable suppliers using the DEA approach are presented in Figure 2a–c. As shown in Figure 2a–c, the popularity of the DEA literature regarding the selection of sustainable suppliers using the DEA approach has increased, and a growing trend is apparent from 2017 until now, but it is also expected for the DEA approach in the field of selection of sustainable suppliers to grow even more in the future. This is understandable considering the overall increasing trend and awareness regarding sustainability in business and its positive impact as “a source of success, innovation, and profitability of companies” [44] throughout the world.

Sustainable development as well as sustainable management are global goals which require a long-term strategy of policy coordination for economic, social, and environmentally sustainable development. Today’s businesses should strive to ensure and harmonize their own economic development and progress on the one hand and protect the environment on the other. The concept of sustainable development originated and began being mentioned in the 1980s, and the focus itself is on the connection between the development of society and environmental protection. Respecting the concept of sustainable development and doing business sustainably require companies to define strategies and create an economy that can regenerate itself and be sustainable. In order to ensure sustainable development in the company’s business, numerous business transformations—from digital to technological—are necessary. This should all be conducted while respecting economic concepts which hold great potential and are possible solutions for sustainable development. Today, the strategy of sustainable development should be an integral part of the corporate strategy of the company, creating a business policy of the company aimed at the system of sustainable development. This way it contributes to the balance of company management through the cooperation of dynamic business and economic activities of the set system and changes caused by a dynamic ecological system.

The word cloud of the DEA literature regarding the selection of sustainable suppliers using the DEA approach based on the most popular keywords in DEA is presented in Figure 3. A word cloud is “a visualization technique that provides the user with an overview of the content of a collection of texts”, where the word size in the word cloud represents the frequency with which the words appear in the text collection [45]. Moreover, a “word cloud is a visual representation of word frequency”, which means that the larger the words, the more frequently does the term appear within the text collection [46]. Word clouds have been widely used in bibliometric analysis as a guidance tool that helps identify the focus of the large set of published articles. The word cloud in this article was presented with the text mining technique and the word cloud option in R (Figure 3). This simple analysis and the tool we used prove that the search strategy of this article was chosen correctly, as the most popular keywords and key phrases which appear in the word cloud of the selection of sustainable suppliers with DEA also appear in the search keywords.

3.3. Authors and Journals Analysis

In the second step of the bibliometric analysis, statistics based on authors and journals in the DEA literature regarding the selection of sustainable suppliers using the DEA approach are extracted. The top contributing authors and the most influential journals in the DEA field regarding the selection of sustainable suppliers using the DEA approach are proposed in Table 3 and Table 4, respectively.

The identification of the top contributing authors presents the authors who have the highest impact in the research field. In addition, the presentation of the top journals helps researchers to identify journals that are suitable to submit their papers. In Table 3, the number of research works by each author is presented separately by the type of publication (JP, CP) and the order of the author (1st, 2nd, 3rd, 4th) among the authors of the research.

As seen in Table 3, Saen R.F. from the Department of Industrial Management, Faculty of Management and Accounting, Karaj Branch, Islamic Azad University, with 18 co-authored research works, dominates the list of top contributing authors in the DEA literature regarding the selection of sustainable suppliers using the DEA approach. Izadikhah M. from the Department of Mathematics, College of Science, Arak Branch, Islamic Azad University with 7 research works and one co-authored paper is also a top contributing author. The authors’ analysis of the DEA field regarding the selection of sustainable (green) suppliers using the DEA approach from a qualitative aspect shows that authors Kuo R.J., Wang Y.C., and Tien F.C. are the pioneering researchers in the DEA field regarding the selection of sustainable (green) suppliers using the DEA approach and are the top-cited authors in this field.

According to Table 4, articles in the DEA field regarding the selection of sustainable (green) suppliers using the DEA approach were published in top-tier journals such as Annals of Operations Research (3 articles), Benchmarking (2 articles), Computers and Industrial Engineering (4 articles), Computers and Operations Research (2 articles), Environmental Science and Pollution Research (2 articles), Group Decision and Negotiation (2 articles), Industrial Management and Data (2 articles), International Journal of Industrial and Systems Engineering (2 articles), International Journal of Production Economics (6 articles), Journal of Cleaner Production (6 articles), Neural Computing and Applications (2 articles), Sustainability (Switzerland) (6 articles), and Transportation Research Part D: Transport and Environment (2 articles). This indicates the attractiveness of the DEA approach regarding the selection of sustainable (green) suppliers using the DEA methodology.

3.4. Affiliations Analysis

As a third step of the bibliometric analysis, a statistical overview of the top contributing affiliations in sustainable supplier selection with the application of DEA methodology was conducted and is presented in Table 5. A brief presentation of the most influential countries/territories as well as organizations/institutions along with the number of publications containing the DEA literature regarding the selection of sustainable suppliers using the DEA approach is laid out. According to the results shown in Table 5, the top 10 contributing countries in the DEA area regarding the selection of sustainable suppliers using the DEA approach are Iran, China, the United States, India, Taiwan, Hungary, Malaysia, Australia, Germany, and Canada. On the other hand, the top five contributing universities and institutions in the DEA field regarding the selection of sustainable suppliers using the DEA approach are Islamic Azad University, North Carolina State University, Indian Institute of Technology, Corvinus University of Budapest, and the National Institute of Industrial Engineering.

3.5. Citations Analysis

Out of the total number of published articles implementing DEA in the sustainable supplier selection (87), the number of cited articles is 73 (83.91%). In the fourth phase of this bibliometric analysis, the statistics regarding citations of the surveyed papers were analyzed based on data from the Scopus and WoS databases.

The list of the most cited research in the field of sustainable supplier selection is presented in Table 6. As seen in Table 6, the most cited documents in the research field of sustainable supplier selection with DEA were published in top-tier journals such as the International Journal of Production Economics (4 articles), Journal of Cleaner Production (4 articles), Annals of Operations Research (3 papers), Computers and Industrial Engineering (3 articles), Environmental Science and Pollution Research (2 papers), Computers and Operations Research (2 articles), Group Decision and Negotiation (2 articles), Neural Computing and Applications (2 articles), Sustainability (2 articles), Transportation Research Part D: Transport and Environment (2 articles), Benchmarking (1 article), Omega (1 article), Processes (1 article), Mathematics (1 article), Advanced Materials Research (1 article), and Applied Soft Computing Journal (1 article). The citation analysis implies the importance of the selection of sustainable suppliers.

Figure 4 shows the citations’ overview and tracks citation data generated from a set of selected documents (87 selected documents) and introduces citation frequency in the field of sustainable supplier selection with the DEA literature with their number of documents as well as the h-index of sustainable supplier selection with the DEA literature. The h-index is based on the highest number of papers included that have had at least the same number of citations.

The citation analysis enables researchers “to identify the most cited studies in a specific field and the evolution of their popularity over time” and it is believed that “the number of citations to a particular document indicates its impact on literature” [47].

3.6. Co-Authorship Analysis

With the help of the VOSviewer software, the network visualization for country co-authorship was created (Figure 5), and the network visualization of the top authors’ density visualization is presented in Figure 6. A total of 32 countries were identified as contributing countries. However, only 7 of them are shown in Figure 5 since they are the most contributing countries in the area of sustainable supplier selection with DEA. Moreover, as shown in Figure 6, five authors appear in the density visualization due to their contribution to this field (these are Saen R.F., Izadikhah M., Yousefi S., and Azadi M.).

This analysis attempts to “shed light on clustered research among scholars from a particular region”, and this information can be used “to justify and spark new research among scholars in underrepresented regions”. Moreover, this visualization presents mapped collaborations between different countries and different authors and could inspire “prospective scholars to reach out to established and trending scholars in the respective research field” [48].

3.7. Research Gap

Regarding the research area, the obvious problem is the lack of existing research. By identifying relevant publications and cited literature in selected publications, keywords, abstract, and title, the DEA application in the selection of sustainable suppliers is showing a research gap. The DEA application in the selection of sustainable suppliers has not been studied and elaborated enough. It is required to extend the research in this area so it could be useful to academics but also managers in practice. It is necessary to encourage researchers to research in this area and to make the area interesting to as many researchers as possible in order to raise awareness of the importance of sustainable suppliers for the development of a sustainable supply chain. The key question is whether researchers and practitioners recognize and understand the DEA methodology, its application, and benefits so that it can be used in the selection of sustainable suppliers. There may be a problem of misunderstanding the DEA and how it is used in selecting sustainable suppliers, where the goal should be to make it recognizable and understandable to the wider community (both researchers and practitioners). It is important to address how the DEA exists since the 1980s and has proved useful in many areas, which is why it is necessary that the area related to the selection of sustainable suppliers will be also recognized through the DEA.

With all of the mentioned above, this review is relevant for academics but also has a practical relevance. It provides exploratory insights of the DEA benefits in the area of the selection of sustainable suppliers. This review can also make managers within the supply chain aware of choosing sustainable suppliers through the DEA methodology. This research can also assist business in setting sustainable supply chain strategies through the DEA by connecting with sustainable suppliers. This research should be viewed as an addition on the current way and process of how companies are selecting (sustainable) suppliers in adjusting their business strategy and thus affecting its supply chain. Within the process of selecting sustainable suppliers using DEA, the literature emphasized the importance of environmental factors as one of the main reasons where DEA can have the greatest impact in such selection. Therefore, the future role of DEA in the selection of sustainable suppliers should be further researched to determine how its influence will shift and what effects it will have. Through presented research papers and its analysis, it is obvious that it can assist academics but also managers in practice in making and predicting how and which factors are crucial in changing behavior in the supply chain by selecting sustainable suppliers using DEA.

4. Discussion, Implications, and Future Trends for DEA in the Selection of Sustainable Suppliers

The DEA methodology is a powerful, popular mathematical programming technique that has found a wide application in many industries and decision-making processes. It is often combined with different MCDM techniques, which according to the survey were mostly techniques for order of preference by similarity to ideal solution (TOPSIS), analytic hierarchy process (AHP), Artificial neural network (ANN), analytic network process (ANP), fuzzy DEA, Green DEA (GDEA) models, etc. As opposed to the traditional approach to the supplier selection problem, “modern approaches are also considering the green aspects because of the rising importance of environmental impacts to comply with environmental standard” [1].

Additionally, there are some suggested guidelines and directions for further research in the sustainable supplier selection with the DEA area, which are as follows: there is an evident trend in developing the GDEA (green DEA) models for practical implications, and that is encouraged and expected from both scholars and managerial teams and future analysts; the AHP, TOPSIS, and ANN MCDM models in combination with DEA are increasingly popular in this area and “are already linked to strategic purchasing practices and relate to a more advanced level” [49]. The fuzzy DEA models have been increasingly implemented in this area (especially in the period from 2015 to 2022). As shown in Table 7, the most common research areas in which the DEA and MCDM models were combined are Artificial Intelligence, Supply Chain Management, Machine Learning, and Supplier Selection. However, there is huge potential for an even greater use of the combined DEA and MCDM models in many different industries in the future.

Thus, “to solve the MCDM problem in supplier selection (due to fact that sustainable supplier selection is here in focus) the study was conducted and it proposes a new fuzzy DEA model based on the super-efficiency DEA model, and confirms their interconnectedness” [50] (p. 91).

In addition, finally, as [47] put it, “the most valuable suggestion for future studies would be to apply existing or extended DEA approaches in real-world problems and applications”.

The DEA is expected to experience an even greater use by managers and data analysts in the process of decision-making. Moreover, managers ought to use the DEA as an efficiency-and-performance “supporting tool in the decision-making process” [57]. Having in mind the primary goal of the DEA, which is to provide a classification of relatively efficient and relatively inefficient units, this opens up vast opportunities for its wider use in supply chain management and specifically in the selection of sustainable suppliers. DEA has a wide usage in different industries, which proves its usefulness and benefits. As the research itself shows, DEA applications are experiencing growth from 2017 until now (2022), and researchers are realizing the necessity to research this area. However, the field is still insufficiently researched, and too few researchers deal with and analyze the advantages and benefits of DEA in selecting sustainable suppliers. Although the DEA methodology is not a new concept, the increase in its usage is evident in a number of different industries and different fields of business operations through the years, and its application changes how companies operate.

The results of the presented bibliometric literature review show that sustainable development and sustainable suppliers nowadays are imperative for doing business, and DEA influences and helps this process. Sustainable development and sustainable suppliers affect the business process and increase the efficiency and effectiveness of a company. A successful supply chain depends on quality suppliers and, above all else, sustainable suppliers, which affect the long-term growth and development of the company and make for greater visibility and respect for the environmental component of the company. The goal of the companies is to build a resilient supply chain while striving to strengthen all business processes and all business components of the company where sustainable suppliers are one of the necessary conditions that companies need to meet and fulfill. It is evident that according to the growing trend of the observed articles, this area is becoming the interest of an increasing number of researchers (although currently insufficient) and that the imperative of the company is sustainable growth and development, where sustainable suppliers are one of the key factors. In the field of scientific research, it is necessary to place increasing emphasis on the importance of sustainable suppliers and to encourage research in this area in order to highlight their importance in today’s modern and global business. The idea and the aim of the paper was to present and show that sustainable suppliers are indispensable stakeholders in the supply chain that facilitate and contribute to the success of the entire process of the supply chain.

This study provides a summary of the work that was conducted to develop models and approaches for the selection of sustainable suppliers. Moreover, an overview of the real-life applications of DEA in the sustainable supplier selection was created in Figure 7. The process of selection of a sustainable supplier with DEA has been widely applied in various industries. According to Figure 7, DEA was extensively used in real-life decision-making situations, such as the food processing industry (oil and palm oil industry), the manufacturing industry, the textile and apparel industry, the semiconductor industry, the automotive industry, the logistics industry, the payment and banking industry, the cell-phone industry, the construction industry, the urban transportation industry, and even the civil society industry and the blockchain technology industry. This implies the applicability and effectiveness of DEA in the decision-making process in supply chain management.

Moreover, this research area needs more empirical, theoretical, and quantitative evidence and is the scope for many studies to be carried out in the future. The conducted research gives a brief overview of a research field that is insufficiently explored through the scientific literature and presents a wide area for further research.

5. Conclusions

This study was designed to survey articles that implement the DEA methodology in sustainable supplier evaluation and selection which asked for an analysis of bibliometric data on DEA articles in the selection of sustainable suppliers in peer-reviewed journals listed in the Scopus and Elsevier Web of Science databases. This research considered a range of 87 articles published over a period of 13 years (from 2010 to 2022) (Appendix A) and, thus, the state of the art on this subject could be properly mapped. This mapping considered bibliometrics, scope, the DEA models and extensions used, and the interfaces with other techniques and methodologies. The findings of this research indicate that the impact of DEA in the selection of sustainable suppliers on the supply chain is on the verge to disrupt current supply chain processes and overall business processes in companies affecting its sustainability on market. Moreover, this is the research area that needs more empirical evidence, theoretical and quantitative evidence, and presents a scope for many studies to be carried out in the future. DEA in selection of sustainable suppliers brings many challenges to managing the supply chain where supply chain operations will change accordingly with the use of DEA. The main question for researchers and managers remains how to integrate and manage DEA methodology in the selection of sustainable suppliers as an efficient and effective way in order to optimize the whole supply chain of the organization.

The findings reveal the top contributing authors (Saen R.F., Izadikhah M., Yousefi S., and Azadi M.), the top contributing journals (International Journal of Production Economics, Journal of Cleaner Production and Sustainability-Switzerland), the top contributing countries (Iran, China, United States, India, Taiwan, Hungary, Malaysia, Australia, Germany, and Canada), the top contributing affiliations (Islamic Azad University, North Carolina State University, Indian Institute of Technology, Corvinus University of Budapest, and the National Institute of Industrial Engineering), and present network visualizations of the countries and authors using the VOSviewer software. Moreover, a word cloud with the most frequently used keywords was presented. It should be noted that literature in DEA applied to the process of selection of sustainable suppliers relies on a small number of authors and journals. Namely, studies revolving around sustainable supplier selection with the application of DEA are particularly concentrated and result from the work of a limited academic research network (as shown in Figure 5 and Figure 6). On top of that, one author (Saen R.F.) represents more than 20% of all the publications. As for the analyzed geographic areas, around 70% of all the published papers focus on Iran.

The bibliometric review shows that the use of DEA in sustainable supplier evaluation and selection has shown an increasing trend in the past decade. However, the number of surveyed 87 papers is relatively modest, and even though most of them developed models with practical implications, the results are in line with [49], who suggest that “only a small part of this articles presented a practical application, suggesting that the adoption of DEA is still low in the industry”.

This research, however, is not without limitations. Although three researchers were included in the content analysis, the relevance criterion used for the survey analysis of the papers remained somewhat subjective. Moreover, even though the two most reputable scientific databases (Web of Science and Scopus) were consulted, there may have been relevant studies and papers that were not indexed in the abovementioned databases that would have otherwise enriched the study.

In conclusion, there are many opportunities for future investigation and methodological developments of the DEA in many areas related to this research. By analyzing the relevant scientific literature and by the results of bibliometric research, it is possible to conclude that there is currently insufficient awareness and use of DEA in the selection of sustainable suppliers, but through the evaluation of 87 papers, the recognition and appreciation of DEA are visible, which should ultimately lead to its increasing application. Finally, this review depicts directions which should be investigated in future work (the DEA methodology could be used in combination with other models such as GDEA, AHP, TOPSIS, and ANN MCDM models), and the present review provides a sound and solid database for such future studies.

Author Contributions

Conceptualization, K.F.Č. and I.M.; methodology, K.F.Č. and I.M.; software, K.F.Č.; validation, K.F.Č. and I.M.; formal analysis, K.F.Č. and I.M.; investigation, K.F.Č. and I.M.; resources, I.M.; data curation, K.F.Č.; writing—original draft preparation, K.F.Č. and I.M.; writing—review and editing, K.F.Č. and I.M.; visualization, J.L.; supervision, K.F.Č., I.M. and J.L.; project administration, I.M. and J.L. All authors have read and agreed to the published version of the manuscript.

Funding

Institutional Review Board Statement

Informed Consent Statement

Data Availability Statement

Conflicts of Interest

Appendix A. List of Selected Papers for Bibliometric Literature Review

References

1. Dutta, P.; Jaikumar, B.; Arora, M.S. Applications of data envelopment analysis in supplier selection between 2000 and 2020: A literature review. Ann. Oper. Res. 2021, 1–56. [Google Scholar] [CrossRef]
2. Bao, X.; Li, F. A methodology for supplier selection under the curse of dimensionality problem based on fuzzy quality function deployment and interval data envelopment analysis. PLoS ONE 2021, 16 , e0253917. [Google Scholar] [CrossRef] [PubMed]
3. Roostaee, R.; Izadikhah, M.; Lotfi, F.H.; Rostamy-Malkhalifeh, M. A Multi-Criteria Intuitionistic Fuzzy Group Decision Mak-ing Method for Supplier Selection with VIKOR Method. Int. J. Fuzzy Syst. 2012, 2 , 1–17. [Google Scholar]
4. Moghaddas, Z.; Tosarkani, B.M.; Yousefi, S. A Developed Data Envelopment Analysis Model for Efficient Sustainable Supply Chain Network Design. Sustainability 2022, 14 , 262. [Google Scholar] [CrossRef]
5. Charnes, A.; Cooper, W.; Rhodes, E. Measuring the efficiency of decision-making units. Eur. J. Oper. Res. 1978, 2 , 429–444. [Google Scholar] [CrossRef]
6. Cooper, W.W.; Seiford, L.M.; Tone, K. Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software , 2nd ed.; Springer: New York, NY, USA, 2007. [Google Scholar]
7. Emrouznejad, A.; Yang, G.L. A survey and analysis of the first 40 years of scholarly literature in DEA: 1978–2016. Socio-Econ. Plan. Sci. 2018, 61 , 4–8. [Google Scholar] [CrossRef]
8. Crkvenčić, M.; Buntak, K.; Krpan, L. Upravljanje Lancima Opskrbe ; Sveučilište Sjever: Koprivnica, Croatia, 2018. [Google Scholar]
9. Zelenika, R.; Plic Skender, H. Upravljanje Logističkim Mrežama ; Ekonomski Fakultet Sveučilišta u Rijeci: Rijeka, Croatia, 2007. [Google Scholar]
10. Cavone, G.; Dotoli, M.; Epicoco, N.; Morelli, D.; Seatzu, C. A Game-theoretical Design Technique for Multi-Stage Supply Chains under Uncertainty. In Proceedings of the 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE), Munich, Germany, 20–24 August 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 528–533. [Google Scholar] [CrossRef]
11. Borozan, Đ. Makroekonomija ; Ekonomski Fakultet u Osijeku: Osijek, Croatia, 2006. [Google Scholar]
12. Pravdić, V. Sustainability and Sustainable Development: The Use in Policies and the Ongoing Debate on These Terms. Croa-Tian Int. Relat. Rev. 2001, 7 , 93–100. [Google Scholar]
13. Solow, R.M. An almost practical step towards sustainability. Resour. Policy 1993, 19 , 162–172. [Google Scholar] [CrossRef]
14. Pearce, D.W.; Barbier, E.; Markandya, A. Sustainable Development: Economics and Environment in the Third World ; Elgar, E., Hants, G.B., Brookfield, V.T., Eds.; Routledge: Abingdon, UK, 1990. [Google Scholar]
15. Amindoust, A.; Shamsuddin, A.; Saghafinia, A. Using Data Envelopment Analysis for Green Supplier Selection in Manufac-turing under Vague Environment. Adv. Mater. Res. 2012, 622–623 , 1682–1685. [Google Scholar] [CrossRef]
16. Mahdiloo, M.; Saen, R.F.; Lee, K.-H. Technical, environmental and eco-efficiency measurement for supplier selection: An extension and application of data envelopment analysis. Int. J. Prod. Econ. 2015, 168 , 279–289. [Google Scholar] [CrossRef]
17. Shabanpoura, H.; Fathi, A.; Yousefi, S.; Saen, R.F. Ranking sustainable suppliers using congestion approach of data envelop-ment analysis. J. Clean. Prod. 2019, 240 , 118190. [Google Scholar] [CrossRef]
18. Wang, C.-N.; Tsai, H.-T.; Ho, T.-P.; Nguyen, V.-T.; Huang, Y.-F. Multi-Criteria Decision Making (MCDM) Model for Supplier Evaluation and Selection for Oil Production Projects in Vietnam. Processes 2020, 8 , 134. [Google Scholar] [CrossRef] [Green Version]
19. Hatami-Marbini, A.; Hekmat, S.; Agrell, P.J. A strategy-based framework for supplier selection: A grey PCA-DEA approach. Oper. Res. Int. J. 2022, 22 , 263–297. [Google Scholar] [CrossRef]
20. Dobos, I.; Vörösmarty, G. Green supplier selection and evaluation using DEA-type composite indicators. Int. J. Prod. Econ. 2014, 157 , 273–278. [Google Scholar] [CrossRef]
21. Cavone, G.; Dotoli, M.; Epicoco, N.; Morelli, D.; Seatzu, C. Design of Modern Supply Chain Networks Using Fuzzy Bargaining Game and Data Envelopment Analysis. IEEE Trans. Autom. Sci. Eng. 2020, 17 , 1221–1236. [Google Scholar] [CrossRef]
22. Moheb-Alizadeh, H.; Handfield, R. Sustainable supplier selection and order allocation: A novel multi-objective pro-gramming model with a hybrid solution approach. Comput. Ind. Eng. 2019, 129 , 192–209. [Google Scholar] [CrossRef]
23. Bai, C.; Sarkis, J. Determining and applying sustainable supplier key performance indicators. Supply Chain Manag. 2014, 19 , 275–291. [Google Scholar] [CrossRef]
24. Shi, P.; Yan, B.; Shi, S.; Ke, C. A decision support system to select suppliers for a sustainable supply chain based on a systematic DEA approach. Inf. Technol. Manag. 2015, 16 , 39–49. [Google Scholar] [CrossRef]
25. Jauhar, S.K.; Pant, M.; Nagar, M.C. Differential evolution for sustainable supplier selection in pulp and paper industry: A DEA based approach. Comput. Methods Mater. Sci. 2015, 15 , 118–126. [Google Scholar]
26. Kumar, A.; Jain, V.; Kumar, S.; Chandra, C. Green supplier selection: A new genetic/immune strategy with industrial applica-tion. Enterp. Inf. Syst. 2016, 10 , 911–943. [Google Scholar] [CrossRef]
27. Jain, V.; Kumar, S.; Kumar, A.; Chandra, C. An integrated buyer initiated decision-making process for green supplier selection. J. Manuf. Syst. 2016, 41 , 256–265. [Google Scholar] [CrossRef]
28. Zarbakhshnia, N.; Jaghdani, T.J. Sustainable supplier evaluation and selection with a novel two-stage DEA model in the presence of uncontrollable inputs and undesirable outputs: A plastic case study. Int. J. Adv. Manuf. Technol. 2018, 97 , 2933–2945. [Google Scholar] [CrossRef]
29. Boudaghi, E.; Saen, R.F. Developing a novel model of data envelopment analysis–discriminant analysis for predicting group membership of suppliers in sustainable supply chain. Comput. Oper. Res. 2018, 89 , 348–359. [Google Scholar] [CrossRef]
30. Sharafi, H.; Soltanifar, M.; Hosseinzadeh Lotfi, F. Selecting a green supplier utilizing the new fuzzy voting model and the fuzzy combinative distance-based assessment method. EURO J. Decis. Processes 2022, 10 , 100010. [Google Scholar] [CrossRef]
31. Fotova Čikovič, K.; Lozić, J. Application of Data Envelopment Analysis (DEA) in Information and Communication Technol-ogies. Teh. Glas. 2022, 16 , 129–134. [Google Scholar] [CrossRef]
32. Cavone, G.; Dotoli, M.; Epicoco, N.; Seatzu, C. Intermodal terminal planning by Petri Nets and Data Envelopment Analysis. Control. Eng. Pract. 2017, 69 , 9–22. [Google Scholar] [CrossRef]
33. Cavone, G.; Dotoli, M.; Epicoco, N.; Seatzu, C. A decision making procedure for robust train rescheduling based on mixed inte-ger linear programming and Data Envelopment Analysis. Appl. Math. Model. 2017, 52 , 255–273. [Google Scholar] [CrossRef]
34. Obadić, A.; Aristovnik, A. Relative Efficiency of Higher Education in Croatia and Slovenia: An International Comparison. Amfiteatru Econ. J. 2011, 13 , 362–376. [Google Scholar]
35. Cvetkoska, V. Data Envelopment Analysis Approach and Its Application in Information and Communication Technologies. In Proceedings of the International Conference on Information and Communication Technologies for Sustainable Agriproduction and Environment (HAICTA 2011), Skiathos, Greece, 8–11 September 2011. [Google Scholar]
36. Cvetkoska, V.; Fotova Čiković, K. Efficiency Analysis of Macedonian and Croatian Banking Sectors with DEA. Econ. Bus. Dev. 2021, 2 , 1–19. [Google Scholar] [CrossRef]
37. Amirteimoori, A.; Zadmirzaei, M.; Hassanzadeh, F. Developing a new integrated artificial immune system and fuzzy non-discretionary DEA approach. Soft Comput. 2021, 25 , 8109–8127. [Google Scholar] [CrossRef]
38. Jorda, P.; Cascajo, R.; Monzón, A. Analysis of the Technical Efficiency of Urban Bus Services in Spain Based on SBM Models. ISRN Civ. Eng. 2012, 2012 , 984758. [Google Scholar] [CrossRef] [Green Version]
39. Škrinjarić, T. Evaluation of environmentally conscious tourism industry: Case of Croatian counties. Tour. Int. Interdiscip. J. 2018, 66 , 254–268. [Google Scholar]
40. Fotova Čiković, K.; Lozić, J.; Milković, M. Applications of data envelopment analysis (DEA) in empirical studies regarding the Croatian tourism. Tour. Int. Interdiscip. J. 2022, accepted. [Google Scholar]
41. Cooper, W.W.; Seiford, L.M.; Tone, K. Introduction to Data Envelopment Analysis and Its Uses: With DEA-Solver Software and References ; Springer: New York, NY, USA, 2006. [Google Scholar]
42. Hodžić, S.; Jurlina Alibegović, D. The efficiency of regional government expenditure in smart tourist destination: The case of Croatia. Tour. South East Eur. 2019, 5 , 307–318. [Google Scholar] [CrossRef]
43. Giustiniani, A.; Ross, K. Bank competition and efficiency in the FYR Macedonia. South-East. Eur. J. Econ. 2008, 2 , 145–167. [Google Scholar]
44. Baumgartner, R.J. Managing Corporate Sustainability and CSR: A Conceptual Framework Combining Values, Strategies and Instruments Contributing to Sustainable Development. Corp. Soc. Responsib. Environ. Manag. 2014, 21 , 258–271. [Google Scholar] [CrossRef]
45. Tessem, B.; Bjørnestad, S.; Chen, W.; Nyre, L. Word cloud visualisation of locative information. J. Locat. Based Serv. 2015, 9 , 254–272. [Google Scholar] [CrossRef] [Green Version]
46. Atenstaedt, R. Word cloud analysis of the BJGP. Br. J. Gen. Pract. J. R. Coll. Gen. Pract. 2012, 62 , 148. [Google Scholar] [CrossRef] [Green Version]
47. Peykani, P.; Saen, R.F.; Esmaeili, F.S.S.; Gheidar-Kheljani, J. Window data envelopment analysis approach: A review and bibliometric analysis. Expert Syst. 2021, 38 , e12721. [Google Scholar] [CrossRef]
48. Donthu, N.; Kumar, S.; Mukherjee, D.; Pandey, N.; Lim, W.M. How to conduct a bibliometric analysis: An overview and guidelines. J. Bus. Res. 2021, 133 , 285–296. [Google Scholar] [CrossRef]
49. Vorosmarty, G.; Dobos, I. A literature review of sustainable supplier evaluation with Data Envelopment Analysis. J. Clean. Prod. 2020, 264 , 121672. [Google Scholar] [CrossRef]
50. Tsai, C.M.; Lee, H.S.; Gan, G.Y. A New Fuzzy DEA Model for Solving the MCDM Problems in Supplier Selection. J. Mar. Sci. Technol. 2021, 29 , 89–95. [Google Scholar] [CrossRef]
51. Fallahpour, A.; Olugu, E.U.; Musa, S.N.; Khezrimotlagh, D.; Wong, K.Y. An integrated model for green supplier selection under fuzzy environment: Application of data envelopment analysis and genetic programming approach. Neural Comput. Appl. 2017, 27 , 707–725. [Google Scholar] [CrossRef]
52. Amindoust, A. Supplier selection considering sustainability measures: An application of weight restriction fuzzy-DEA ap-proach. RAIRO-Oper. Res. 2018, 52 , 981–1001. [Google Scholar] [CrossRef] [Green Version]
53. Rashidi, K.; Cullinane, K. A comparison of fuzzy DEA and fuzzy TOPSIS in sustainable supplier selection: Implications for sourcing strategy. Expert Syst. Appl. 2019, 121 , 266–281. [Google Scholar] [CrossRef]
54. Izadikhah, M.; Farzipoor Saen, R.; Ahmadi, K.; Shamsi, M. How to use fuzzy screening system and data envelopment analysis for clustering sustainable suppliers? A case study in Iran. J. Enterp. Inf. Manag. 2020, 34 , 199–229. [Google Scholar] [CrossRef]
55. Ngobeni, V.; Breitenbach, M.C. Production and scale efficiency of South African water utilities: The case of water boards. Water Policy 2021, 23 , 862–879. [Google Scholar] [CrossRef]
56. Aggarwal, I.; Gunreddy, N.; Rajan, A.J. A Hybrid Supplier Selection Approach Using Machine Learning and Data Envelopment Analysis. In Proceedings of the 2021 Innovations in Power and Advanced Computing Technologies, Kuala Lumpur, Malaysaia, 27–29 November 2021; IEEE: Piscataway, NJ, USA, 2021. [Google Scholar]
57. Cvetkoska, V.; Savić, G. DEA in banking: Analysis and visualization of bibliometric data. Data Envel. Anal. J. 2021, 5 , 455–485. [Google Scholar] [CrossRef]
58. Amindoust, A. A resilient-sustainable based supplier selection model using a hybrid intelligent method. Comput. Ind. Eng. 2018, 126 , 122–135. [Google Scholar] [CrossRef]
59. Amindoust, A.; Ahmed, S.; Saghafinia, A. Supplier performance measurement of palm oil industries from a sustainable point of view in malaysia. BioTechnol. Indian J. 2012, 6 , 155–158. [Google Scholar]
60. Azadi, M.; Izadikhah, M.; Ramezani, F.; Khadeer, F. A mixed ideal and anti-ideal DEA model: An application to evaluate cloud service providers. IMA J. Manag. Math. 2020, 31 , 233–256. [Google Scholar] [CrossRef]
61. Azadi, M.; Jafarian, M.; Saen, R.F.; Mirhedayatian, S.M. A new fuzzy DEA model for evaluation of efficiency and effectiveness of suppliers in sustainable supply chain management context. Comput. Oper. Res. 2015, 54 , 274–285. [Google Scholar] [CrossRef]
62. Azadi, M.; Mirhedayatian, S.M.; Saen, R.F.; Hatamzad, M.; Momeni, E. Green supplier selection: A novel fuzzy double frontier data envelopment analysis model to deal with undesirable outputs and dual-role factors. Int. J. Ind. Syst. Eng. 2017, 25 , 160–181. [Google Scholar] [CrossRef]
63. Bajec, P.; Tuljak-Suban, D.; Zalokar, E. A distance-based AHP-DEA super-efficiency approach for selecting an electric bike sharing system provider: One step closer to sustainability and a win–win effect for all target groups. Sustainability 2021, 13 , 549. [Google Scholar] [CrossRef]
64. Chang, K.-H. A novel contractor selection technique using the extended PROMETHEE II method. Math. Probl. Eng. 2021, 2021 , 3664709. [Google Scholar] [CrossRef]
65. Cheaitou, A.; Larbi, R.; Al Housani, B. Decision making framework for tender evaluation and contractor selection in public organizations with risk considerations. Socio-Econ. Plan. Sci. 2019, 68 , 100620. [Google Scholar] [CrossRef]
66. Choudhury, N.; Raut, R.D.; Gardas, B.B.; Kharat, M.G.; Ichake, S. Evaluation and selection of third party logistics services providers using data envelopment analysis: A sustainable approach. Int. J. Bus. Excell. 2018, 14 , 427–453. [Google Scholar] [CrossRef]
67. Dania, W.A.P.; Sitepu, I.B.B.; Rucitra, A.L. Collaboration quality assessment in the sustainable rice supply chain by using an integrated model of QFD-FANP-DEA: A case study of the rice industry in Malang. IOP Conf. Ser. Earth Environ. Sci. 2021, 733 , 012041. [Google Scholar] [CrossRef]
68. Dania, W.A.P.; Xing, K.; Amer, Y. The assessment of collaboration quality: A case of sugar supply chain in Indonesia. Int. J. Product. Perform. Manag. 2022, 71 , 504–539. [Google Scholar] [CrossRef]
69. Dobos, I.; Vörösmarty, G. Green supplier selection using a common weights analysis of DEA and EOQ types of order allocation. Manag. Decis. Econ. 2021, 42 , 612–621. [Google Scholar] [CrossRef]
70. Ershadi, M.J.; Qhanadi Taghizadeh, O.; Hadji Molana, S.M. Selection and performance estimation of green lean six sigma projects: A hybrid approach of technology readiness level, data envelopment analysis and ANFIS. Environ. Sci. Pollut. Res. 2021, 28 , 29394–29411. [Google Scholar] [CrossRef] [PubMed]
71. Fakhrzad, M.B.; Nasrollahi, S. A developed model of data envelopment analysis-discriminant analysis for predicting group membership of suppliers in green supply chain. Int. J. Value Chain Manag. 2018, 9 , 378–392. [Google Scholar] [CrossRef]
72. Hatami-Marbini, A.; Agrell, P.J.; Tavana, M.; Khoshnevis, P. A flexible cross-efficiency fuzzy data envelopment analysis model for sustainable sourcing. J. Clean. Prod. 2017, 142 , 2761–2779. [Google Scholar] [CrossRef]
73. Hekmat, S.; Amiri, M.; Madraki, G. Strategic supplier selection in payment industry: A multi-criteria solution for insufficient and interrelated data sources. Int. J. Inf. Technol. Decis. Mak. 2021, 20 , 1711–1745. [Google Scholar] [CrossRef]
74. Huang, Y.; Wang, Y.-M.; Lin, J. Two-stage fuzzy cross-efficiency aggregation model using a fuzzy information retrieval method. Int. J. Fuzzy Syst. 2019, 21 , 2650–2666. [Google Scholar] [CrossRef]
75. Izadikhah, M.; Farzipoor Saen, R. Ranking sustainable suppliers by context-dependent data envelopment analysis. Ann. Oper. Res. 2020, 293 , 607–637. [Google Scholar] [CrossRef]
76. Izadikhah, M.; Farzipoor Saen, R. Solving voting system by data envelopment analysis for assessing sustainability of suppliers. Group Decis. Negot. 2019, 28 , 641–669. [Google Scholar] [CrossRef]
77. Izadikhah, M.; Farzipoor Saen, R.; Ahmadi, K. How to assess sustainability of suppliers in volume discount context? A new data envelopment analysis approach. Transp. Res. Part D Transp. Environ. 2017, 51 , 102–121. [Google Scholar] [CrossRef]
78. Izadikhah, M.; Saen, R.F.; Ahmadi, K. How to assess sustainability of suppliers in the presence of dual-role factor and volume discounts? A data envelopment analysis approach. Asia-Pac. J. Oper. Res. 2017, 34 , 1740016. [Google Scholar] [CrossRef]
79. Izadikhah, M.; Saen, R.F.; Roostaee, R. How to assess sustainability of suppliers in the presence of volume discount and negative data in data envelopment analysis? Ann. Oper. Res. 2018, 269 , 241–267. [Google Scholar] [CrossRef]
80. Jafarzadeh Ghoushchi, S.; Dodkanloi Milan, M.; Jahangoshai Rezaee, M. Evaluation and selection of sustainable suppliers in supply chain using new GP-DEA model with imprecise data. J. Ind. Eng. Int. 2018, 14 , 613–625. [Google Scholar] [CrossRef] [Green Version]
81. Jauhar, S.K.; Amin, S.H.; Zolfagharinia, H. A proposed method for third-party reverse logistics partner selection and order allocation in the cellphone industry. Comput. Ind. Eng. 2021, 162 , 107719. [Google Scholar] [CrossRef]
82. Jauhar, S.K.; Pant, M. Using differential evolution to develop a carbon-integrated model for performance evaluation and selection of sustainable suppliers in Indian automobile supply chain. In Proceedings of the Fifth International Conference on Soft Computing for Problem Solving, Uttarakhand, India, 18–20 December 2016; Springer: Berlin/Heidelberg, Germany, 2016. [Google Scholar]
83. Karimi, A.; Jafarzadeh-Ghoushchi, S.; Mohtadi-Bonab, M.A. Presenting a new model for performance measurement of the sustainable supply chain of Shoa Panjereh Company in different provinces of Iran (case study). Int. J. Syst. Assur. Eng. Manag. 2020, 11 , 140–154. [Google Scholar] [CrossRef]
84. Karimi, B.; Azadi, M.; Farzipoor Saen, R.; Fosso Wamba, S. Theory of binary-valued data envelopment analysis: An application in assessing the sustainability of suppliers. Ind. Manag. Data Syst. 2022, 122 , 682–701. [Google Scholar] [CrossRef]
85. Kaur, H.; Prakash Singh, S. Multi-stage hybrid model for supplier selection and order allocation considering disruption risks and disruptive technologies. Int. J. Prod. Econ. 2021, 231 , 107830. [Google Scholar] [CrossRef]
86. Kumar, A.; Jain, V.; Kumar, S. A comprehensive environment friendly approach for supplier selection. Omega 2014, 42 , 109–123. [Google Scholar] [CrossRef]
87. Kuo, R.J.; Wang, Y.C.; Tien, F.C. Integration of artificial neural network and MADA methods for green supplier selection. J. Clean. Prod. 2010, 18 , 1161–1170. [Google Scholar] [CrossRef]
88. Li, F.; Deng, L.; Li, L.; Cheng, Z.; Yu, H. A two-stage model for monitoring the green supplier performance considering dual-role and undesirable factors. Asia Pac. J. Mark. Logist. 2020, 32 , 253–280. [Google Scholar] [CrossRef]
89. Li, F.; Wu, L.; Zhu, Q.; Yu, Y.; Kou, G.; Liao, Y. An eco-inefficiency dominance probability approach for Chinese banking operations based on data envelopment analysis. Complexity 2020, 2020 , 3780232. [Google Scholar] [CrossRef]
90. Mahmoudi, A.; Abbasi, M.; Deng, X. Evaluating the performance of the suppliers using hybrid DEA-OPA model: A sustainable development perspective. Group Decis. Negot. 2022, 31 , 335–362. [Google Scholar] [CrossRef]
91. Moheb-Alizadeh, H.; Handfield, R. An integrated chance-constrained stochastic model for efficient and sustainable supplier selection and order allocation. Int. J. Prod. Res. 2018, 56 , 6890–6916. [Google Scholar] [CrossRef]
92. Moheb-Alizadeh, H.; Handfield, R.; Warsing, D. Efficient and sustainable closed-loop supply chain network design: A two-stage stochastic formulation with a hybrid solution methodology. J. Clean. Prod. 2021, 308 , 127323. [Google Scholar] [CrossRef]
93. Momeni, M.; Vandchali, H.R. Providing a structured methodology for supplier selection and evaluation for strategic outsourcing. Int. J. Bus. Perform. Supply Chain Model. 2017, 9 , 66–85. [Google Scholar] [CrossRef]
94. Nemati, M.; Farzipoor Saen, R.; Matin, R.K. A data envelopment analysis approach by partial impacts between inputs and desirable-undesirable outputs for sustainable supplier selection problem. Ind. Manag. Data Syst. 2021, 121 , 809–838. [Google Scholar] [CrossRef]
95. Nikabadi, M.S.; Moghaddam, H.F. An integrated approach of adaptive neuro-fuzzy inference system and dynamic data envelopment analysis for supplier selection. Int. J. Math. Oper. Res. 2021, 18 , 503–527. [Google Scholar] [CrossRef]
96. Nikfarjam, H.; Rostamy-Malkhalifeh, M.; Noura, A. A new robust dynamic data envelopment analysis approach for sustainable supplier evaluation. Adv. Oper. Res. 2018, 2018 , 7625025. [Google Scholar] [CrossRef] [Green Version]
97. Pishchulov, G.; Trautrims, A.; Chesney, T.; Gold, S.; Schwab, L. The voting analytic hierarchy process revisited: A revised method with application to sustainable supplier selection. Int. J. Prod. Econ. 2019, 211 , 166–179. [Google Scholar] [CrossRef]
98. Rajak, S.; Parthiban, P.; Dhanalakshmi, R. A DEA model for evaluation of efficiency and effectiveness of sustainable transportation systems: A supply chain perspective. Int. J. Logist. Syst. Manag. 2021, 40 , 220–241. [Google Scholar] [CrossRef]
99. Rashidi, K. AHP versus DEA: A comparative analysis for the gradual improvement of unsustainable suppliers. Benchmarking 2020, 27 , 2283–2321. [Google Scholar] [CrossRef]
100. Raut, R.; Kharat, M.; Kamble, S.; Kumar, C.S. Sustainable evaluation and selection of potential third-party logistics (3PL) providers: An integrated MCDM approach. Benchmarking 2018, 25 , 76–97. [Google Scholar] [CrossRef]
101. Ryu, Y.; Sueyoshi, T. Examining the relationship between the economic performance of technology-based small suppliers and socially sustainable procurement. Sustainability 2021, 13 , 7220. [Google Scholar] [CrossRef]
102. Shabanpour, H.; Yousefi, S.; Farzipoor Saen, R. Future planning for benchmarking and ranking sustainable suppliers using goal programming and robust double frontiers DEA. Transp. Res. Part D Transp. Environ. 2017, 50 , 129–143. [Google Scholar] [CrossRef]
103. Shabanpour, H.; Yousefi, S.; Saen, R.F. Forecasting efficiency of green suppliers by dynamic data envelopment analysis and artificial neural networks. J. Clean. Prod. 2017, 142 , 1098–1107. [Google Scholar] [CrossRef]
104. Soltanifar, M.; Sharafi, H. A modified DEA cross efficiency method with negative data and its application in supplier selection. J. Comb. Optim. 2022, 43 , 265–296. [Google Scholar] [CrossRef]
105. Tavana, M.; Nazari-Shirkouhi, S.; Farzaneh Kholghabad, H. An integrated quality and resilience engineering framework in healthcare with Z-number data envelopment analysis. Health Care Manag. Sci. 2021, 24 , 768–785. [Google Scholar] [CrossRef]
106. Tavana, M.; Shabanpour, H.; Yousefi, S.; Farzipoor Saen, R. A hybrid goal programming and dynamic data envelopment analysis framework for sustainable supplier evaluation. Neural Comput. Appl. 2017, 28 , 3683–3696. [Google Scholar] [CrossRef]
107. Tavassoli, M.; Saen, R.F.; Zanjirani, D.M. Assessing sustainability of suppliers: A novel stochastic-fuzzy DEA model. Sustain. Prod. Consum. 2020, 21 , 78–91. [Google Scholar] [CrossRef]
108. Torres-Ruiz, A.; Ravindran, A.R. Use of interval data envelopment analysis, goal programming and dynamic eco-efficiency assessment for sustainable supplier management. Comput. Ind. Eng. 2019, 131 , 211–226. [Google Scholar] [CrossRef]
109. Vörösmarty, G.; Dobos, I. Supplier evaluation with environmental aspects and common DEA weights. Period. Polytech. Soc. Manag. Sci. 2019, 27 , 17–25. [Google Scholar] [CrossRef]
110. Wang, C.-N.; Ho, H.T.-X.; Luo, S.-H.; Lin, T.-F. An integrated approach to evaluating and selecting green logistics providers for sustainable development. Sustainability 2017, 9 , 218. [Google Scholar] [CrossRef] [Green Version]
111. Wang, C.-N.; Nguyen, V.T.; Thai, H.T.N.; Tran, N.N.; Tran, T.L.A. Sustainable supplier selection process in edible oil production by a hybrid fuzzy analytical hierarchy process and green data envelopment analysis for the SMEs food processing industry. Mathematics 2018, 6 , 302. [Google Scholar] [CrossRef] [Green Version]
112. Wang, C.-N.; Pham, T.-D.T.; Nhieu, N.-L. Multi-layer fuzzy sustainable decision approach for outsourcing manufacturer selection in apparel and textile supply chain. Axioms 2021, 10 , 262. [Google Scholar] [CrossRef]
113. Wu, M.-Q.; Zhang, C.-H.; Liu, X.-N.; Fan, J.-P. Green supplier selection based on DEA model in interval-valued pythagorean fuzzy environment. IEEE Access 2019, 7 , 108001–108013. [Google Scholar] [CrossRef]
114. Yan, X.; Bao, X.; Zhao, R.; Li, F. Performance measurement for green supplier selection based on data envelopment analysis. Environ. Sci. Pollut. Res. 2022. [Google Scholar] [CrossRef]
115. Yousefi, S.; Mohamadpour Tosarkani, B. An analytical approach for evaluating the impact of blockchain technology on sustainable supply chain performance. Int. J. Prod. Econ. 2022, 246 , 108429. [Google Scholar] [CrossRef]
116. Yu, M.-C.; Su, M.-H. Using fuzzy DEA for green suppliers selection considering carbon footprints. Sustainability 2017, 9 , 495. [Google Scholar] [CrossRef] [Green Version]
117. Zhang, Z.; Liao, H. A stochastic cross-efficiency DEA approach based on the prospect theory and its application in winner determination in public procurement tenders. Ann. Oper. Res. 2022, 1–29. [Google Scholar] [CrossRef]
118. Zhao, S.; Wang, J.; Ye, M.; Huang, Q.; Si, X. An evaluation of supply chain performance of China’s prefabricated building from the perspective of sustainability. Sustainability 2022, 14 , 1299. [Google Scholar] [CrossRef]
119. Zhou, X.; Li, L.; Wen, H.; Tian, X.; Wang, S.; Lev, B. Supplier’s goal setting considering sustainability: An uncertain dynamic data envelopment analysis based benchmarking model. Inf. Sci. 2021, 545 , 44–64. [Google Scholar] [CrossRef]
120. Zhou, X.; Pedrycz, W.; Kuang, Y.; Zhang, Z. Type-2 fuzzy multi-objective DEA model: An application to sustainable supplier evaluation. Appl. Soft Comput. J. 2016, 46 , 424–440. [Google Scholar] [CrossRef]
121. Zoroofchi, K.H.; Saen, R.F.; Lari, P.B.; Azadi, M. A combination of range-adjusted measure, cross-efficiency and assurance region for assessing sustainability of suppliers in the presence of undesirable factors. Int. J. Ind. Syst. Eng. 2018, 29 , 163–176. [Google Scholar] [CrossRef]

Figure 1. The selection process of the papers for the bibliometric analysis. Source: Author’s work, 2022.

Figure 1. The selection process of the papers for the bibliometric analysis. Source: Author’s work, 2022.