Journal of Industrial Management Perspective

Journal of Industrial Management Perspective

Developing a Data-Driven Hybrid Approach Based on Data Envelopment Analysis for Evaluating the Performance of Manufacturing Units Considering Sustainability, Agility, and Industry 4.0 Dimensions

Document Type : Original Article

Authors
Ali Nokhbeh Zaeim 1
Vahid Baradaran 2
Amin Mostafaee 3
1 Ph.D. Student, Department of Industrial Engineering, Islamic Azad University, Tehran North Branch, Tehran, Iran.
2 Associate Professor, Department of Industrial Engineering, Islamic Azad University, Tehran North Branch, Tehran, Iran.
3 Associate Professor, Department of Mathematics, Islamic Azad University, Tehran North Branch, Tehran, Iran.
10.48308/jimp.16.2.37
Abstract
Introduction: In today’s competitive industrial environment, performance evaluation of manufacturing units has become a critical factor for improving productivity, quality, and competitiveness. Organizations must simultaneously address three major challenges — agility in responding to rapid market changes, sustainability across economic, social, and environmental dimensions, and digital transformation driven by Industry 4.0 technologies. However, most of the existing performance evaluation frameworks focus on a single dimension and lack the ability to integrate these three aspects concurrently. Accordingly, this study aims to develop a data-driven hybrid model based on Data Envelopment Analysis (DEA) and the Random Forest (RF) algorithm to evaluate the performance of manufacturing units considering agility, sustainability, and digitalization dimensions. The proposed framework enables independent evaluation of each manufacturing unit without the need for direct comparison with others.
Methods: In the first stage, a comprehensive set of performance evaluation indicators was identified through literature review and expert consultation, categorized into four groups: agility, sustainability, digitalization, and general indicators. To determine the relative importance of these indicators, the Fuzzy Best–Worst Method (FBWM) was employed, allowing consistent pairwise comparisons among the selected indicators. After calculating the weights, eight key indicators with the highest importance were selected as the input and output variables for the DEA model to compute efficiency scores. Data were collected from 40 listed manufacturing firms over a five-year period, resulting in 400 data records. The efficiency scores obtained from the Andersen–Peterson (AP-DEA) model were then used as labeled data for training the Random Forest model. Seventy percent of the dataset was used for training and the remaining 30 percent for testing. The model’s predictive accuracy was evaluated using four standard metrics: Accuracy, Precision, Recall, and F1 Score.
Results and Discussion: The FBWM results revealed that agility is the most influential dimension in performance evaluation, followed by sustainability and digitalization. Within the agility dimension, speed, quality, flexibility, and production capacity emerged as the most critical indicators. In the sustainability dimension, financial infrastructure, environmental certification, and waste management showed a significant impact, while in the digitalization dimension, smart operations and data-driven planning were identified as key contributors. The DEA model successfully classified manufacturing units into four performance levels — inefficient, weakly efficient, efficient, and highly efficient — enabling a structured labeling process for subsequent machine learning analysis. The developed Random Forest model achieved an accuracy of 0.925 and an F1 Score of 0.911, outperforming comparative models such as Support Vector Machine (SVM) and Artificial Neural Network (ANN). The confusion matrix results confirmed that most prediction errors occurred between adjacent classes, demonstrating the model’s robustness and reliability. Furthermore, the feature importance analysis indicated that agility-related variables, particularly speed and flexibility, had the highest influence on performance prediction. These findings highlight the superiority of data-driven hybrid models in extracting meaningful patterns and enhancing interpretability in complex decision environments.
Conclusions: The study demonstrated that adopting a data-driven hybrid approach can overcome the limitations of traditional performance evaluation models and substantially improve accuracy, interpretability, and managerial applicability. The developed model provides a comprehensive framework that allows independent assessment of each manufacturing unit while simultaneously incorporating agility, sustainability, and Industry 4.0 dimensions. This approach offers a valuable decision-support tool for managers to enhance operational efficiency, plan strategically, and evaluate the impact of industrial policies. The model’s high predictive accuracy confirms its practical applicability in real manufacturing environments. Future studies are recommended to integrate advanced machine learning algorithms and dynamic datasets to capture temporal changes in performance and further strengthen model generalization and explanatory power.
Keywords
Agility
Data Envelopment Analysis (DEA)
Data-driven model
Industry 4.0
Machine learning
Random Forest
Sustainability
Subjects
1.     Abdel-Basset, M., Ding, W., Mohamed, R., & Metawa, N. (2020). An integrated plithogenic MCDM approach for financial performance evaluation of manufacturing industries. Risk management, 22, 192-218.  
2.     Ali, S. S., Kaur, R., Persis, D. J., Saha, R., Pattusamy, M., & Sreedharan, V. R. (2023). Developing a hybrid evaluation approach for the low carbon performance on sustainable manufacturing environment. Annals of Operations Research, 1-33.  
3.     Alonso-Gómez, L. A., Solarte-Toro, J. C., Bello-Pérez, L. A., & Cardona-Alzate, C. A. (2020). Performance evaluation and economic analysis of the bioethanol and flour production using rejected unripe plantain fruits (Musa paradisiaca L.) as raw material. Food and bioproducts processing, 121, 29-42.  
4.     Alsaadi, B., & Saeedi, K. (2022). Data-driven effort estimation techniques of agile user stories: a systematic literature review. Artificial Intelligence Review, 55(7), 5485-5516.
5.     Asadabadi, M. J. R., Rad, S. H., Taraghi, M., Moghimi, M., & Ahmadi, R. (2025). ANN-boosted optimization and performance evaluation of a hybrid solar-geothermal high temperature cascaded ORC for electricity, liquid hydrogen, freshwater, and cooling purposes. Applied Thermal Engineering, 262, 125251.
6.            Ayyoubzadeh, B., Ebrahimnejad, S., Bashiri, M., Baradaran, V. (2021). Modelling and an improved NSGA-II algorithm for sustainable manufacturing systems with energy conservation under environmental uncertainties: a case study, International Journal of Sustainable Engineering, 14(3), 255-279. 10.1080/19397038.2021.1923083.
7.     Badi, I., & Abdulshahed, A. (2021). Sustainability performance measurement for libyan iron and steel company using rough AHP. Journal of Decision Analytics and Intelligent Computing, 1(1), 22-34.
8.     Baradaran V, & Yaghoubi N. (2016). Evaluate the Efficiency of Iranian Electric Power Distribution Companies by Stochastic Nonparametric Envelopment of Data (StoNED) Approach. Ieijqp, 4 (2) :15-26. http://ieijqp.ir/article-1-217-fa.html.
9.     Baradaran, V., Farokhi, S., Ahamdi, Z. (2022). A model for evaluation and development of citizens' electronic readiness for deployment of an E-city using structural equation modeling, Journal of Global Information Management (JGIM) 26 (4), 135-157.
10.  Bianchi, M., Marzi, G., & Guerini, M. (2020). Agile, Stage-Gate and their combination: Exploring how they relate to performance in software development. Journal of Business Research, 110, 538-553.
11.  Boateng, E. Y., Otoo, J., & Abaye, D. A. (2020). Basic tenets of classification algorithms K-nearest-neighbor, support vector machine, random forest and neural network: a review. Journal of Data Analysis and Information Processing, 8(4), 341-357.  
12.  Du, J., Wang, W., Gao, X., Hu, M., & Jiang, H. (2023). Sustainable operations: A systematic operational performance evaluation framework for Public–Private Partnership transportation infrastructure projects. Sustainability, 15(10), 7951.  
13.  Dursun, M., & Ogunclu, O. (2021). Agile supplier evaluation using hierarchical TOPSIS method. WSEAS Transactions on Information Science and Applications, 18, 12-19.  
14.  Dutta, P., Jaikumar, B., & Arora, M. S. (2022). Applications of data envelopment analysis in supplier selection between 2000 and 2020: A literature review. Annals of Operations Research, 315(2), 1399-1454.  
15.  Eskandari, M., Hamid, M., Masoudian, M., & Rabbani, M. (2022). An integrated lean production-sustainability framework for evaluation and improvement of the performance of pharmaceutical factory. Journal of Cleaner Production, 376, 134132.
16.  Fan, C., Ghaemi, S., Khazaei, H., & Musilek, P. (2020). Performance evaluation of blockchain systems: A systematic survey. IEEE Access, 8, 126927-126950.
17.  ForouzeshNejad, A. A. (2023). Leagile and sustainable supplier selection problem in the Industry 4.0 era: A case study of the medical devices using hybrid multi-criteria decision making tool. Environmental Science and Pollution Research, 30(5), 13418-13437.  
18.  Ghahramani, M., Qiao, Y., Zhou, M. C., O'Hagan, A., & Sweeney, J. (2020). AI-based modeling and data-driven evaluation for smart manufacturing processes. IEEE/CAA Journal of Automatica Sinica, 7(4), 1026-1037.  
19.  Govindan, K., Kannan, D., Jørgensen, T. B., & Nielsen, T. S. (2022). Supply Chain 4.0 performance measurement: A systematic literature review, framework development, and empirical evidence. Transportation Research Part E: Logistics and Transportation Review, 164, 102725.
20.  Hsu, S. Y., Yang, C. Y., Chen, Y. L., & Lu, C. C. (2023). Agricultural efficiency in different regions of china: An empirical analysis based on dynamic SBM-DEA model. Sustainability, 15(9), 7340.  
21.  Javanmardan, A., Golpîra, H., Baradaran, V. (2026). Robust optimization of sustainable circular fruit supply chain under quality and carbon emission considerations, Food and Bioproducts Processing, 158, 82-102. https://doi.org/10.1016/j.fbp.2026.04.011.
22.  Karimi Sani M A, Baradaran V, & Mostafaei A. (2021). Developing the Modified Russell Model for Decision Making Units under General Two-Stage Structure with the Effect of Intermediate Data on the Efficiency. Journal of Operational Research and Its Applications (Applied Mathematics, 18 (3), 139-160. URL: http://jamlu.liau.ac.ir/article-1-1639-fa.html.  
23.  Li, L., Mao, C., Sun, H., Yuan, Y., & Lei, B. (2020). Digital twin driven green performance evaluation methodology of intelligent manufacturing: hybrid model based on fuzzy rough-sets AHP, multistage weight synthesis, and PROMETHEE II. Complexity, 2020, 1-24.
24.  Luo, L., Chen, H., Yang, Y., Wu, G., & Chen, L. (2022). A three-stage network DEA approach for performance evaluation of BIM application in construction projects. Technology in Society, 71, 102105.  
25.  Moghimi, F., Baradaran, V., Hosseinian, AH. (2023). Identifying the influential factors on the effectiveness of industrial parks and using an MCDM method to rank them: case study of Iran, Journal of Facilities Management 21 (5), 816-844.
26.  PulickalMannath, S., Kundu, P., Plotnikov, E., & Martemianov, D. (2021). Performance evaluation and intensification of novel buoyant filter bioreactor with associated secondary treatment process for rice processing industries. Chemical Engineering and Processing-Process Intensification, 169, 108619.  
27.  Renna, P. (2024). Performance Evaluation of Reconfiguration Policy in Reconfigurable Manufacturing Systems including Multi-Spindle Machines: An Assessment by Simulation. Applied Sciences, 14(7), 2778.
28.  RL, M., & Mishra, A. K. (2022). Agricultural production efficiency of Indian states: Evidence from data envelopment analysis. International Journal of Finance & Economics, 27(4), 4244-4255.
29.  Rostami, O., Tavakoli, M., Tajally, A., & GhanavatiNejad, M. (2023). A goal programming-based fuzzy best–worst method for the viable supplier selection problem: a case study. Soft Computing, 27(6), 2827-2852.
30.  Sazvar, Z., Tavakoli, M., Ghanavati-Nejad, M., & Nayeri, S. (2022). Sustainable-resilient supplier evaluation for high-consumption drugs during COVID-19 pandemic using a data-driven decision-making approach. Scientia iranica.
31.  Sepehr, A., Manteghi, M. and Baradaran, V. (2023). Innovation Strategy Based on Industry 4.0 and Digital Transformation in SME Companies in Iran. International Journal of Digital Content Management, 4(7), 41-58. doi: 10.22054/dcm.2022.69080.1132.
32.  Sharshir, S. W., Rozza, M. A., Elsharkawy, M., Youns, M. M., Abou-Taleb, F., & Kabeel, A. E. (2022). Performance evaluation of a modified pyramid solar still employing wick, reflectors, glass cooling and TiO2 nanomaterial. Desalination, 539, 115939.
33.  Sheykhmousa, M., Mahdianpari, M., Ghanbari, H., Mohammadimanesh, F., Ghamisi, P., & Homayouni, S. (2020). Support vector machine versus random forest for remote sensing image classification: A meta-analysis and systematic review. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, 6308-6325.
34.  Telemaco, U., Oliveira, T., Alencar, P., & Cowan, D. (2020). A catalogue of agile smells for agility assessment. IEEE Access, 8, 79239-79259.
35.  Tiguh, E. E., Delel, M. A., Ali, A. N., Gelaw, G. K. M., Fanta, S. W., & Bayable, M. (2025). Development and performance evaluation of greenhouse solar dryer for unthreshed Teff crops. Results in Engineering, 25, 104495.
36.  Tiwari, A., & Kumar, A. (2025). Performance evaluation of solar-assisted humidification dehumidification system for seawater desalination: An experimental approach. Separation and Purification Technology, 356, 129899.
37.  Varnasseri, N., Saati, S., Baradaran, V. (2026). Diagnosing Critical Performance Drivers in the Iranian Gas Industry under Uncertainty: A Hybrid Fuzzy DEA–EFQM Approach, International Journal of Mathematical Modelling & Computations, 10.57647/ijm2c.2027.1701.07.
38.  Wang, L., Chen, L., Gao, P., & Li, C. (2025). Construction and Application of Carbon Performance Evaluation Index System for Chinese Industrial Enterprises from the Perspective of Low‐Carbon Transition. Journal of International Development, 37(3), 736-757.
39.  Wankhede, V. A., & Vinodh, S. (2024). Assessment of Industry 4.0 performance using scoring approach: a case study. The TQM Journal, 36(2), 499-522.
40.  Wohlgemuth, M., Fries, C. E., Sant’Anna, Â. M. O., Giglio, R., & Fettermann, D. C. (2020). Assessment of the technical efficiency of Brazilian logistic operators using data envelopment analysis and one inflated beta regression. Annals of Operations Research, 286(1), 703-717.