Introduction: In today’s competitive industrial environment, performance evaluation of manufacturing units has become a critical factor for improving productivity, quality, and competitiveness. Organizations must simultaneously address three major challenges — agility in responding to rapid market changes, sustainability across economic, social, and environmental dimensions, and digital transformation driven by Industry 4.0 technologies. However, most of the existing performance evaluation frameworks focus on a single dimension and lack the ability to integrate these three aspects concurrently. Accordingly, this study aims to develop a data-driven hybrid model based on Data Envelopment Analysis (DEA) and the Random Forest (RF) algorithm to evaluate the performance of manufacturing units considering agility, sustainability, and digitalization dimensions. The proposed framework enables independent evaluation of each manufacturing unit without the need for direct comparison with others.
Methods: In the first stage, a comprehensive set of performance evaluation indicators was identified through literature review and expert consultation, categorized into four groups: agility, sustainability, digitalization, and general indicators. To determine the relative importance of these indicators, the Fuzzy Best–Worst Method (FBWM) was employed, allowing consistent pairwise comparisons among the selected indicators. After calculating the weights, eight key indicators with the highest importance were selected as the input and output variables for the DEA model to compute efficiency scores. Data were collected from 40 listed manufacturing firms over a five-year period, resulting in 400 data records. The efficiency scores obtained from the Andersen–Peterson (AP-DEA) model were then used as labeled data for training the Random Forest model. Seventy percent of the dataset was used for training and the remaining 30 percent for testing. The model’s predictive accuracy was evaluated using four standard metrics: Accuracy, Precision, Recall, and F1 Score.
Results and Discussion: The FBWM results revealed that agility is the most influential dimension in performance evaluation, followed by sustainability and digitalization. Within the agility dimension, speed, quality, flexibility, and production capacity emerged as the most critical indicators. In the sustainability dimension, financial infrastructure, environmental certification, and waste management showed a significant impact, while in the digitalization dimension, smart operations and data-driven planning were identified as key contributors. The DEA model successfully classified manufacturing units into four performance levels — inefficient, weakly efficient, efficient, and highly efficient — enabling a structured labeling process for subsequent machine learning analysis. The developed Random Forest model achieved an accuracy of 0.925 and an F1 Score of 0.911, outperforming comparative models such as Support Vector Machine (SVM) and Artificial Neural Network (ANN). The confusion matrix results confirmed that most prediction errors occurred between adjacent classes, demonstrating the model’s robustness and reliability. Furthermore, the feature importance analysis indicated that agility-related variables, particularly speed and flexibility, had the highest influence on performance prediction. These findings highlight the superiority of data-driven hybrid models in extracting meaningful patterns and enhancing interpretability in complex decision environments.
Conclusions: The study demonstrated that adopting a data-driven hybrid approach can overcome the limitations of traditional performance evaluation models and substantially improve accuracy, interpretability, and managerial applicability. The developed model provides a comprehensive framework that allows independent assessment of each manufacturing unit while simultaneously incorporating agility, sustainability, and Industry 4.0 dimensions. This approach offers a valuable decision-support tool for managers to enhance operational efficiency, plan strategically, and evaluate the impact of industrial policies. The model’s high predictive accuracy confirms its practical applicability in real manufacturing environments. Future studies are recommended to integrate advanced machine learning algorithms and dynamic datasets to capture temporal changes in performance and further strengthen model generalization and explanatory power.