Evaluating Reliability Metrics of Artificial Intelligence Systems in Healthcare: An SEM_FCM Approach

Document Type : Original Article

Authors

1 Ph.D. student, Department of Industrial Engineering, Faculty of Industrial Engineering, K. N. Toosi University of Technology, Tehran, Iran.

2 Assistant Professor, Department of Industrial Engineering, Faculty of Industrial Engineering, K. N. Toosi University of Technology, Tehran, Iran.

10.48308/jimp.16.1.77

Abstract

Introduction: As the field of computer science evolves, and with the emergence of concepts such as artificial intelligence (AI), machine learning (ML), and deep learning (DL), significant opportunities for achieving smart urban systems have been created. These transformative technologies are reshaping numerous industries, particularly healthcare, where their impact has been profound. AI-powered tools are now employed to manage patient medical histories, conduct digital consultations, and optimize drug administration. However, despite their vast potential, these tools are not without limitations. A significant challenge faced by these systems is the low accuracy of decision-making outputs, which hinders their effective implementation in critical areas. To address these issues, the present study evaluates reliability metrics specific to AI systems in healthcare. By focusing on these metrics, the research identifies key factors that improve trustworthiness, using the Fuzzy Cognitive Mapping (FCM) approach.
Methods: The study begins with the extraction of reliability metrics through a detailed literature review and interviews with healthcare professionals, ensuring that the metrics are both comprehensive and grounded in real-world applications. Subsequently, using the Delphi method, the critical criteria for evaluating the reliability of artificial intelligence systems in the targeted domain were identified. In the next step, a causal model was developed based on a review of the relevant literature. This model was then validated using the Structural Equation Modeling (SEM) approach. Following that, causal relationships were derived using the validated SEM model and expert opinions, and the interactions among the identified criteria were analyzed through the application of the Fuzzy Cognitive Mapping (FCM) method. This advanced method provided a clear understanding of which factors were most influential and which were most impacted, offering deeper insights into AI system reliability. For data collection, a range of questionnaires, including Likert scale, AHP, and FCM-based tools, were distributed to participants. The data collected was then analyzed using SmartPLS software, a powerful tool for path analysis and structural equation modeling.
Findings: The findings reveal that "continuous monitoring of generated outcomes and system reconfiguration" is the most effective metric for evaluating AI system reliability in healthcare. This underscores the importance of ongoing oversight and adaptability to maintain system accuracy and relevance. Another crucial finding identifies the "use of non-deterministic algorithms" as the most impacted metric, highlighting the need for flexible and probabilistic methods in AI systems. In total, six primary metrics were identified and evaluated:

Trusted and homogeneous data to ensure consistent results.
Data security and privacy to protect sensitive medical information.
Weekly updates to improve system performance.
Use of non-deterministic algorithms to enhance adaptability.
Stakeholder evaluation structures for transparency and accountability.
Continuous monitoring of results to identify and address emerging issues.

These metrics collectively form a comprehensive framework for enhancing AI system reliability in healthcare.
Conclusion: This study provides a detailed examination of AI system reliability in healthcare, emphasizing the critical role of continuous monitoring and regular updates in improving accuracy and trustworthiness. Moreover, ensuring data security and privacy is highlighted as essential for building confidence in these systems. The findings serve as a practical guide for AI developers in healthcare, helping them design reliable and efficient tools. Additionally, the study underscores the broader benefits of these improvements, such as enhanced medical service quality and increased patient trust in AI systems. Ultimately, adopting innovative approaches and focusing on the identified key components will drive significant advancements and transformations in healthcare delivery.

Keywords

References

  1. Ã, J. K. W. W., & Li, H. (2008). Application of the analytic hierarchy process ( AHP ) in multi-criteria analysis of the selection of intelligent building systems. 43, 108–125. https://doi.org/10.1016/j.buildenv.2006.11.019
  2. Alanazi, H. O., Abdullah, A. H., & Qureshi, K. N. (2017). A critical review for developing accurate and dynamic predictive models using machine learning methods in medicine and health care. Journal of Medical Systems, 41(4), 1–10.
  3. Alelyani T. A validated framework for responsible AI in healthcare autonomous systems. Sci Rep 2025;15:1–12. https://doi.org/https://doi.org/10.1038/s41598-025-25266-z.
  4. Ansari S, Baur B, Singh K, Admon AJ. Challenges in the Postmarket Surveillance of Clinical Prediction Models. Natl Libr Med 2025;2:1–4. https://doi.org/DOI: 10.1056/AIp2401116 PERSPECTIVE.
  5. Asan, O., Bayrak, A. E., & Choudhury, A. (2020). Artificial Intelligence and Human Trust in Healthcare: Focus on Clinicians. In Journal of Medical Internet Research (Vol. 22, Issue 6). https://doi.org/10.2196/15154
  6. Bajwa, J., Usman Munir, Nori, A., & Williams, B. (2021). Artificial intelligence in healthcare: transforming the practice of medicine. Future Healthcare Journal, 8(9), 188–194.
  7. Chang, Z., Zhiqing Zhan, Zhao, Z., You, Z., Liu, Y., Yan, Z., Fu, Y., Liang, W., & Zhao, L. (2021). Application of artificial intelligence in COVID-19 medical area: a systematic review. Journal of Thoracic Disease, 13(12), 7034–7053. https://doi.org/10.21037/jtd-21-747
  8. Chumney, F. L. (2013). Structural Equation Models With Small Samples : A Comparative Study of Four Approaches. 146.
  9. Gerogiannis, D., Arsenos, A., Kollias, D., Nikitopoulos, D., & Kollias, S. (2024). Covid-19 Computer-Aided Diagnosis through AI-Assisted CT Imaging Analysis: Deploying a Medical AI System. Proceedings - International Symposium on Biomedical Imaging. https://doi.org/10.1109/ISBI56570.2024.10635484
  10. Hamamoto, R. (2021). Application of Artificial Intelligence for Medical Research. 10–13.
  11. Hlávka, J. P. (2020). Security, privacy, and information-sharing aspects of healthcare artificial intelligence. In Artificial Intelligence in Healthcare (pp. 235–270). Elsevier.
  12. Jhantasana, C. (2023). Should A Rule of Thumb be used to Calculate PLS-SEM Sample Size. Asia Social Issues, 16(5), 1–23. https://doi.org/10.48048/asi.2023.254658
  13. Johnson, B. (2022). Metacognition for artificial intelligence system safety–An approach to safe and desired behavior. Safety Science, 151, 105743.
  14. Lekadir K, Frangi AF, Porras AR, Glocker B, Cintas C, Langlotz CP, et al. FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare Karim. Res METHODS Report Futur 2025;388:1–22.
  15. Miyashita, M., & Brady, M. (2019). The Health Care Benefits of Combining Wearables and AI. Bus. Rev.
  16. Moose A, Horner B. Earning Trust for AI in Health: A Collaborative Path Forward 2025:1–20.
  17. Okwor, I. A., Hitch, G., Hakkim, S., Akbar, S., Sookhoo, D., & Kainesie, J. (2024). Digital Technologies Impact on Healthcare Delivery : A Systematic Review of AI and ML Adoption , Challenges , and Opportunities Digital Technologies Impact on Healthcare Delivery : A Systematic Review of AI and ML Adoption , Challenges , and Opportunities. 5(4), 1918–1941.
  18. Patrick, J. (2019). How to Check the Reliability of Artificial Intelligence Solutions—Ensuring Client Expectations are Met. Applied Clinical Informatics, 10(02), 269–271.
  19. Pinsky, M. R., Bedoya, A., Bihorac, A., Celi, L., Churpek, M., Economou-Zavlanos, N. J., Elbers, P., Saria, S., Liu, V., Lyons, P. G., Shickel, B., Toral, P., Tscholl, D., & Clermont, G. (2024). Use of artificial intelligence in critical care: opportunities and obstacles. Critical Care, 28(1). https://doi.org/10.1186/s13054-024-04860-z
  20. Rahimi Kolour, H., & Nikkhah, Y. (2024). Explaining the Effect of Customer Knowledge Management on Innovation Quality through Strategic Agility with the Moderating Role of Competition Intensity: A Study in Medical Equipment Manufacturing Firms. Industrial Management Perspective, 14(2), 66–84. https://doi.org/10.48308/JIMP.14.2.66 (in persian)
  21. Rokneddini, S. A., & Ardakani, D. A. (2024). Analysis of Organizational Factors Affecting the Adoption of Industry 4.0 Technologies in Small and Medium-Sized Companies. Industrial Management Perspective, 14(2), 84–112. https://doi.org/10.48308/JIMP.14.2.85 (in persian)
  22. Shafaghsorkh, O., Ayough, A., & Alem-Tabriz, A. (2024). Identifying the Problems Facing the Development of Patient Telemonitoring Systems and Providing Solutions from Users’ Perspectives: Application of Soft Systems Methodology. Industrial Management Perspective, 14(2), 9–33. https://doi.org/10.48308/JIMP.14.2.9 Industrial (in persian)
  23. Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023). Human–AI collaboration enables more empathic conversations in text-based peer-to-peer mental health support. Nature Machine Intelligence, 5(1), 46–57. https://doi.org/10.1038/s42256-022-00593-2
  24. Shusterman, R., Tucker, D. M., Waters, A. C., Neill, S. O., Bangs, M., & Luu, P. (2025). An active inference strategy for prompting reliable responses from large language models in medical practice. Npj Digital Medicine. https://doi.org/10.1038/s41746-025-01516-2
  25. Tagde, P., Tagde, S., Bhattacharya, T., Tagde, P., Chopra, H., Akter, R., Kaushik, D., & Rahman, M. (2021). Blockchain and artificial intelligence technology in e-Health. Environmental Science and Pollution Research, 28(38), 52810–52831.
  26. Tan Ming En, J. (2022). Non-Deterministic Artificial Intelligence Systems and the Future of the Law on Unilateral Mistakes in Singapore. SAcLJ, 34, 91.
  27. Topol, E. J. (2019). High-performance medicine: the convergence of human and artificial intelligence. Nature Medicine, 25(1), 44–56.
  28. Tsopra, R., Fernandez, X., Luchinat, C., Alberghina, L., Lehrach, H., Vanoni, M., Dreher, F., Sezerman, O. U., Cuggia, M., & de Tayrac, M. (2021). A framework for validating AI in precision medicine: considerations from the European ITFoC consortium. BMC Medical Informatics and Decision Making, 21(1), 1–14.
  29. Ueda, D., Kakinuma, T., Fujita, S., Kamagata, K., Fushimi, Y., Ito, R., Matsui, Y., Nozaki, T., Nakaura, T., Fujima, N., Tatsugami, F., Yanagawa, M., Hirata, K., Yamada, A., Tsuboyama, T., Kawamura, M., Fujioka, T., & Naganawa, S. (2024). Fairness of artificial intelligence in healthcare: review and recommendations. In Japanese Journal of Radiology (Vol. 42, Issue 1, pp. 3–15). https://doi.org/10.1007/s11604-023-01474-3
  30. Vinuesa, R., Azizpour, H., Leite, I., Balaam, M., Dignum, V., Domisch, S., Felländer, A., Langhans, S. D., Tegmark, M., & Fuso Nerini, F. (2020). The role of artificial intelligence in achieving the Sustainable Development Goals. Nature Communications, 11(1), 1–10.
  31. Zezza, M. (2025). The promise of AI in healthcare: transforming communication and decision-making for patients. Journal of Communication in Healthcare, 18(1), 6–9. https://doi.org/10.1080/17538068.2025.2452100

Files

Share

How to cite

Statistics