Autors: Budakova, D. V., Vasilev, V. E., Dakovski L.
Title: Generalized Net Model for Analysis of Behavior and Efficiency of Intelligent Virtual Agents in Risky Environment †
Keywords: behavior, efficiency, generalized network models, imitation learning, intelligent virtual agent, reinforcement learning, risky environment

Abstract: In this article, two generalized net models (GNMs) are proposed to study the behavior and effectiveness of intelligent virtual agents (IVA) working in a risky environment under different scenarios and training algorithms. The proposed GNMs allow for the selection of machine learning algorithms such as intensity of characteristics Q-learning (InCh-Q), as well as the modification of multi-plan reinforcement learning (RL), proximal policy optimization (PPO), soft actor–critic (SAC), the generative adversarial imitation learning (GAIL) algorithm, and behavioral cloning (CB). The choice of action, the change in priorities, and the achievement of goals by the IVA are studied under different scenarios, such as fire extinguishing, rescue operations, evacuation, patrolling, and training. Transitions in the GNMs represent the scenarios and learning algorithms. The tokens that pass through the GNMs can be the GNMs of the IVA architecture or the IVA memory model, which are enriched with knowledge and experience during the experiments, as the scenarios develop. The proposed GNMs are formally correct and, at the same time, understandable, practically applicable, and convenient for interpretation. Achieving GNMs that meet these requirements is a complex problem. Therefore, issues related to the design and use of GNMs for the reliable modeling and analysis of the behavior and effectiveness of IVAs operating in a dynamic and risky environment are discussed. Some advantages and challenges in using GNMs compared to other classical models used to study IVA behavior are considered.

References

  1. Atanassov K. Generalized Nets World Scientific Publishing Company Singapore 1991 Available online: https://www.perlego.com/book/846423/generalized-nets-pdf (accessed on 3 July 2025)
  2. Zoteva D. Angelova N. Generalized Nets. An Overview of the Main Results and Applications Research in Computer Science in the Bulgarian Academy of Sciences Atanassov K.T. Studies in Computational Intelligence Springer Cham, Switzerland 2021 Volume 934 10.1007/978-3-030-72284-5_10
  3. Angelova N. Programmatic Implementation of Generalized Networks and Modeling Applications Ph.D. Thesis Bulgarian Academy of Sciences Institute of Biophysics and Biomedical Engineering, Section “Bioinformatics and Mathematical Modeling” Sofia, Bulgaria 2017 Available online: https://biomed.bas.bg/bg/wp-content/uploads/2017/02/NAngelova-avtoreferat.pdf (accessed on 3 July 2025)
  4. Dimitrov D.G. GN IDE—A Software Tool for Simulation with Generalized Nets Proceedings of the Tenth International Workshop on Generalized Nets Sofia, Bulgaria 5 December 2009 70 75
  5. Mengov G. Georgiev K. Pulov S. Trifonov T. Atanassov K. Fast computation of a gated dipole field Neural Netw. 2006 19 1636 1647 10.1016/j.neunet.2006.05.031 16899351
  6. Petkov T. Sotirov S. Generalized Net Model of the Cognitive and Neural Algorithm for Adaptive Resonance Theory 1 Int. J. Bioautomotion 2013 17 207 216
  7. Budakova D. Vasilev V. Dakovski L. Stefanov S. Virtual Agent Behavior Modeling in Case of a Risky Situation in a Virtual Electrical Substation Proceedings of the 15th International Conference on Agents and Artificial Intelligence (ICAART 2023) Lisbon, Portugal 22–24 February 2023
  8. ML-Agents Overview Available online: https://unity-technologies.github.io/ml-agents/ML-Agents-Overview/ (accessed on 3 July 2025)
  9. Immersive Limit Unity ML-Agents—Demonstration Recorder for Imitation Learning Available online: https://www.youtube.com/watch?v=Dhr4tHY3joE (accessed on 3 July 2025)
  10. Code Monkey, Teach Your AI! Imitation Learning with Unity ML-Agents! Available online: https://www.youtube.com/watch?v=supqT7kqpEI (accessed on 3 July 2025)
  11. Budakova D. Vasilev V. Dakovski L. Modeling Virtual Agent Behavior Using an Adaptive Multi-Plan Evacuation Strategy AIP Conf. Proc. 2025 3274 040004
  12. Vasilev V. Stefanov S. Modeling an Outdoor Substation with Dynamically Occurring Faults and Conducting a Preventive Inspection AIP Conf. Proc. 2024 3078 040003 10.1063/5.0209284
  13. Harbaliev G. Vasilev V. Budakova D. An Approach to Modeling and Studying the Behavior of Firefighting Drones Using Unity ML-Agents Proceedings of the 2024 12th International Scientific Conference on Computer Science (COMSCI) Sozopol, Bulgaria 13–15 September 2024
  14. Budakova D. Vasilev V. Dakovski L. A Reinforcement Learning Algorithm for the Optimal Evacuation Route Finding from an Electrical Substation AIP Conf. Proc. 2024 3078 040005 10.1063/5.0209018
  15. Silva M. Modeling, Analysis, and Control with Petri Nets Encyclopedia of Systems and Control Baillieul J. Samad T. Springer International Publishing Cham, Switzerland 2021 1301 1311 10.1007/978-3-030-44184-5_53
  16. Lafortune S. Discrete Event Systems: Modeling, Observation, and Control Annu. Rev. Control Robot. Auton. Syst. 2019 2 141 159 10.1146/annurev-control-053018-023659
  17. Cassandras C.G. Lafortune S. Introduction to Discrete Event Systems Springer International Publishing Cham, Switzerland 2021 10.1007/978-3-030-72274-6
  18. Yaneva V. Rajan A. Dubach C. GPU acceleration of finite state machine input execution: Improving scale and performance Softw. Test. Verif. Reliab. 2022 32 e1796 10.1002/stvr.1796
  19. Lteif G. Finite State Machines: An Introduction to FSMs and Their Role in Computer Science Available online: https://softwaredominos.com/home/software-engineering-and-computer-science/finite-state-machines-an-introduction-to-fsms-and-their-role-in-computer-science/ (accessed on 6 May 2025)
  20. Dorri A. Kanhere S.S. Jurdak R. Multi-Agent Systems: A Survey IEEE Access 2018 6 28573 28593 10.1109/ACCESS.2018.2831228
  21. Li H. Li Y. Zhao H.V. Modeling Decision Process in Multi-Agent Systems: A Graphical Markov Game based Approach Proceedings of the 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) Auckland, New Zealand 7–10 December 2020 197 204 Available online: https://ieeexplore.ieee.org/document/9306222 (accessed on 6 May 2025)
  22. Mukhopadhyay S. Jain B. Multi-agent Markov decision processes with limited agent communication Proceedings of the 2001 IEEE International Symposium on Intelligent Control (ISIC ’01) (Cat. No.01CH37206) Mexico City, Mexico 5–7 September 2001 7 12 10.1109/ISIC.2001.971476
  23. Jiet M.M. Verma P. Kamble A. Puri C. A Review on Bayesian Methods for Uncertainty Quantification in Machine Learning Models Enhancing Predictive Accuracy and Model Interpretability Proceedings of the 2024 Second International Conference on Intelligent Cyber Physical Systems and Internet of Things (ICoICI) Coimbatore, India 28–30 August 2024 1671 1675 10.1109/ICoICI62503.2024.10696308
  24. Bayesian Neural Networks Minimize Uncertainty in Your AI Models Available online: https://shelf.io/blog/bayesian-neural-networks-minimize-uncertainty-in-your-ai-models/ (accessed on 6 May 2025)
  25. Andonov V. Poryazov S. Saranova E. On the Conceptual Optimization of Generalized Net Models Recent Advances in Computational Optimization Fidanova S. Studies in Computational Intelligence Springer International Publishing Cham, Switzerland 2022 Volume 986 349 369 10.1007/978-3-030-82397-9_18

Issue

Engineering Proceedings, vol. 100, 2025, Albania, https://doi.org/10.3390/engproc2025100056

Вид: статия в списание, публикация в реферирано издание, индексирана в Scopus