Autors: Slavova, A. V., Hristov, V. D.
Title: Mapless Navigation with Deep Reinforcement Learning in Indoor Environment †
Keywords: Deep Learning, Deep Reinforcement Learning, navigating, Policy Optimization, wheeled robot

Abstract: One of the crucial tasks for autonomous robots is learning to safely navigate through obstacles in real-world environments. An intelligent robot must not only perform the assigned task but also adapt to changes in its environment as quickly as possible. In this work, we propose an improved version of the Deep Reinforcement Learning (DRL) Proximal Policy Optimization (PPO) algorithm by modifying a deep neural network of the Actor and Critic. Then we compare the results of our work by comparing them with those of classical PPO. Algorithm testing is conducted in a Flatland simulation environment, which allows for integration with the ROS2 operating environment.

References

  1. Sutton R.S. Barto A.G. Reinforcement Learning: An Introduction MIT Press Cambridge, MA, USA 1998 10.0262039249
  2. Hemming N. Menon V. Deep Reinforcement Learning Based Efficient and Robust Navigation Method for Autonomous Applications Proceedings of the 2023 IEEE 35th International Conference on Tools with Artificial Intelligence (ICTAI) Atlanta, GA, USA 6–8 November 2023 287 293
  3. ROS—Robot Operating System Available online: https://www.ros.org/ (accessed on 1 March 2025)
  4. Flatland Simulation Available online: https://flatland-simulator.readthedocs.io (accessed on 1 March 2025)
  5. Gazebo Simulation Available online: https://gazebosim.org/ (accessed on 1 March 2025)
  6. Riaz Z. Pervez A. Ahmer M. Iqbal J. A fully autonomous indoor mobile robot using SLAM Proceedings of the 2010 International Conference on Information and Emerging Technologies Karachi, Pakistan 14–16 June 2010 10.1109/ICIET.2010.5625691
  7. Alyasin A. Abbas E.I. Hasan S.D. An Efficient Optimal Path Finding for Mobile Robot Based on Dijkstra Method Proceedings of the 2019 4th Scientific International Conference Najaf (SICN) Al-Najaf, Iraq 29–30 April 2019 10.1109/SICN47020.2019.9019345
  8. Mnih V. Kavukcuoglu K. Silver D. Rusu A.A. Veness J. Bellemare M.G. Petersen S. Human-level control through deep reinforcement learning Nature 2015 518 529 533 10.1038/nature14236 25719670
  9. Silver D. Empowering mobile robots with DDPG for autonomous navigation Robot. Auton. Syst. 2018 123 456 467
  10. Silver D. Lever G. Heess N. Degris T. Wierstra D. Riedmiller M. Deterministic Policy Gradient Algorithms Proceedings of the International Conference on Machine Learning Beijing, China 21–26 June 2014
  11. Zhu H. End-to-end navigation for robots using convolutional neural networks and deep reinforcement learning Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) Singapore 29 May–3 June 2017 1234 1245
  12. Zhu Y. Mottaghi R. Kolve E. Lim J.J. Gupta A. Target-driven visual navigation in indoor scenes using deep reinforcement learning Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) Singapore 29 May–3 June 2017 3357 3364
  13. Kahn G. Villaflor A. Ding B. Abbeel P. Levine S. Self-Supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) Brisbane, QLD, Australia 21–25 May 2018 5129 5136
  14. Shulman J. Wolski F. Dhariwal P. Radford A. Klimov O. Proximal policy optimization algorithms arXiv 2017 10.48550/arXiv.1707.06347
  15. Konda V.R. Tsitsiklis J.N. Actor-Critic Algorithms Adv. Neural Inf. Process. Syst. 1999 12 1008 1014
  16. Kalidas A.P. Joshua C.J. Quadir Md A. Basheer S. Mohan S. Sakri S. Deep Reinforcement Learning for Vision-Based Navigation of UAVs in Avoiding Stationary and Mobile Obstacles Drones 2023 7 245 10.3390/drones7040245
  17. He K. Zhang X. Ren S. Sun J. Deep Residual Learning for Image Recognition Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Las Vegas, NV, USA 27–30 June 2016 770 778 10.1109/CVPR.2016.90
  18. Popov V. Shakev N. Ahmed S. Toplaov A. Recognition of Dynamic Targets using a Deep Convolutional Neural Network Proceedings of the ANNA ‘18—Advances in Neural Networks and Applications 2018 St. Konstantin and Elena Resort, Bulgaria 15–17 September 2018 1 6
  19. PyTorch Available online: https://pytorch.org/ (accessed on 1 March 2025)

Issue

Engineering Proceedings, vol. 100, 2025, Switzerland, https://doi.org/10.3390/engproc2025100063

Цитирания (Citation/s):
1. Sazonov A., Kuchkin O., Cherepanska I., Lipnickas A., S3PM: Entropy-Regularized Path Planning for Autonomous Mobile Robots in Dense 3D Point Clouds of Unstructured Environments, 2026, Sensors, issue 2, vol. 26, DOI 10.3390/s26020731, eissn 14248220 - 2026 - в издания, индексирани в Scopus

Вид: статия в списание, публикация в издание с импакт фактор, публикация в реферирано издание, индексирана в Scopus