Е-Публикации
Технически университет - София

Abstract: In the context of the global transformation towards sustainable mobility, the management of electric vehicle (EV) fleets poses new challenges related to limited battery capacity, dynamic demands, and limited charging resources. This study proposes an intelligent approach for management optimization using the Q-learning technique. A simulation environment is created in which the agent makes real-time decisions based on the current state of the EV, charging, station load, and request priority. The approach includes options for actions such as delivery, waiting, request skipping, and adaptive charging. The results of multiple iterations show that the agent successfully minimizes the number of missed requests, maintains high overall utility, and ensures balanced resource utilization. Analysis of logs, Q-values, and visualizations confirms the effectiveness of the reinforcement learning and stabilization strategy. The study demonstrates the potential of learning to address complex logistical problems and offers a feasible framework for implementation in real EV fleets.

References

1. Zulfiqar, M., Alshammari, N. F., & Rasheed, M. B. (2023). Reinforcement Learning-Enabled Electric Vehicle Load Forecasting for Grid Energy Management. Mathematics, 11(7), 1680. https://doi.org/10.3390/math11071680.
2. Arwa, Erick & Folly, Komla. (2020). Power Flow Management in Electric Vehicles Charging Station Using Reinforcement Learning. 10.1109/CEC48606.2020.9185652.
3. Q. Dang, D. Wu and B. Boulet, "A Q-Learning Based Charging Scheduling Scheme for Electric Vehicles," 2019 IEEE Transportation Electrification Conference and Expo (ITEC), Detroit, MI, USA, 2019, pp. 1-5, doi: 10.1109/ITEC.2019.8790603.
4. Dawei Qiu, Yi Wang, Weiqi Hua, Goran Strbac, "Reinforcement learning for electric vehicle applications in power systems:A critical review," Renewable and Sustainable Energy Reviews, Volume 173, 2023, 113052, ISSN 1364-0321, https://doi.org/10.1016/j.rser.2022.113052.
5. M. Dabbaghjamanesh, A. Moeini and A. Kavousi-Fard, "Reinforcement Learning-Based Load Forecasting of Electric Vehicle Charging Station Using Q-Learning Technique," in IEEE Transactions on Industrial Informatics, vol. 17, no. 6, pp. 4229-4237, June 2021, doi: 10.1109/TII.2020.2990397.
6. T. Qian, C. Shao, X. Wang and M. Shahidehpour, "Deep Reinforcement Learning for EV Charging Navigation by Coordinating Smart Grid and Intelligent Transportation System," in IEEE Transactions on Smart Grid, vol. 11, no. 2, pp. 1714-1723, March 2020, doi: 10.1109/TSG.2019.2942593.
7. Ma, J., Zhang, Y., Duan, Z., & Tang, L. (2023). PROLIFIC: Deep Reinforcement Learning for Efficient EV Fleet Scheduling and Charging. Sustainability, 15(18), 13553. https://doi.org/10.3390/su151813553.
8. M. Aljaidi, N. Aslam, X. Chen, O. Kaiwartya, Y. A. Al-Gumaei and M. Khalid, "A Reinforcement Learning-based Assignment Scheme for EVs to Charging Stations," 2022 IEEE 95th Vehicular Technology Conference: (VTC2022-Spring), Helsinki, Finland, 2022, pp. 1-7, doi: 10.1109/VTC2022-Spring54318.2022.9860535.
9. Mamidala, S., Venkata Pavan Kumar, Y., & Mallipeddi, R. (2025). Revolutionizing Electric Vehicle Charging Stations with Efficient Deep Q Networks Powered by Multimodal Bioinspired Analysis for Improved Performance. Energies, 18(7), 1750. https://doi.org/10.3390/en18071750.
10. K. Zhan, Y. Huang, J. Su, M. Xu, L. Nong and H. Zeng, "Smart Grid-Driven Intelligent EV Charging Scheduling Strategy Based on Deep Q-Networks," 2024 IEEE 12th International Conference on Information, Communication and Networks (ICICN), Guilin, China, 2024, pp. 223-228, doi: 10.1109/ICICN62625.2024.10761617.
11. N. Nikolov, O. Nakov and D. Gotseva, "Design and Research of Smart IoT Control System for Electrical Appliances," 2021 29th National Conference with International Participation (TELECOM), Sofia, Bulgaria, 2021, pp. 39-42, doi: 10.1109/TELECOM53156.2021.9659650.
12. N. Nikolov and D. Gotseva, "Design and research of IoT room monitoring system," 2021 XXX International Scientific Conference Electronics (ET), Sozopol, Bulgaria, 2021, pp. 1-4, doi: 10.1109/ET52713.2021.9579638.
13. N. Nikolov, O. Nakov and D. Gotseva, "Research of MQTT versus LwM2M IoT communication protocols for IoT," 2021 56th International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST), Sozopol, Bulgaria, 2021, pp. 45-48, doi: 10.1109/ICEST52640.2021.9483477.
14. N. Nikolov, O. Nakov and D. Gotseva, "Operating Systems for IoT Devices," 2021 56th International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST), Sozopol, Bulgaria, 2021, pp. 41-44, doi: 10.1109/ICEST52640.2021.9483469.

Issue

Copyright IEEE