Autors: Ivanov G., Mateev, V. M., Marinova, I. I., Gruber W., Marth E., Mallinger S.
Title: Reinforcement Learning Optimization of Coaxial Magnetic Gear Geometry with Finite Element Analysis
Keywords: actor–critic algorithm, design optimization, electromagnetic modeling, magnetic gear, neural network, reinforcement learning

Abstract: This manuscript presents a reinforcement learning (RL) agent method to optimize the geometry of a coaxial magnetic gear using a 2D finite element magnetic (FEM) simulation. The proposed optimization algorithm aims to improve the maximum torque within given boundaries of the magnetic gear geometry by adjusting parameterized radii. A linear actor–critic gradient algorithm is implemented, where the actor learns a policy to adjust and discover the values of five geometric parameters of the magnetic gear model, and the critic evaluates the performance of the resulting designs. The RL agent interacts with an environment integrated with a 2D FEM simulation, which provides feedback by calculating the total torque of the new geometry discovered. The optimization algorithm uses a greedy exploration method that uses the total torque as a reward system, which the RL agent aims to maximize. The results obtained for the magnetic gear optimization demonstrate the effectiveness of the proposed RL algorithm, which can be applied to automate multiparameter geometric optimization using artificial intelligence systems.

References

  1. Ruiz-Ponce G. Arjona M.A. Hernandez C. Escarela-Perez R. A review of magnetic gear technologies used in mechanical power transmission Energies 2023 16 1721 10.3390/en16041721
  2. Rasmussen P.O. Andersen T. Jorgensen F. Nielsen O. Development of a high-performance magnetic gear IEEE Trans. Ind. Appl. 2005 41 764 770 10.1109/TIA.2005.847319
  3. Sezen S. Yilmaz K. Aktas S. Ayaz M. Dindar T. Solid Core Magnetic Gear Systems: A Comprehensive Review of Topologies, Core Materials, and Emerging Applications Appl. Sci. 2025 15 8560 10.3390/app15158560
  4. McGilton B. Crozier R. McDonald A. Mueller M. Review of magnetic gear technologies and their applications in marine energy IET Renew. Power Gener. 2018 12 174 181 10.1049/iet-rpg.2017.0210
  5. Filippini M. Alotto P. Coaxial magnetic gear design and optimization IEEE Trans. Ind. Electron. 2017 64 9934 9942 10.1109/TIE.2017.2721918
  6. Todorova M. Mateev V. Marinova I. Magnetic gear design optimization by genetic algorithm with ANN controlled crossover and mutation Proceedings of the 2021 IV International Conference on High Technology for Sustainable Development (HiTech) Sofia, Bulgaria 7–8 October 2021
  7. Liu X. Zhao Y. Chen Z. Luo D. Huang S. Multi-objective robust optimization for a dual-flux-modulator coaxial magnetic gear IEEE Trans. Magn. 2019 55 8002008 10.1109/TMAG.2018.2887273
  8. Lei G. Zhu J. Guo Y. Liu C. Ma B. A review of design optimization methods for electrical machines Energies 2017 10 1962 10.3390/en10121962
  9. Hong H. Kim S. Kim W. Kim W. Jeong J.M. Kim S.S. Design optimization of 3D printed kirigami-inspired composite metamaterials for quasi-zero stiffness using deep reinforcement learning integrated with bayesian optimization Compos. Struct. 2025 359 119031 10.1016/j.compstruct.2025.119031
  10. Wang C. Dong T. Chen L. Zhu G. Chen Y. Multi-objective optimization approach for permanent magnet machine via improved soft actor–critic based on deep reinforcement learning Expert Syst. Appl. 2025 264 125834 10.1016/j.eswa.2024.125834
  11. Ha V.T. Tuan D.A. Van T.T. Torque control of PMSM motors using reinforcement learning agent algorithm for electric vehicle application Bull. Electr. Eng. Inform. 2025 14 2571 2581 10.11591/eei.v14i4.7852
  12. Hou S. Hao X. Pan D. Wu W. Physics-informed neural network for simulating magnetic field of coaxial magnetic gear Eng. Appl. Artif. Intell. 2024 133 108302 10.1016/j.engappai.2024.108302
  13. Li Y. Lei G. Bramerdorfer G. Peng S. Sun X. Zhu J. Machine learning for design optimization of electromagnetic devices: Recent developments and future directions Appl. Sci. 2021 11 1627 10.3390/app11041627
  14. Marinova I. Mateev V. Second order genetic algorithm for magnetic design optimization AIP Conference Proceedings AIP Publishing LLC Melville, NY, USA 2022 2505 080018 10.1063/5.0100799
  15. Van Rossum G. Drake F.L. An Introduction to Python Network Theory Ltd. Bristol, UK 2003
  16. Python version 24 Python Releases for Windows Python Software Foundation Wolfeboro Falls, NH, USA 2021
  17. Meeker D. FEMM 4.2 Magnetostatic Tutorial. Computer Program 2006 Available online: https://www.femm.info/wiki/MagneticsTutorial (accessed on 10 September 2020)
  18. Dos Santos Mignon A. da Rocha R.L.d.A. An adaptive implementation of ε-greedy in reinforcement learning Procedia Comput. Sci. 2017 109 1146 1151 10.1016/j.procs.2017.05.431
  19. Tokic M. Adaptive ε-greedy exploration in reinforcement learning based on value differences Annual Conference on Artificial Intelligence Springer Berlin/Heidelberg, Germany 2010
  20. Todorova M. Mateev V. Marinova I. Permanent magnets for a magnetic gear Proceedings of the 2016 19th International Symposium on Electrical Apparatus and Technologies (SIELA) Bourgas, Bulgaria 29 May–1 June 2016
  21. Marth E. Gruber W. Mallinger S. Mateev V. Marinova I. Magnetic-Geared Bearingless Motor Unit with Central Exterior Output Proceedings of the 2024 23rd International Symposium on Electrical Apparatus and Technologies (SIELA) Bourgas, Bulgaria 12–15 June 2024
  22. Di Barba P. Gottvald A. Savini A. Global Optimization of Loney’s Solenoid: A Benchmark Problem Int. J. Appl. Electromagn. Mech. 1995 6 273 276 10.1177/138354169500600404
  23. PyTorch Foundation Actor-Critic Methods. Meta 2022 Available online: https://docs.pytorch.org/rl/main/reference/objectives_actorcritic.html (accessed on 10 October 2025)
  24. Tadepalli P. Ok D. Model-based average reward reinforcement learning Artif. Intell. 1998 100 177 224 10.1016/S0004-3702(98)00002-2
  25. Oja E. Principal components, minor components, and linear neural networks Neural Netw. 1992 5 927 935 10.1016/S0893-6080(05)80089-9
  26. Yolcu U. Egrioglu E. Aladag C.H. A new linear & nonlinear artificial neural network model for time series forecasting Decis. Support Syst. 2013 54 1340 1347 10.1016/j.dss.2012.12.006
  27. Meeker D. Finite element method magnetics FEMM 2010 4 162
  28. Bello I. Pham H. Le Q.V. Norouzi M. Bengio S. Neural combinatorial optimization with reinforcement learning arXiv 2016 1611.09940
  29. Zhang B. Zhao J. Xia Y. Peng X. Shi X. Zhu X. Qu B. Yang K. Multi-Objective Collaborative Optimization of Magnetic Gear Compound Machines Using Parameter Grouping and Kriging Surrogate Models Energies 2025 18 6153 10.3390/en18236153
  30. Salon S. Finite Element Analysis of Electrical Machines Springer Troy, NY, USA 1995
  31. Rosenstein M. Barto A.G. Si J. Barto A. Powell W. Wunsch D. Supervised actor-critic reinforcement learning Learning and Approximate Dynamic Programming: Scaling Up to the Real World John Wiley and Sons, Inc. New York, NY, USA 2004 359 380
  32. Gruslys A. Dabney W. Azar M.G. Piot B. Bellemare M. Munos R. The reactor: A fast and sample-efficient actor-critic agent for reinforcement learning arXiv 2017 1704.04651

Issue

Machines, vol. 13, 2025, Switzerland, https://doi.org/10.3390/machines13121143

Вид: статия в списание, публикация в издание с импакт фактор, публикация в реферирано издание, индексирана в Scopus и Web of Science