Е-Публикации
Технически университет - София

Abstract: This article presents the synthesis, implementation, and experimental study of a PI-based adaptive actor–critic displacement volume controller of an axial-piston pump intended for open-loop circuit hydraulic drive systems. The proposed control structure combines a conventional PI actor with an adaptive critic that estimates the infinite-horizon cost through Bellman-error minimization. By using the tracking error and its integral as actor inputs, the controller avoids the need for an accurate plant model while retaining a compact and practically implementable structure. The adaptive laws are derived using gradient-based learning, and a Lyapunov-based analysis establishes closed-loop stability for sufficiently small adaptation gains. The controller is implemented in a fixed-step Simulink® environment and deployed on a rapid prototyping platform with real-time communication to an industrial microcontroller and proportional valve amplifier. The experimental results obtained under four fixed loading conditions and dynamic load variations demonstrate a stable operation, bounded critic behavior, and a near-zero Bellman error during learning. Comparative tests against a classical PI controller, a Lyapunov-based model reference adaptive controller, and a generic actor–critic scheme show that the proposed PI-based actor–critic achieves the lowest performance index and the shortest settling times in most cases.

References

1. Findeisen D. Helduser S. Ölhydraulik Springer Berlin/Heidelberg, Germany 2015
2. Ivantysyn J. Ivantysynova M. Hydrostatic Pumps and Motors: Principles, Design, Performance, Modelling, Analysis, Control and Testing Academia Books International New Delhi, India 2001
3. Manring N. Fluid Power Pumps and Motors: Analysis, Design, and Control McGraw-Hill Education New York, NY, USA 2013
4. Frankenfield T. Using Industrial Hydraulics Rexroth Worldwide Hydraulics, Penton Publishing Inc. New York, NY, USA 1984
5. Tonyan M. Electronically Controlled Proportional Valves Marcel Dekker, Inc. New York, NY, USA, USA 1985
6. Skarpetis M.G. Automatic control of Hydraulic Systems Nova Science Publishers, Inc. New York, NY, USA 2023
7. Zhou K. Doyle J. Robust and Optimal Control Prentice Hall International Upper Saddle River, NJ, USA 1996
8. Park S. Lee J. Kim J. Robust control of the pressure in a control-cylinder with direct drive valve for the variable displacement axial piston pump Proc. Inst. Mech. Eng. Part I J. Syst. Control. Eng. 2009 223 455 465 10.1243/09596518JSCE677
9. Zeiger G. Akers A. The application of linear optimal control techniques to axial piston pump controller design SAE Tech. Pap. 1989 98 38 49 10.4271/890953 25382306
10. Lin S. Akers A. Optimal control theory applied to pressure-controlled axial piston pump design J. Dyn. Syst. Meas. Control. Trans. ASME 1990 112 475 481 10.1115/1.2896167
11. Berg H. Ivantysynova M. Design and testing of a robust linear controller for secondary controlled hydraulic drive Proc. Inst. Mech. Eng. Part I J. Syst. Control. Eng. 1999 213 375 385 10.1243/0959651991540223
12. Zeiger G. Akers A. Dynamic Analysis of an Axial Piston Pump Swashplate Control Proc. Inst. Mech. Eng. Part C J. Mech. Eng. Sci. 1986 200 49 58 10.1243/PIME_PROC_1986_200_093_02
13. Berg H. Ivantysynova M. Robust closed loop speed and angular position control for variable displacement hydraulic motors supplied from a constant pressure mains system Olhydraulik Pneum. 1999 43 405 410
14. Zhang R. Alleyne A. Prasetiawan E. Modeling and H 2/H ∞ MIMO control of an earthmoving vehicle powertrain J. Dyn. Syst. Meas. Control. Trans. ASME 2002 124 625 636 10.1115/1.1515326
15. Lennevi J. Palmberg J.-O. Application and implementation of LQ design method for the velocity control of hydrostatic transmissions Proc. Inst. Mech. Eng. Part I J. Syst. Control. Eng. 1995 209 255 268 10.1243/pime_proc_1995_209_393_02
16. Heybroek K. Larsson J. Palmberg J.-O. Open Circuit Solution for Pump Controlled Actuators Proceedings of the 4th FPNI-PhD Symposium Sarasota, FL, USA 13–17 June 2006 27 40 10.1243/PIME_PROC_1995_209_393_02
17. Zhang P. Li Y. Research on Control Methods for the Pressure Continuous Regulation Electrohydraulic Proportional Axial Piston Pump of an Aircraft Hydraulic System Appl. Sci. 2019 9 1376 10.3390/app9071376
18. Kemmetmüller W. Fuchshumer F. Kugi A. Nonlinear pressure control of self-supplied variable displacement axial piston pumps Control. Eng. Pract. 2010 18 84 93 10.1016/j.conengprac.2009.09.006
19. Wei J. Guo K. Fang J. Tian Q. Nonlinear supply pressure control for a variable displacement axial piston pump Proc. Inst. Mech. Eng. Part I J. Syst. Control. Eng. 2015 229 614 624 10.1177/0959651815577546
20. Åström K.J. Wittenmark B. Adaptive Control Courier Corporation North Chelmsford, MA, USA 2013
21. Helian B. Mustalahti P. Mattila J. Chen Z. Yao B. Adaptive robust pressure control of variable displacement axial piston pumps with a modified reduced-order dynamic model Mechatronics 2022 87 102879 10.1016/j.mechatronics.2022.102879
22. Feng Y. Jian Z. Li J. Tao Z. Wang Y. Xue J. Advanced Control Systems for Axial Piston Pumps Enhancing Variable Mechanisms and Robust Piston Positioning Appl. Sci. 2023 13 9658 10.3390/app13179658
23. Busquets E. Ivantysynova M. Handroos H. Discontinuous projection-based adaptive robust control for displacement-controlled actuators J. Dyn. Syst. Meas. Control. Trans. ASME 2015 137 8 10.1115/1.4030064
24. Haggag S.A. Robust control and modelling of a heavy equipment variable displacement pump hydraulic system Int. J. Heavy Veh. Syst. 2011 18 288 302 10.1504/IJHVS.2011.041595
25. Guo K. Wei J. Adaptive robust control of variable displacement pumps Proceedings of the American Control Conference Washington, DC, USA 17–19 June 2013
26. Koivumäki J. Mattila J. Adaptive and nonlinear control of discharge pressure for variable displacement axial piston pumps J. Dyn. Syst. Meas. Control. Trans. ASME 2017 139 10 10.1115/1.4036537
27. Slavov T. Mitov A. Kralev J. Lyapunov-Based Two-Degree-of-Freedom Model Reference Adaptive Control of Axial-Piston Pump Mathematics 2025 13 3513 10.3390/math13213513
28. Mitov A. Slavov T. Kralev J. Comparison of Advanced Multivariable Control Techniques for Axial-Piston Pump Processes 2024 12 1797 10.3390/pr12091797
29. Slavov T. Mitov A. Kralev J. Novel Approach for Robust Control of Axial Piston Pump Mathematics 2025 13 643 10.3390/math13040643
30. Ljung L. System Identification: Theory for the User 2nd ed. Prentice Hall Upper Saddle River, NI, USA 1999
31. Mitov A. Kralev J. Slavov T. Identification of Variable Displacement Axial-Piston Pump with Proportional Valve Control AIP Conf. Proc. 2014 3064 030008 10.1063/5.0198991
32. Zhou J. Zhang T. Zhang H. Zhang Z. Hong J. Yang J. Energy management strategy for electro-hydraulic hybrid electric vehicles considering optimal mode switching: A soft actor-critic approach trained on a multi-modal driving cycle Energy 2024 305 132172 10.1016/j.energy.2024.132172
33. Krishnakumar K. Limes G. Gundy-Burlet K. Bryant D. An adaptive critic approach to reference model adaptation Proceedings of the AIAA Guidance, Navigation, and Control Conference and Exhibit Austin, TX, USA 11–14 August 2003
34. Akraminia M. Tatari M. Fard M. Jazar R.N. Designing active vehicle suspension system using critic-based control strategy Nonlinear Eng. 2015 4 141 154 10.1515/nleng-2015-0004
35. Agvik R. Vännman S. Adaptive Control of Hydraulic Drive System Chalmers University of Technology Gothenburg, Sweden 2023
36. Han T. Nie X. Que N. Lu J. Yao J. Yu X. Predefined-Time Tracking Control of Servo Hydraulic Cylinder Based on Reinforcement Learning Actuators 2026 15 9 10.3390/act15010009
37. Li Y. Qi X. Research on Motion Control of Hydraulic Manipulator Based on Prescribed Performance and Reinforcement Learning Actuators 2026 15 39 10.3390/act15010039
38. Wei X. Ye J. Xu J. Tang Z. Adaptive Dynamic Programming-Based Cross-Scale Control of a Hydraulic-Driven Flexible Robotic Manipulator Appl. Sci. 2023 13 2890 10.3390/app13052890
39. Su Q. Pei Z. Tang Z. Tracking Control for a Lower Extremity Exoskeleton Based on Adaptive Dynamic Programing Biomimetics 2023 8 353 10.3390/biomimetics8040353 37622958
40. He J. Zhou L. Li C. Li T. Huang J. Su S. Control Strategy of Hydraulic Servo Control Systems Based on the Integration of Soft Actor-Critic and Adaptive Robust Control IEEE Access 2024 12 63629 63643 10.1109/ACCESS.2024.3396283
41. Teng W. Wang G. Adaptive Optimal Control Based on Critic-Actor Architecture for Hydraulic Support Cylinder System with Asymmetric Output Error Constraints Eng. Lett. 2025 33 3535 3542
42. Kong Y. Wang Y. Wang Y. Zhu S. Zhang R. Wang L. Deep Reinforcement Learning Trajectory Tracking Control for a Six-Degree-of-Freedom Electro-Hydraulic Stewart Parallel Mechanism Eng 2025 6 212 10.3390/eng6090212
43. Yuan X. Wang Y. Zhang R. Gao Q. Zhou Z. Zhou R. Yin F. Reinforcement Learning Control of Hydraulic Servo System Based on TD3 Algorithm Machines 2022 10 1244 10.3390/machines10121244
44. Singh Sidhu H. Siddhamshetty P. Kwon J.S. Approximate Dynamic Programming Based Control of Proppant Concentration in Hydraulic Fracturing Mathematics 2018 6 132 10.3390/math6080132
45. Rout R. Kumawat A.K. Reinforcement Learning-Based Position Tracking Control for Proportional Directional Control Valve-Based Electro-Hydraulic System IEEE Access 2025 13 159597 159609 10.1109/ACCESS.2025.3608253
46. Jia C. Yu T. Song Z. Robust reinforcement learning with augmented state for leveling control of multi-cylinder hydraulic system J. Supercomput. 2025 81 1 10.1007/s11227-024-06681-x
47. Khater A.A. Fekry M. El-Bardini M. El-Nagar A.M. Deep reinforcement learning-based adaptive fuzzy control for electro-hydraulic servo system Neural Comput. Appl. 2025 37 30 10.1007/s00521-024-10741-x
48. Hao X. Xin Z. Huang W. Wan S. Qiu G. Wang T. Wang Z. Deep reinforcement learning enhanced PID control for hydraulic servo systems in injection molding machines Sci. Rep. 2025 15 23005 10.1038/s41598-025-05904-2 40596206
49. Hu P. Wen T. Zhang D. Bayesian reinforcement learning for adaptive control of energy recuperation in hydraulic excavator arms Sci. Rep. 2026 16 6195 10.1038/s41598-026-35391-y 41582225
50. Wang Z. Zhang X. Li W. Feng Z. Real-Time UUV Obstacle Avoidance Through Flexible Steering Technology Based on Improved Soft Actor-Critic Framework IEEE Trans. Instrum. Meas. 2025 74 1 14 10.1109/TIM.2025.3612643
51. Kralev J. Mitov A. Slavov T. Approximated Adaptive Dynamic Programming Control of Axial-Piston Pump Mathematics 2026 14 1127 10.3390/math14071127
52. Rexroth Bosch Group Pressure and Flow Control System Technical Data Sheet, RE 30630 Rexroth Bosch Group Lohr am Main, Germany 2015
53. Rexroth Bosch Group Proportional Directional Valves, Direct Operated, with Electrical Position Feedback as Pilot Control Valve for Control Systems SY(H)DFE Technical Data Sheet, RE 29016 Rexroth Bosch Group Lohr am Main, Germany 2019
54. Kordak R. Hydrostatic Drives with Control of the Secondary Unit The Hydraulic Trainer Vol. 6 Mannesmann Rexroth GmbH Lohr am Main, Germany 1996 Volume 6
55. Mitov A. Slavov T. Kralev J. Rapid Prototyping of H∞ Algorithm for Real-Time Displacement Volume Control of Axial Piston Pumps Algorithms 2023 16 120 10.3390/a16020120
56. Mitov A. Kralev J. Slavov T. Angelov I. Design of Embedded Control System for Open Circuit Axial Piston Pump Proceedings of the 22nd International Symposium on Electrical Apparatus and Technologies, SIELA 2022—Proceedings Bourgas, Bulgaria 1–4 June 2022 10.1109/SIELA54794.2022.9845787
57. Danfoss Plus+1 Controllers MC012-020 and 022. Data Sheet, 11077167, Rev DA, 2013 Available online: https://assets.danfoss.com/documents/75707/AI152886481115en-000504.pdf (accessed on 19 June 2026)

Issue