Y. Wang, M. Xiao and Z. Wu, "Safe Transfer-Reinforcement-Learning-Based Optimal Control of Nonlinear Systems," in IEEE Transactions on Cybernetics, vol. 54, no. 12, pp. 7272-7284, Dec. 2024, doi: 10.1109/TCYB.2024.3485697.
Abstract:
—Traditional reinforcement learning (RL) methods
for optimal control of nonlinear processes often face challenges,
such as high computational demands, long training times, and
difficulties in ensuring the safety of closed-loop systems during
training. To address these issues, this work proposes a safe
transfer RL (TRL) framework. The TRL algorithm leverages
knowledge from pretrained source tasks to accelerate learning
in a new, related target task, significantly reducing both learning time and computational resources required for optimizing
control policies. To ensure safety during knowledge transfer and
training, data collection and optimization of the control policy
are performed within a control invariant set (CIS) throughout
the learning process. Furthermore, we theoretically analyze the
errors between the approximate and optimal control policies by
accounting for the differences between source and target tasks.
Finally, the proposed TRL method is applied to the case studies of
chemical processes to demonstrate its effectiveness in solving the
optimal control problem with improved computational efficiency
and guaranteed safety
License type:
Attribution 4.0 International (CC BY 4.0)
Funding Info:
This research / project is supported by the Agency for Science, Technology and Research - Manufacturing, Trade, and Connectivity (MTC) Young Individual Research Grants 2022
Grant Reference no. : M22K3c0093