Physics-Informed Imitation Learning for Autonomous Ships
Journal Title:
International Maritime and Port Technology and Development Conference (MTEC) and the 6th International Conference on Maritime Autonomous Surface Ship (ICMASS) 2024
Xingrui Yu, Liangbin Zhao, Hongwei Wang, Xiuju Fu and Ivor Tsang. Physics-Informed Imitation Learning for Autonomous Ships. International Maritime and Port Technology and Development Conference (MTEC) and the 6th International Conference on Maritime Autonomous Surface Ship (ICMASS) 2024.
Abstract:
The collision avoidance decision system is the core guarantee of navigation safety for autonomous ships. Automation in marine traffic faces numerous challenges due to the complex dynamics of ships and the absence of physical constraints on water, particularly in scenarios with high traffic density and multiple ship encounters. Deep reinforcement learning (DRL) is considered a promising solution because of its end-to-end learning capability and its ability to generalize to complex environments. However, achieving ship maneuvering control at a level close to or comparable with human performance is challenging when applying DRL. A key step in this multi-objective optimization process is the configuration of the reward function, which has proven to be both difficult and abstract. One objective that is challenging to quantify in design is ensuring that unmanned vessels can adhere to the same rules and conventions as traditional manned vessels. This means that avoidance maneuvers under DRL model control need to comply with the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs).
In this paper, we explore a solution to this issue by investigating an imitation learning-based collision avoidance method for autonomous collision avoidance. Specifically, we propose a novel physics-informed inverse reinforcement learning method to train autonomous ships, enabling them to exhibit safe, efficient, and navigation rule-compliant ship maneuvering behavior for collision avoidance. Firstly, we collect human expert demonstrations in a simulator with a systematically designed series of complex multi-ship encounter scenarios. Then, we train a novel physics-informed reward model on these demonstrations, capturing the underlying intention of human expert behaviors and inferring intrinsic rewards for policy training. Through simulation, our method demonstrates its ability to overcome the challenges associated with human-designed reward functions in complex encounter scenarios. Furthermore, the physics-informed rewards inherently ensure that the maneuvering of autonomous ships adhere to COLREGs, resembling those of human experts. Extensive experiments conducted across a range of complex multi-ship encounter scenarios validate the effectiveness and efficiency of our approach.
License type:
Attribution 4.0 International (CC BY 4.0)
Funding Info:
This research / project is supported by the Maritime and Port Authority of Singapore - Maritime Artificial Intelligence (AI) Research Programme
Grant Reference no. : SMI-2022-MTP-06