Automatic collision avoidance of multiple ships based on deep Q-learning

Abstract As the number of ships for marine transportation increases with the globalisation of the world economy, waterways are becoming more congested than before. This situation will raise the risk of collision of the ships; hence, an automatic collision avoidance system needs to be developed. In this paper, a novel approach based on deep reinforcement learning (DRL) is proposed for automatic collision avoidance of multiple ships particularly in restricted waters. A training method and algorithms for collision avoidance of ships, incorporating ship manoeuvrability, human experience and navigation rules, are presented in detail. The proposed approach is investigated not only by numerical simulations but also by model experiments using three self-propelled ships. Through the systematic numerical and experimental validation, it is demonstrated the developed approach based on the DRL has great possibility for realising automatic collision avoidance of ships in highly complicated navigational situations.