Evaluation of a Deep-Reinforcement-Learning-based Controller for the Control of an Autonomous Underwater Vehicle

The development of efficient controllers in underwater environments has long been a challenging topic, hindered mostly by the lack of understanding in process variations under these conditions. It is without a doubt difficult for an autonomous underwater vehicle to behave as instructed while being constantly trying to compensate for the disturbing forces that act on its body. Recently, noteworthy improvements have been made in the model-free control theory, allowing the use of Reinforcement-Learning-based controllers in terrestrial and aerial robotic contexts. In contrast, the underwater control field has been largely dominated by controllers based on the Proportional-Integral-Derivative structure. In this paper we compare a PID controller with a leading-edge Deep Reinforcement Learning algorithm, the Soft Actor-Critic. These controllers have been tested within marine robotics simulations, using a realistic simulated environment including external disturbances for a waypoint tracking mission. The results obtained reveal the superiority of PID controllers in terms of success rate and precision, but not in terms of thruster usage. Future works will include the test of these controllers with an underactuated AUV, to demonstrate the guidance abilities of the learning approach.