Evaluation of a Deep-Reinforcement-Learning-based Controller for the Control of an Autonomous Underwater Vehicle

Yoann Sola, Thomas Chaffre, Gilles Le Chenadec, Karl Sammut, Benoit Clement

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)


The development of efficient controllers in underwater environments has long been a challenging topic, hindered mostly by the lack of understanding in process variations under these conditions. It is without a doubt difficult for an autonomous underwater vehicle to behave as instructed while being constantly trying to compensate for the disturbing forces that act on its body. Recently, noteworthy improvements have been made in the model-free control theory, allowing the use of Reinforcement-Learning-based controllers in terrestrial and aerial robotic contexts. In contrast, the underwater control field has been largely dominated by controllers based on the Proportional-Integral-Derivative structure. In this paper we compare a PID controller with a leading-edge Deep Reinforcement Learning algorithm, the Soft Actor-Critic. These controllers have been tested within marine robotics simulations, using a realistic simulated environment including external disturbances for a waypoint tracking mission. The results obtained reveal the superiority of PID controllers in terms of success rate and precision, but not in terms of thruster usage. Future works will include the test of these controllers with an underactuated AUV, to demonstrate the guidance abilities of the learning approach.

Original languageEnglish
Title of host publication2020 Global Oceans 2020
Subtitle of host publicationSingapore - U.S. Gulf Coast
Place of PublicationUnited States of America
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages7
ISBN (Electronic)9781728154466
ISBN (Print)9781728184098
Publication statusPublished - 5 Oct 2020
Event2020 Global Oceans: : Singapore - U.S. Gulf Coast, OCEANS 2020 - Biloxi, United States
Duration: 5 Oct 202030 Oct 2020

Publication series

Name2020 Global Oceans 2020: Singapore - U.S. Gulf Coast


Conference2020 Global Oceans:
Country/TerritoryUnited States


  • AUV
  • control theory
  • deep reinforcement learning
  • precision
  • simulation
  • thrusters usage


Dive into the research topics of 'Evaluation of a Deep-Reinforcement-Learning-based Controller for the Control of an Autonomous Underwater Vehicle'. Together they form a unique fingerprint.

Cite this