Direct adaptive pole-placement controller using deep reinforcement learning: Application to AUV control

In this paper we investigate a direct adaptive learning-based tuning strategy for the control of an underwater vehicle under unknown disturbances. This process can be seen as a double integrator without delay and is usually regulated using a PD/PID type controller. A trade-off between performance and robustness may be found when tuning their parameters because a single optimal controller for multiple operating condition does not exist. Therefore, we use a re-parametrization of the PID controller gains in a space of poles where controller stability is guaranteed. We propose to use the maximum entropy deep reinforcement learning algorithm called SAC to explore this space. The adaptation procedure is able to capture a great variety of desired pole locations in order to adapt to process variations without measuring them. Simulation outcomes show the advantages of this approach.