A Proposal of a Behavior-based Control Architecture with Reinforcement Learning for an Autonomous Underwater Robot

Carreras, Marc
A Proposal of a Behavior-based Control Architecture with Reinforcement Learning for an Autonomous Underwater Robot
(pdf - 4 MB)

Abstract: The achievement of a mission with an autonomous robot in an unknown and unstructured environment is still a non-solved problem. This thesis proposes the use of a behavior-based control architecture and a reinforcement learning algorithm to accomplish the sub-goals of a mission. The behavior-based approach provides real-time capabilities and reactivity to the perceived environment. Reinforcement learning is used to learn the state-action mappings that each behavior must contain. The biggest problem of a reinforcement learning algorithm when it is applied to a real system is the generalization problem. In the presented approach, this is solved by combining the Q_learning algorithm with a Neural Network. However, the use of a NN to learn a non-uniform set of samples (as happens in real experiments) causes what it is known as the interference problem. The RL algorithm presented in this thesis solves this new problem with a database of learning samples. Real experiments show the effectiveness of the approach with an autonomous underwater robot. Also, the generalization capability of the reinforcement learning algorithm is successfully tested with the “mountain-car” benchmark.