Optimal robust formation control for heterogeneous multi-agent systems based on reinforcement learning

Abstract

In this paper, a reinforcement learning-based robust control strategy is proposed for uncertain heterogeneous multi-agent systems to achieve optimal collision-free time-varying formations. Without using any global information, a fully distributed adaptive observer is developed to estimate both dynamics and states of the reference and disturbance systems. The observer parameters are found by an observed model-based or a model-free off-policy reinforcement learning algorithm. Using the internal model principle, a novel optimal robust formation control strategy is developed based on another proposed off-policy reinforcement learning algorithm. The algorithm addresses the non-quadratic optimization problem when the system model is completely unknown. Taking the bushfire edge tracking and patrolling task for an unmanned aerial vehicle-unmanned ground vehicle heterogeneous system as an example, the effectiveness, and robustness of the developed control strategy are verified by simulations.