Inverse Reinforcement Learning from Failure

The paper “Inverse Reinforcement Learning from Failure” was published at the Workshop on Learning from Demonstration: Inverse Optimal Control, Reinforcement Learning, and Lifelong Learning at RSS-15 in Rome.

Kyriacos Shiarlis, Joao Messias, Maarten van Someren, and Shimon Whiteson. Inverse Reinforcement Learning from Failure. In RSS 2015: Proceedings of the 2015 Robotics: Science and Systems Conference, Workshop on Learning from Demonstration: Inverse Optimal Control, Reinforcement Learning, and Lifelong Learning, July 2015.

In this paper, we approach the problem of Inverse Reinforcement Learning (IRL) from a rather different perspective. Instead of trying to only mimic an expert as in traditional IRL, we present a method that can also utilise failed or bad demonstrations of a task. In particular, we propose a new IRL algorithm that extends the state-of-the-art method of Maximum Causal Entropy Inverse Reinforcement Learning to exploit such failed demonstrations. Furthermore, we present experimental results showing that our method can learn faster and better than its original counterpart.