Energy-Based Models for End-to-End Autonomous Driving

Abstract

Energy-based models (EBMs), a promising class of machine learning models, have shown impressive results in several domains, from natural language generation to computer vision. Learning to imitate expert demonstrations using an EBM has recently achieved state-of-the-art results in robotics, made possible by EBMs’ better ability to handle multimodal probability distributions and learn behavior with abrupt command changes. In this work, EBMs are tested for the first time in the end-to-end autonomous driving domain on a real car. As a result, it is discovered that a simple EBM variant performs slightly better and is more stable than a baseline conventional neural network architecture. At the same time, EBMs turn out to exhibit a higher variability of predictions over time, or whiteness. As a solution to this problem, this work introduces a regularization technique that makes the predictions more smooth over time. In addition, an energybased uncertainty metric is proposed, but its usefulness could not be assessed with sufficient reliability due to an insufficient number of real car evaluations. The thesis suggests several ideas for future work, such as using a different sampling method and comparing against mixture density networks.