Learning Methods of Improving Generalization Performance for Autonomous Driving

Abstract

Autonomous driving is of great interest to industry and academia alike. The use of machine learning approaches for autonomous driving has long been studied, but mostly in the context of perception for a traditional modular approach. In recent years, the end-to-end approach has been receiving interest as a promising controller to treat the limitations of the modular system caused by high software complexity and fixed driving behavior. The end-to-end frameworks have been outstanding achievements by successfully deploying an imitation learning strategy and become a growing trend in autonomous driving research. However, there are still challenging problems in the new driving environment since even minor visual changes could make the trained model completely fail. This dissertation introduces two vision-based behavior cloning frameworks for autonomous driving by utilizing novel learning strategies to enhance the generalization ability under the unseen environments in the training process. We propose a future action and states network (FASNet) that uses predicted future actions and states to calculate control values based on a voting mechanism for degrading an adverse effect of a single incorrect prediction. To eliminate a temporal gap between a current observation and the future actions, we generate the future states without additional observations using an unsupervised generative method and vehicle dynamics. Based on multi-task learning, the perception network of the proposed FASNet is shared between closely related sub-tasks: two for target control tasks and two for auxiliary localization tasks. The joint optimization improves the generalization and regularization abilities of the trained model. To demonstrate our approach, we show the validity of the proposed FASNet by conducting experiments, including ablation studies, under several driving scenarios. FASNet significantly increases the driving performance by 7.9% in average success rate (SR) on CARLA NoCrash benchmark and 7.6% on CARLA AnyWeather benchmark by applying proposed concepts to base network. We propose a mixture of domain experts (MoDE) network that predicts control values by utilizing the cooperation of experts specialized in domain-specific sub-tasks. The proposed MoDE separates a latent space using two-stage disentangled representation learning. Domain-specific and domain-general features are separated by cycle consistency loss. Then, dynamic-object features, which contain information about dynamic objects, are extracted using mutual information estimator. Since the domain-general and dynamic-object features are consistent across multiple domains, these are applied in estimating the control values. On the other hand, the domain-specific features, which contain only specific domain information, are used to calculate the importance weight of the domain experts. We show the validity of the proposed MoDE by visualizing the disjoint latent spaces using t-SNE plots and conducting several driving experiments. MoDE has a 2.1% and 1.2% higher average SRs than FASNet on CARLA NoCrash benchmark and CARLA AnyWeather benchmark. Finally, the proposed frameworks outperform prior state-of-the-art approaches under several driving tasks, especially in unseen driving environments.