Automatic Bi-LSTM Architecture Search Using Bayesian Optimisation for Vehicle Activity Recognition

This paper presents a novel method to find optimal Bidirectional Long-Short Term Memory Neural Network (Bi-LSTM) using Bayesian Optimisation method for vehicle trajectory classification. We extend our previous approach to be able to classify a larger number of vehicle trajectories collected from different sources in a single Bi-LSTM network. We also explored the use of deep learning visual explainability by highlighting the parts of the activity (or trajectory) contribute to the classification decision of the network. In particular, Qualitative Trajectory Calculus (QTC), spatio-temporal calculus, method is used to encode the relative movement between vehicles as a trajectory of QTC states. We then develop a Bi-LSTM network (called VNet) to classify QTC trajectories that represent vehicle pairwise activities. Existing Bi-LSTM networks for vehicle activity analysis are manually designed without considering the optimisation of the whole architecture nor its trainable hyperparameters. Therefore, we adapt Bayesian Optimisation method to search for an optimal Bi-LSTM architecture for classifying QTC trajectories of vehicle interaction. To test the validity of the proposed VNet, four datasets of 8237 trajectories of 9 unique vehicle activities in different traffic scenarios are used. We further compare our VNet model’s performance with the state-of-the-art methods. The results on the combined dataset (accuracy of 98.21%) showed that the proposed method generates light and most robust Bi-LSTM model. We also demonstrate that Activation Map is a promising approach for visualising the Bi-LSTM model decisions for vehicle activity recognition.