Deep Learning For Model-Based Multi-Object Tracking

Abstract: Multi-object tracking (MOT) is the task of estimating the state of multiple objects based on noisy sensor measurements. MOT is essential in various applications, such as pedestrian monitoring, vehicle tracking, animal behavior analysis, and others. It can be broadly divided into two categories: model-free and model-based, depending on whether accurate and tractable models of the measurement sensor and objects' dynamics are available for methods to use. In model-based MOT, closed-form, Bayes-optimal solutions can be derived for certain model families. These solutions achieve the best possible performance in expectation, but become intractable as the time-horizon increases due to an exponential growth in the number of terms. Approximations are necessary to make these methods feasible, but they result in performance degradation for challenging tracking tasks. The main objective of this thesis is to use deep learning (DL) to address this limitation. The approach taken is to treat MOT as a sequence-to-sequence learning task, devising methods that learn to map measurement sequences to state estimates directly. This perspective frees methods from the need to explicitly consider all possible associations between objects and measurements, thereby side-stepping the intractability of traditional approaches. Furthermore, the available models of the environment are leveraged to generate unlimited synthetic data. This is used to train modern DL architectures that excel in the regime of big data, unlocking their power to reason about complicated and long-term temporal interactions in their inputs. When developing the aforementioned methods, it became necessary to compare their predictions and estimated uncertainties to the state-of-the-art trackers for the model-based setting. To allow for this, another contribution of this thesis is with the paper "An Uncertainty-Aware Performance Measure for Multi-Object Tracking", which proposes the first uncertainty-aware, hyperparameter-free, mathematically principled performance measure for MOT.