Spatio-Temporal Scale-Space Theory

Abstract: This thesis addresses two important topics in developing a systematic space-time geometric approach to real-time, low-level motion vision. The first one concerns measuring of image flow, while the second one focuses on how to find low level features.We argue for studying motion vision in terms of space-time geometry rather than in terms of two (or a few) consecutive image frames. The use of Galilean Geometry and Galilean similarity geometry for this  purpose is motivated and relevant geometrical background is reviewed.In order to measure the visual signal in a way that respects the geometry of the situation and the causal nature of time, we argue that a time causal Galilean spatio-temporal scale-space is needed. The scale-space axioms are chosen so that they generalize popular axiomatizations of spatial scale-space to spatio-temporal  geometries.To be able to derive the scale-space, an infinitesimal framework for scale-spaces that respects a more general class of Lie groups (compared to previous theory) is developed and applied.Perhaps surprisingly, we find that with the chosen axiomatization, a time causal Galilean scale-space is not possible as an evolution process on space and time. However, it is possible on space and memory. We argue that this actually is a more accurate and realistic model of motion vision.While the derivation of the time causal Galilean spatio-temporal scale-spaces requires some exotic mathematics, the end result is as simple as one possibly could hope for and a natural extension of  spatial scale-spaces. The unique infinitesimally generated scale-space is an ordinary diffusion equation with drift on memory and a diffusion equation on space. The drift is used for velocity  adaption, the "velocity adaption" part of Galilean geometry (the Galilean boost) and the temporal scale-space acts as memory.Lifting the restriction of infinitesimally generated scale spaces, we arrive at a new family of scale-spaces. These are generated by a family of fractional differential evolution equations that generalize the ordinary diffusion equation. The same type of evolution equations have recently become popular in research in e.g. financial and physical modeling.The second major topic in this thesis is extraction of features from an image flow. A set of low-level features can be derived by classifying basic Galilean differential invariants. We proceed to derive invariants for two main cases: when the spatio-temporal  gradient cuts the image plane and when it is tangent to the image plane. The former case corresponds to isophote curve motion and the later to creation and disappearance of image structure, a case that is not well captured by the theory of optical flow.The Galilean differential invariants that are derived are equivalent with curl, divergence, deformation and acceleration. These  invariants are normally calculated in terms of optical flow, but here they are instead calculated directly from the the  spatio-temporal image.