Multi-scale feature tracking and motion estimation

Abstract: This thesis studies the problems of feature tracking and motion estimation andpresents an application of these concepts to human-computer interaction. Thepresentation is divided into three parts.The first part addresses feature tracking in a multi-scale context. Featuresin an image appear at different scales, and these scales can be expected tochange over time due to the size variations that occur when objects move relativeto the camera. A scheme for feature tracking is presented, which incorporatesa mechanism for automatic scale selection and it is argued thatsuch a mechanism is necessary to handle size variations over time. Experimentsdemonstrate how the proposed scheme is robust to size variations insituations where a traditional fixed scale tracker fails. This leads to extendedfeature trajectories, which are valuable for motion and structure estimation.It is also shown how an object representation suitable for tracking can bebuilt in a conceptually simple way as a multi-scale feature hierarchy withqualitative relations between features at different scales. Experiments illustratethe capability of the proposed hierarchy to handle occlusions and semirigidobjects.The second part of the thesis develops a geometric framework for computingestimates of 3D structure and motion from sparse feature correspondencesin monocular sequences. A tool is presented, called the centered affine trifocaltensor, for motion estimation from three affine views. Moreover, a factorizationapproach is developed which simultaneously handles point and linecorrespondences in multiple affine views. Experiments show the influence ofseveral factors on the accuracy of the structure and motion estimates, includingnoise in the feature localization, perspective effects and the number of featurecorrespondences. This motion estimation framework is also applied tofeature correspondences obtained from the abovementioned feature tracker.The last part integrates the functionalities from the first two parts into apre-prototype system which explores new principles for human-computer interaction.The idea is to transfer 3D orientation to a computer using no otherequipment than the operator’s hand.