Estimating Human Limb Motion Using Skin Texture and Particle Filtering

Abstract: Estimating human motion is the topic of this thesis. We are interested in accurately estimating the motion of a human body using only video images capturing the subject in motion. Video images from up to two cameras are considered.The first main topic of the thesis is to investigate a new type of input data. This data consists of some sort of texture. This texture can be added to the human body segment under study or it can be the actual texture of the skin.In paper I we investigate if added texture together with the use of a two camera system can provide enough information to make it possible to estimate the knee joint center location. Evaluation is made using a marker based system that is run in parallel to the two camera video system. The results from this investigation show promise for the use of texture. The marker and texture based estimates differ in absolute values but the variations are similar indicating that texture is in fact usable for this purpose.In paper II and III we investigate further the usability in images of skin texture as input for motion estimation. Paper II approaches the problem of estimating human limb motion in the image plane. An image histogram based mutual information criterion is used to decide if an extracted image patch from frame k is a good match to some location in frame k+1. Eval- uation is again performed using a marker based system synchronized to the video stream. The results are very promising for the application of skin texture based motion estimation in 2D. In paper III, basically the same approach is taken as in paper II with the substantial difference that here estimation of three dimensional motion is addressed. Two video cameras are used and the image patch matching is performed both between cameras (inter-camera) in frame k and also in each cameras images (intra-camera) for frame k to k+1. The inter-camera matches yield triangulated three dimensional estimates on the approximate surface of the skin. The intra-camera matches provide a way to connect the three dimensional points between frame k and k+1 The resulting one step three dimensional trajectories are then used to estimate rigid body motion using least squares methods. The results show that there is still some work to be done before this texture based method can be an alternative to the marker based methods.In paper IV the second main topic of the thesis is discussed. Here we present an investigation in using model based techniques for the purpose of estimating human motion. A kinematic model of the thigh and shank segments are built with an anatomic model of the knee. Using this model, the popular particle filter and typical simulated data from the triangulation in paper III, an estimate of the motion variables in the thigh and shank segment can be achieved. This also includes one static model parameter used to describe the knee model. The results from this investigation show good promise for the use of triangulated skin texture as input to such a model based approach.