Video based analysis and visualization of human action

Abstract: Analyzing human motion is important in a number of ways. An athlete constantly needs to evaluate minute details about his or her motion pattern. In physical rehabilitation, the doctor needs to evaluate how well a patient is rehabilitating from injuries. Some systems are being developed in order to identify people only based on their gait. Automatic interpretation of sign language is another area that has received much attention. While all these applications can be considered useful in some sense, the analysis of human motion can also be used for pure entertainment. For example, by filming a sport activity from one view, it is possible to create a 3D reconstruction of this motion, that can be rendered from a view where no camera was originally placed. Such a reconstruction system can be enjoyable for the TV audience. It can also be useful for the computer-game industry. This thesis presents ideas and new methods on how such reconstructions can be obtained. One of the main purposes of this thesis is to identify a number of qualitative constraints that strongly characterizes a certain class of motion. These qualitative constraints provide enough information about the class so that every motion satisfying the constraints will "look nice" and appear, according to a human observer, to belong to the class. Further, the constraints must not be too restrictive; a large variation within the class is necessary. It is shown how such qualitative constraints can be learned automatically from a small set of examples.Another topic that will be addressed concerns analysis of motion in terms of quality assessment as well as classification. It is shown that in many cases, 2D projections of a motion carries almost as much information about the motion as the original 3D representation. It is also shown that single-view reconstruction of 2D data for the purpose of analysis is generally not useful. Using these facts, a prototype of a "virtual coach" that is able to track and analyze image data of human action is developed. Potentials and limitations of such a system are discussed in the the thesis.