Model-based coding : extraction, coding, and evaluation of face model parameters

Abstract: This thesis deals with model-based coding of human faces for low bitrate communication. The basicidea of model-based coding is that the appearance and motion of a human face are analysed. By thisanalysis, compact parameters allowing realistic visualization of a synthetic face are extracted. The parameterscan be transmitted at very low bitrates, potentially allowing video face-to-face communicationover narrow channels like GSM or PSTN. Recently, numerous web-based applications for animatedfaces have emerged as well.Although the idea is two decades old by now, there are still several technical problems to be solved,and some of them are treated in this thesis. With the advent of the MPEG-4 standard for face animationwhich provides a standardized way for storing and transmitting animation parameters, model-basedcoding and face animation have been increasingly popular research topics.The first topic treated here is th ecompression of face model parameters. We propose a new compressionscheme, showing that such parameters can be transmitted with reasonable quality at bitrates lowerthan 1 kbit/s.Then, techniques for analysis of images and image sequences containing a human face are treated. Forstatic face images, a method for the extraction of facial features is proposed. For image sequences, amethod for face tracking using the active appearance model search algorithmis proposed and foundto be useful in practical experiments. Methods for achieving real-time performance have been developed,and the robustness and accuracy are analysed.To set the analysis in the context of possible applications, different ways of video synthesis using theextracted parameters are discussed as well.Finally, we take a look at the synthesized face models. Comparing them to real video sequences, wetry to evaluate how well the synthetic face models can convey emotions. Standard data sets and performancemeasures are suggested.