Interpolation Techniques with Applications in Video Coding

Abstract: Recent years have seen the advent of RGB+D video (color+depth video), which enables new applications like free-viewpoint video, 3D and virtual reality. This is however achieved by adding additional data, thus increasing the bitrate. On the other hand, the added geometrical data can be used for more accurate frame prediction, thus decreasing bitrate. Modern encoders use previously decoded frames to predict other ones, meaning they only need to encode the difference. When geometrical data is available, previous frames can instead be projected to the frame that is currently predicted, thus reaching a higher accuracy and a higher compression.In this thesis, different techniques are described and evaluated enabling such a prediction scheme based on projecting from depth-images, so called depth-image based rendering (DIBR). A DIBR method is found that maximizes image quality, in terms of minimizing the differences of the projected frame to the groundtruth of the frame it was projected to, i.e. the frame that is to be predicted. This was achieved by evaluating combinations of both state-of-the-art methods for DIBR as well as own extensions, meant to solve artifacts that were discovered during this work. Furthermore, a real-time version of this DIBR method is derived and, since the deph-maps will be compressed as well, the impact of depth-map compression on the achieved projection quality is evaluated, for different compression methods including novel extensions of existing methods. Finally, spline methods are derived for both geometrical and color interpolation.Although all this was done with a focus on video compression, many of the presented methods are useful for other applications as well, like free-viewpoint video or animation.