Evaluation and Reduction of Temporal Issues in Remote VR

Abstract: The aim of this thesis is to study and advance knowledge and technologies surrounding remote rendering of Virtual Reality (VR). In particular regarding temporal aspects such as latency and video stalling events. In remote rendering, rendered content is commonly streamed as video images in network packets from a server to a client. The main purpose is to be able to utilize the processing power available in stationary machines on thin clients that are otherwise limited by weight and size due to their mobility requirements. Achieving this process in real-time with excellent quality is not trivial in interactive VR due to the requirements on low latency and high visual fidelity. The dissertation brings to light the main challenges of the field as well as a set of new proposals and knowledge on the topic. As an introduction to the field, the dissertation begins with a study on 360-video streaming, which is a form of VR but less interactive. Moving on into real-time remote rendering, a commercial wireless VR adapter is studied and a method for monitoring its data traffic is proposed and implemented. The monitoring is able to provide a baseline in terms of video stalling events in a commercial remote-VR product. Moving on, a prototype remote renderer for VR is implemented using a proposed architecture, it is furthermore tested in various network conditions to determine under which conditions such remote rendering may be viable. Having constructed the remote renderer, a study is conducted that shows the effect of headset movements on the resulting video bitrate requirements in remote VR. Furthermore, a method that can reduce the codec image size in remote VR is proposed and its viability is tested with the prototype. Finally, two works are reported, in which human participants are involved, one for studying the subjective effects of video stalls in VR and one for studying the objective effects of hand-controller latency on aiming accuracy in VR.