Virtual reality (VR) experiences that use filmed visual content are much less immersive than computer-generated scenes. The ILLUMINATE project will innovate in the areas of content creation, content representation, interactive streaming, visualization and quality of experience. The goal is to enable the same level of navigational freedom within a confined space for such camera-captured VR content as in computer-generated content.
The use of virtual reality is increasing in applications as diverse as entertainment, education, medicine and more. Driving technologies such as advanced capturing systems and head-mounted displays (HMDs) are also advancing and offer more degrees of freedom (DoF) of movement than other forms of media content. Six degrees of freedom (6DoF) offers the most immersive media experience, since users can view a virtual or captured environment from any location and direction. Computer-generated 6DoF experiences are well-known (cf. 3D gaming), but they are virtually non-existent for camera-captured content (e.g., one can look around in a 360-degree video but moving one’s head does not result in a different view of the scene, which is detrimental to the quality of experience).
Offering 6DoF using cinematographic content poses key challenges, such as intelligent capturing setups, massive data rates, and the need for streaming and random-access functionality for streaming delivery and to allow users to interact with the content. The ILLUMINATE consortium will address these challenges in developing a 6DoF environment using camera-captured images in which viewers can move freely and view content from any location and direction.
The ILLUMINATE consortium has outlined five primary innovation goals in the development of an end-to-end 6DoF demonstrator that uses photorealistic images:
ILLUMINATE will produce 6DoF demonstrators targeting HMDs that offer compelling and immersive experiences. In addition, the project will result in a significant leap of knowledge in the VR field as well as patents, reports, algorithms and software modules that support future developments.
“ILLUMINATE will innovate in the areas of content creation, content representation, interactive streaming, visualization and quality of experience. The goal is to enable the same level of navigational freedom within a confined space for camera-captured content as in computer-generated content.”
Interactive streaming and representation for totally immersive applications.
ILLUMINATE is an imec.icon research project funded by imec and Agentschap Innoveren & Ondernemen.
It started on 01.10.2018 and is set to run until 30.09.2020.