Virtual reality (VR) experiences that use filmed visual content are much less immersive than computer-generated scenes. The ILLUMINATE project will innovate in the areas of content creation, content representation, interactive streaming, visualization and quality of experience. The goal is to enable the same level of navigational freedom within a confined space for such camera-captured VR content as in computer-generated content.
Immersive VR is on the rise
The use of virtual reality is increasing in applications as diverse as entertainment, education, medicine and more. Driving technologies such as advanced capturing systems and head-mounted displays (HMDs) are also advancing and offer more degrees of freedom (DoF) of movement than other forms of media content. Six degrees of freedom (6DoF) offers the most immersive media experience, since users can view a virtual or captured environment from any location and direction. Computer-generated 6DoF experiences are well-known (cf. 3D gaming), but they are virtually non-existent for camera-captured content (e.g., one can look around in a 360-degree video but moving one’s head does not result in a different view of the scene, which is detrimental to the quality of experience).
Moving from computer-generated to camera-captured content
Offering 6DoF using cinematographic content poses key challenges, such as intelligent capturing setups, massive data rates, and the need for streaming and random-access functionality for streaming delivery and to allow users to interact with the content. The ILLUMINATE consortium will address these challenges in developing a 6DoF environment using camera-captured images in which viewers can move freely and view content from any location and direction.
Developing an end-to-end 6DoF demonstrator
The ILLUMINATE consortium has outlined five primary innovation goals in the development of an end-to-end 6DoF demonstrator that uses photorealistic images:
- To create an immersive 6DoF CGI scene and data set that offers a compelling story to users.
- To develop image processing and streaming solutions necessary to offer the experience.
- To develop an interactive streaming protocol to deliver the experience remotely over a 100Mbps internet connection.
- To investigate new ways of visualizing streamed image-based 6DoF content in real time.
- To develop a model and procedure to measure the quality of experience for users.
Fostering trust, reducing dependency on outside parties
ILLUMINATE will produce 6DoF demonstrators targeting HMDs that offer compelling and immersive experiences. In addition, the project will result in a significant leap of knowledge in the VR field as well as patents, reports, algorithms and software modules that support future developments.
“ILLUMINATE will innovate in the areas of content creation, content representation, interactive streaming, visualization and quality of experience. The goal is to enable the same level of navigational freedom within a confined space for camera-captured content as in computer-generated content.”
Interactive streaming and representation for totally immersive applications.
ILLUMINATE is an imec.icon research project funded by imec and Agentschap Innoveren & Ondernemen.
It started on 01.10.2018 and is set to run until 30.09.2020.Download as pdf
- Theo Technologies
- imec - ETRO - VUB
- imec - IDLab Data Science Lab - UGent
- imec - MICT - UGent
- Project Lead: Patrice Rondao Alface
- Research Lead: Peter Lambert
- Proposal Manager: Niels Van Kets
- Innovation Manager: Steven Van Assche