ILLUMINATE

Interactive streaming and representation for totally immersive applications

Virtual reality (VR) experiences that use filmed visual content are much less immersive than computer-generated scenes. The ILLUMINATE project will innovate in the areas of content creation, content representation, interactive streaming, visualization and quality of experience. The goal is to enable the same level of navigational freedom within a confined space for such camera-captured VR content as in computer-generated content.

Immersive VR is on the rise

The use of virtual reality is increasing in applications as diverse as entertainment, education, medicine and more. Driving technologies such as advanced capturing systems and head-mounted displays (HMDs) are also advancing and offer more degrees of freedom (DoF) of movement than other forms of media content. Six degrees of freedom (6DoF) offers the most immersive media experience, since users can view a virtual or captured environment from any location and direction. Computer-generated 6DoF experiences are well-known (cf. 3D gaming), but they are virtually non-existent for camera-captured content (e.g., one can look around in a 360-degree video but moving one’s head does not result in a different view of the scene, which is detrimental to the quality of experience).

Moving from computer-generated to camera-captured content

Offering 6DoF using cinematographic content poses key challenges, such as intelligent capturing setups, massive data rates, and the need for streaming and random-access functionality for streaming delivery and to allow users to interact with the content. The ILLUMINATE consortium will address these challenges in developing a 6DoF environment using camera-captured images in which viewers can move freely and view content from any location and direction.

Developing an end-to-end 6DoF demonstrator

The ILLUMINATE consortium has outlined five primary innovation goals in the development of an end-to-end 6DoF demonstrator that uses photorealistic images:

  1. To create an immersive 6DoF CGI scene and data set that offers a compelling story to users.
  2. To develop image processing and streaming solutions necessary to offer the experience.
  3. To develop an interactive streaming protocol to deliver the experience remotely over a 100Mbps internet connection.
  4. To investigate new ways of visualizing streamed image-based 6DoF content in real time.
  5. To develop a model and procedure to measure the quality of experience for users.

Fostering trust, reducing dependency on outside parties

ILLUMINATE will produce 6DoF demonstrators targeting HMDs that offer compelling and immersive experiences. In addition, the project will result in a significant leap of knowledge in the VR field as well as patents, reports, algorithms and software modules that support future developments.

 

“ILLUMINATE will innovate in the areas of content creation, content representation, interactive streaming, visualization and quality of experience. The goal is to enable the same level of navigational freedom within a confined space for camera-captured content as in computer-generated content.”

ILLUMINATE

Interactive streaming and representation for totally immersive applications.

 

ILLUMINATE is an imec.icon research project funded by imec and Agentschap Innoveren & Ondernemen.

 

It started on 01.10.2018 and is set to run until 30.09.2020.

Download as pdf

Project information

Industry

  • Nokia
  • Theo Technologies
  • VRT

Research

  • imec - ETRO - VUB
  • imec - IDLab Data Science Lab - UGent
  • imec - MICT - UGent

Contact

  • Project Lead: Patrice Rondao Alface
  • Research Lead: Peter Lambert
  • Proposal Manager: Niels Van Kets
  • Innovation Manager: Dirk Hamelinck

This website uses cookies for analytics purposes only without any commercial intent. Find out more here. Our privacy statement can be found here. Some content (videos, iframes, forms,...) on this website will only appear when you have accepted the cookies.

Accept cookies