Advanced Computer Generated Holography with Deep Learning

Motivation                                                                                     

Digital holography is a discipline of science that measures or reconstructs the wavefield of light by means of interference. The wavefield encodes three-dimensional information, which has many applications, such as interferometry, microscopy, non-destructive testing, and data storage.

This property is especially useful for 3D display technology. Holograms can recreate the wavefield of a 3D object, thereby reproducing all depth cues for all viewpoints. They exhibit no accommodation-vergence conflict, continuous parallax, and accurate shading, unlike alternative 3D display solutions, which tend to have one or more limitations. At high quality, the appearance of an object on a holographic display system becomes indistinguishable from a real one.

Digital holographic displays require pixels with sizes close to optical wavelength, which made fabricating large displays with their corresponding extreme resolutions reaching billions of pixels a hurdle that could not be overcome. Novel high-resolution spatial-light modulator technology currently developed at imec could break through this barrier and bring the advent of 3D display of objects and scenes with unprecedented detail and speed.

Project

However, these displays need to be driven by computing these high-resolution digital holograms at high speeds, making this an important computational challenge. This would require the design and development of specialized computer-generated hologram (CGH) algorithms, modeling the propagation of light through space, accounting for the light-material interactions. Given the nature of numerical diffraction computation, where every hologram pixel can affect every point in space, highly efficient parallelizable software implementations are a must.

A significant part of the research will be dedicated to the investigation and use of deep neural networks with the goal to speed-up calculations and better modelling of non-linear behavior. This will involve applying neural radiance fields (NeRF) for hologram synthetization, temporal compensation of successive hologram frames and efficient approximations to light-material interactions with complex object surfaces.

In this PhD, the candidate will design algorithms and simulators for driving high-end holographic display systems. This will include not only the generation of the wavefields of light using efficient GPU/ASIC architectures, but also the investigation of the optical setups and the use of novel spatial light modulators designed at imec. The main goal is to engineer a 3D holographic display software system with resolutions and realism beyond the current state-of-the-art.