Real-time pose estimation for extended reality using 6G radio signals

​

Extended Reality (XR) has been identified as the driving applications for future 6G networks by companies like Nokia [1], Ericsson [2], and Qualcomm [3]. Use cases include education, health care, and gaming. XR requires accurate and real-time pose information to enable seamless mapping of user actions to virtual avatars. Recently, researchers have investigated the use of sub-6 GHz radio signals for pose estimation and the results are very promising. Radio waves at these frequencies offer limited resolution due to low bandwidth. On the other hand, mmWave and sub-THz frequencies (24-300 GHz) not only offer higher data-rates but also higher spatial resolution. This improved spatial resolution can benefit pose estimation in XR applications where accurate motion tracking is important for immersive and realistic experiences.

The aim of this PhD project is to exploit mmWave and sub-THz communication signals for sensing, and more concretely pose estimation. Specifically, the aim is to develop a deep learning methodology that can efficiently (in terms of time and energy consumption) derive information about user poses based on subtle changes in communication signal characteristics (such as CSI and Doppler). By targeting to rely purely on signals used for data communication, we hope to sense without additional transmissions. The challenge lies in generalization towards different a-priori unknown environments and differentiating between multiple users and objects co-existing within the same environment. In this PhD, we will build on and leverage imec’s vast expertise in developing mmWave and sub-THz transceiver hardware.

The successful PhD candidate will be part of a large IMEC team working on the research, implementation and prototyping of future communications and radar systems: experts in digital, analog and mm-wave ASIC design, communications systems, radar systems, joint communication and sensing, processing and machine learning. This is a unique opportunity to develop innovative, multi-disciplinary technology and shape future wireless and sensing networks. You will publish your research in top-level journals and conferences.

[1] https://www.bell-labs.com/research-innovation/what-is-6g/6g-technologies/#gref

[2] https://www.ericsson.com/en/blog/2022/7/future-xr-entertainment

[3] https://www.qualcomm.com/snapdragoninsiders/news-forums/your-guide-to-the-dawn-of-xr