Intelligent sensor fusion for low-power multi-modal applications

Many applications benefit from or rely on the use of information of different modalities. A well know application is the automotive domain where cars become more and more intelligent using different kinds of sensors. In future cars, we will find different cameras, radar, lidar technologies and potentially even others. However, the operation and processing costs of these technologies (in terms of power and resource consumption) are high. In this PhD research, we want to explore how to perform this process more efficiently and develop novel techniques that overall decrease power consumption, while achieving equal or better performance than their counterparts. For example, we can also study the impact of deliberate degradation of sensor outputs (to lower resource consumption), while at the same time enriching the data stream with cross modality information (to increase accuracy).

 An additional challenge lays in the applicability of this work to novel sensor technology or concepts that do not fully exist yet. It is often hard to estimate the impact of design choices and novel sensors upfront as it takes long to create workable prototypes. Hence, this topic also focusses on researching efficient ways of abstracting and implementing novel sensor (concepts) in a realistic simulation environment. One of the challenges will be to research techniques to efficiently mimic realistic behaviour and to account for sensor behaviour under different (physical) circumstances (reflection, scattering, blockage, …). This work can, next to the automotive context, also be applied to other applications fields such as robotics or remote sensing use cases. Some relevant publications are: [1] and [2].

We offer you a challenging, stimulating and pleasant research environment, where you can contribute to international research on artificial intelligence with a close link to the underlaying hardware. Within this topic you will be working together with imec hardware, sensor development and university teams on jointly coming up with novel solutions.

Our ideal candidate for this position has the following skills:

• You have a Master’s degree in Computer Science, Informatics, Physics, Engineering or Electronics.
• You have knowledge about artificial intelligence and machine learning
• You have interest in algorithmic and system design
• You have good programming skills and are flexible in the use of software and coding tools or libraries (git, pytorch, tensorflow, …)
• Understanding of the physics and properties of different sensor modalities (lightwaves, radiowaves, etc.) is considered a plus.
• Understanding of hardware paradigms is considered a plus
• You are able to plan and carry out your tasks in an independent way.
• You have strong analytical skills to interpret the obtained research results.
• You are a responsible, communicative and flexible person.
• You are a team player.
• Your English is fluent, both speaking and writing

References:

[1] Gruber, T., Julca-Aguilar, F., Bijelic, M., & Heide, F. (2019). Gated2depth: Real-time dense lidar from gated images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 1506-1516).

[2] M. Dimitrievski, D. Van Hamme and W. Philips, "Perception System based on Cooperative Fusion of Lidar and Cameras," 2022 IEEE Sensors, Dallas, TX, USA, 2022, pp. 1-4, doi: 10.1109/SENSORS52175.2022.9967331.