Near/In-sensor AI for Physical AI

Energy-efficient computing systems, including neuromorphic and event-driven architectures, are emerging as compelling solutions for low-latency extreme-edge applications. However, deploying robust and reliable intelligence in real-world Physical AI systems requires more than efficient inference alone: it requires sensing systems that can cope with environmental variability, such as illumination change, turbidity, motion, occlusion, and background clutter, as well as sensor variability, such as drift, ageing, damaged channels, calibration shifts, or changing signal quality. These challenges become especially critical in scenarios involving multiple sensory nodes, hardly accessible platforms, high data rates, and limited output bandwidth, such as automotive, underwater, aerial, space, industrial inspection, and contact-rich robotic applications.

This PhD project will investigate AI-driven near/in-sensor computing as a route toward adaptive, energy-efficient, and self-calibrating sensing systems for Physical AI. Instead of treating sensors as passive devices that continuously stream raw measurements to centralized processors, the project will develop a framework in which sensing, local computation, communication, and control are co-designed as active parts of the perception–action loop. The central research question is how physical agents can decide what to sense, when and at what fidelity to sense it, where information should be processed, and how local decisions should trigger action under changing environmental and task conditions.

The main hypothesis is that adaptive feedback at the sensing interface, combined with hierarchical near/in-sensor processing, can maintain or improve sensing quality and closed-loop task performance while reducing bandwidth usage, memory traffic, reaction latency, and energy consumption compared with open-loop static sensing and centralized inference.

The primary focus will be on three tightly connected building blocks:

•  Adaptive sensing. First, the project will develop task- and context-aware sensing policies that adapt sampling rate, resolution, sensitivity, selected regions, channels, modalities, and event thresholds according to environmental conditions and task state.
• Hierarchical near/in-sensor intelligence. Second, it will investigate lightweight intelligence close to the sensor, enabling local filtering, event detection, feature extraction, compression, confidence estimation, and routing before information is sent to higher-level models. These low-level sensing modules are not intended to operate in isolation: they should transmit compact events, features, uncertainties, alerts, or summaries to higher-level reasoning modules in a hierarchical framework, where off-the-shelf edge hardware can run edge-compatible vision-language, vision-language-action, or world-language-action models such as NVIDIA Cosmos and on-device Cosmos 3 Edge policies for robot perception, planning, and control.
• Closed-loop generalization. The end-goal framework will be evaluated in closed-loop settings to determine whether adaptive sensing and local processing improve robustness under illumination changes, turbidity, motion, occlusion, sensor drift, damaged channels, and unseen task configurations while reducing latency, bandwidth, and energy consumption.

While near/in-sensor computing has already shown promise in extreme-edge biomedical applications, its potential for automotive, underwater, aerial, and robotic Physical AI domains remains largely unexplored. A successful outcome of this research will be a system-level methodology for building intelligent sensing systems that sense selectively, compute near the source, adapt to changing conditions, and support reliable real-time action. The project will therefore contribute algorithms, architectures, benchmarks, and demonstrators for the next generation of adaptive, self-calibrating sensing systems for edge Physical AI.

Our ideal candidate for this position has the following skills: 

• You have a Master’s degree in AI, Computer Science, Engineering or Electronics. 

• You have knowledge about AI engineering and embedded AI or robotics
• You have interest in algorithmic and system design 

• You have good programming skills and are flexible in the use of software and coding tools or libraries (git, pytorch, tensorflow, …) 

• Understanding of the physics and properties of different sensor modalities (lightwaves, radiowaves, etc.) is considered a plus. 

• Experience working with edge AI hardware and software is considered a plus 

• You can plan and carry out your tasks in an independent way. 

• You have strong analytical skills to interpret the obtained research results. 

• You are a responsible, communicative, and flexible person and a team player. 

• Your English is fluent, both speaking and writing