Continuous Optimization of Agentic Workloads on Heterogeneous AI Systems

Design the next-generation distributed runtime to orchestrate autonomous AI agents on heterogeneous compute platforms.

Optimized AI: algorithms, compute system and semiconductor technology

The Optimized AI team at imec explores how to design innovative AI compute systems and how best to implement AI applications on them, considering compute-system architecture, algorithm selection, and semiconductor technologies.

Dynamic workloads on multi-vendor multi-compute edge AI systems

Real-world AI systems, such as autonomous robots, are becoming increasingly capable by combining powerful compute platforms, diverse sensors, and components from multiple vendors. AI hardware is very diverse, adding different types of accelerators, going beyond traditional GPU and CPU architectures. At the same time, the software of these systems increasingly depends on agentic AI architectures, including multi-agent pipelines, LLM-based decision loops, and reactive tool-calling agents, to support real-world decision-making. Furthermore, the requirements of these workloads can change rapidly over time, driven both by the task being executed and by the information captured through the sensors. All of this requires an orchestration runtime that a) can continuously adapt the way workload is scheduled on the system depending on the agents' decisions, the environment and the AI task to executed, and b) a runtime that can take into account the best match between the currently running AI tasks and the available heterogeneous hardware components.[C(1] 

Key Research Challenges & Objectives

This research will investigate tools and methods for the scheduling and placement of AI workloads on distributed edge compute systems. The developed algorithms should be

-       Real-time aware: to take into account end-to-end latency and required time to action for the task

-       Heterogeneous Hardware aware: Formulate placement algorithms that schedule agents and their tool invocations based on hardware topology, CPU/GPU loads, network connectivity, memory pressure, and model weights' physical locality

-       Multi-objective aware: to find the best solution depending on real-time, energy, performance and memory requirements

-       Workload aware: to make sure the scheduling and placement is adapted when the workload changes due to a change in AI task or a change in input data

-       Future proof: to make sure the method and the implementation can be adapted to future workloads and compute hardware

 [C(1]Not for the proposal, but Sourav dropped a student of his at my desk today who would like to pursue a PhD. He studied computer science, so Sourav thinks he may be a better fit for Optimized AI. I chatted a bit with him and he sounds good.