Co-Design of AI-Accelerated HPC Architectures for Scientific Computing

The exponential growth in hybrid simulation approaches involving scientific computing and AI demands ever-increasing computational performance, scalability, and energy efficiency. Modern frameworks like NVIDIA PhysicsNemo and JAX-CFD are pushing the boundaries of what is possible in large-scale simulation and AI-accelerated modelling by providing a unified framework. However, scientific simulations and machine learning workloads are very different in nature: e.g, bottlenecked by different parts of system architecture, precision requirements are different, thus, demands heterogeneous system design to achieve optimal system performance or utilization.

This PhD thesis will investigate the interplay between HPC/AI workloads and system architecture, with a focus on identifying, modeling, and overcoming performance bottlenecks. The research will leverage open-source and commercial frameworks (e.g., PhysicsNemo, JAX-CFD) to establish representative workloads, use architectural simulators and analytical modeling to quantify the impact of hardware and software choices.

Possible Research Phases

1. Exploration of HPC/AI Workloads

• Survey and select representative hybrid scientific simulation workloads (possible starting points might include frameworks such as Nvidia's PhysicsNemo and Google's  jax-cfd)
• Identifying the key metrics for optimization (performance/energy/cost...) given the requirement of the workload
• Derive the compute task-graph, analyze the algorithms, key kernels, and data movement patterns to identify critical performance characteristics.

2. Hardware Performance Analysis and Bottleneck Identification

• Benchmark selected workloads on existing hardware platforms (GPUs, CPUs, accelerators).
• Profile hardware utilization, memory bandwidth, interconnects, and energy consumption.
• Identify architectural bottlenecks (e.g., shared/dedicated memory hierarchy, compute throughput, communication overhead, heterogeneous scheduling).

3. Modeling Impact of Software and Hardware Choices

• Develop a clear predictive methodology for performance modeling, using analytical formulations and/or computer architectural simulation tools (e.g., gem5, SST, custom simulators).
• Quantify the impact of software optimizations (e.g., kernel fusion, parallelization strategies) and hardware configurations (e.g., memory types, interconnect topologies) on end-to-end workload performance.

4. Proposal of Novel HW/SW Co-Designs

• Based on insights from modeling and benchmarking, propose innovative hardware and software changes to address identified bottlenecks.
• Evaluate scalability and efficiency of proposed solutions beyond current limitations, using simulation.
• Evaluate the suitability of imec-researched technologies for such workloads.

Required Background

• Master’s degree in electrical engineering, computer science, or related field.
• Solid foundation in processor and system architecture, HPC, and AI concepts.
• Experience with performance analysis, benchmarking, and architectural simulation tools.
• Familiarity with scientific computing frameworks (e.g., PhysicsNemo, JAX-CFD,cuEquivariance) is a plus.
• Interest in hardware/software co-design and optimization.