Performance Benchmarking of emerging memories for embedded GPUs

IMEC’s system technology co-optimization (STCO) program focuses on bringing system level requirements closer to core technologies by tackling scaling, memory, power, and cost implications.

AI language translation models have improved from previous state of the art learning mappings between sequences via neural networks and attention mechanisms. Due to more complex and deep layers in models it becomes expensive to train such models with huge parameters. Another problem arises during inference phase which puts pressure on memory bandwidth. Current LLM inference heavily relies on remote compute resources. Smaller models can run on mobile SoC containing embedded GPUs which can leverage benefits of novel memories.

As part of this work, the candidate will be understanding large language models (LLM) and execute such models to realize the deeper insight of memory transections. The data from profiling tools can then be used with analytical/cycle accurate simulator tools to estimate the memory interface requirements. Understanding these parameters from the profiling tools will give an insight into potential for novel memories.