Student project: Brain-inspired compressed-inference for event-driven neuromorphic processors
What you will do
naturally recorded streaming signals (like natural audio/video recordings) in
real-time is still a challenging task. Even though the state-of-the-art deep
neural networks can perform some of the tasks as good as humans, the execution
of those algorithms in hardware results in high power consumption. Therefore,
makes it infeasible for many applications (e.g. always-on, battery power
Brain’s architecture and our biological sensors are optimized for processing of such streaming data. Even-though ANNs are inspired by biological brains, we are still far from understanding the function of the brain. We lack knowledge in all domains of data acquisition (sensors), learning and inference mechanisms.
Conventional sensors capture data periodically with fixed sampling time. This simple process results in a high amount of spatio-temporal redundancy. Data compression and compressed-sensing techniques  are developed to optimize data acquisition when input data is sparse. However, it is not straight-forward to process the compressed data directly. In this project, we want to focus on the compressed-inference methods for deep neural networks. Compressed-inference uses methods to reduce number of required operations in execution (inference) of deep neural network inference by exploiting spatio-temporal sparsity.
In the neuromorphic group of imec, we are developing advanced neuromorphic algorithms together with optimized processors and hardware architectures by using new silicon technologies. Currently, through the European TEMPO project , we are investigating the feasibility of executing the deep neural networks (including CNN, RNN, DBN, ...) on an event-driven neuromorphic platform.
This internship project aims to develop a new innovative algorithm to improve the efficiency of previously suggested differential inference algorithms [3,4,5]. These simple algorithms convert the change in intensity of neurons to delta events (spikes) which results in reduction of number of operations when input data is not changing. However, many other techniques used in data compression algorithms are not used and explored for these compressed inferences (e.g. ). Additionally, modification of the training algorithms can result in increasing sparsity and can be explored.
The project duration is of 9 months. This project results in a reusable toolbox which optimizes the number of operations, latency, and accuracy of compressed inference in a reasonable amount of time. The outcome of the project may be published in high impact journals and might as well be patented.
We seek very motivated candidates with a relevant background, with strong programming skills in TensorFlow and/or PyTorch and GPU accelerated programming. Strong knowledge of Deep Neural Networks (CNN, RNN, DBN, RBM, ...) is mandatory. Knowledge of Spiking Neural Networks and neuromorphic computing is a plus.
 A Compressed Overview of Sparsity (https://youtu.be/aHCyHbRIz44)
 Technology and hardware for neuromorphic computing (https://cordis.europa.eu/project/id/826655)
 Asynchronous Spiking Neurons, the natural key to exploit temporal sparsity (https://ieeexplore.ieee.org/abstract/document/8890681)
 Delta networks for optimized recurrent network computation (https://dl.acm.org/doi/abs/10.5555/3305890.3305948)
 CBinfer: Change-Based Inference for Convolutional Neural Networks on Video Data (https://dl.acm.org/doi/abs/10.1145/3131885.3131906)
 EVA2: Exploiting Temporal Redundancy in Live Computer Vision (https://dl.acm.org/doi/10.1109/ISCA.2018.00051)
- Literature review on sparse inference algorithms, compression, compressed-sensing and bio-inspired sensors and processing.
- Finding suitable datasets and applications for sparse video/audio processing.
- Improving the current STOA efficient inference algorithms and optimize for sparsity, latency and accuracy.
- Design a reusable toolbox for the automatic conversion and parameter optimization.
- Thesis writing and documentation in imec The Netherlands.
What we do for you
As one of the world top research institute in the field of electronics, we are the center of the excellence in nano-electronics design for Internet of Things and healthcare solutions. In this internship project, you will be working on the cutting-edge research project, under the supervision of the expert researchers from diverse background. Neuromorphic team of imec includes experts in computational neuroscience, neuromorphic algorithm and hardware. We are involved in many interesting research projects including efficient neuromorphic sensing, state of the art learning algorithms and design of efficient analog/digital/in-memory neuromorphic processors in analog and digital hardware.
Who you are
- You are a M.Sc./Ph.D. student with a relevant background (e.g. electrical/computer engineering).
- You are available for a period of at least 9 months.
- Excellent programming skills in Python (TensorFlow/PyTorch) or MATLAB with GPU accelerated processing.
- Good knowledge of deep neural networks (CNN, RNN, DBN, RBM, ...).
- Knowledge of Spiking Neural Networks and bio-inspired sensors/processors is a plus.
- Entitled to do an internship in the Netherlands.
- Motivated student eager to work independently and expand knowledge in the field.
- Good written and verbal English skills.
Click on ‘apply’ to submit your application. You will then be redirected to e-recruiting.
Please be advised that non-EU/EEA country students that are studying outside of the Netherlands, need to have a work-permit to be able to do an internship in the Netherlands.