Peter Schelkens, group leader of ETRO, an imec research group at Vrije Universiteit Brussel (VUB), & expert in multidimensional signal processing and digital holography
2017 was the year in which artificial intelligence (AI) and new display technologies, such as augmented reality (AR), broke through to the general public. The hype surrounding AI reached a peak and is now part of our day-to-day lives, although sometimes we don’t even realize it. Just think of Siri, the sophisticated assistant with voice recognition from Apple, or the personal recommendations that Google and Netflix dish up to us on a daily basis. There are also lots of devices on the market already that use various new display technologies. Microsoft’s HoloLens, for example, has brought AR into our living rooms. But most of the commercial products with these new technologies are still expensive and need to demonstrate that they are more than just gadgets. But having said that, it has been a promising beginning.
AI and AR are areas in which imec is also involved. In the neurosurgery and orthopedic surgery department at the University Hospital (UZ) Jette, for example, we look into the introduction of AR glasses during surgery, so that surgeons could access additional patient information during procedures. This information is projected directly onto the patient’s body. In the European project TherapyLens this technology is introduced in revalidation therapy. Together with Holst Centre, we also recently presented a novel sensing technology to track eye movement in real time. Imec’s self-learning neuromorphic chip attracted a great deal of attention at the Imec Technology Forum (ITF), not just due to its great number-crunching compute power, but also because of its very low energy consumption. The imec Cell Sorter chip, finally, combines both technologies: holographic image-processing for on-chip cell detection, and AI to classify the cells correctly. The unique power of imec lies in its access to expertise in every possible area of application.
Liesbet Lagae, Program Director Life Science Technologies at imec, with the Cell Sorter chip. In 2014, she received an ERC grant for this research.
The makeable brain
Of great interest for micro-electronics are the recent strides made in neuromorphic computing. Our brains are perfect computers for recognizing patterns and processing information from our senses. Neuromorphic chips meet the challenge of imitating the complexity of this neural network in a so-called ‘deep neural network’, in an energy-efficient way. The ultimate aim is to simulate the way neurons work as is the case with deep learning software. Once we can do that, we will be able to integrate this architecture in a chip for neuromorphic computing in sensors and for AI-applications. Imec is already well on the way with the chip that we presented at ITF. This self-learning chip has the ability to compose music, while consuming only very little power. There is a real pioneering role for us to play in this area.
This may be a beginning, but there is still a long way to go before we have silicon “brains”. The deep neural networks only simulate part of our brain activity. They are based on the visual part of data-processing and model it as a sequential pathway from the eyes to the visual cortex, where the information is processed. In reality, of course, this does not happen sequentially. But only if we understand how the brain works, we will be able to make any further progress. Also, current neuromorphic systems are still very dependent on the algorithms that we develop for them. At the moment they lack human creativity, emotions and our ability to reason when making moral decisions. Although progress is slowly being made, in the mean time we still need to learn how to cope with the limitations of AI. At the same time, we also need to take a close look at both the ethical and social issues involved.
Best of both worlds
With innovations in optics and photonics, the move towards miniaturization in AR is well underway. Thanks to technologies such as freeform optics and optical waveguides it is already possible to produce compact display devices, such as head-mounted displays. More miniaturization is on its way, but real breakthroughs are only possible by integrating hardware and software in the same way as with AI. 2017 was the year in which developments in algorithms and innovations in hardware began to work in each other’s favor.