HELP Video!

The HELP Video! project consortium – consisting of key industry players and leading university research groups – will push the limits of embedded video processing. Filip Louagie, research lead of the HELP Video! project: “The aim of the project is to enable multimedia development companies to stay in control of their video processing and IP roadmap, boosting their competitiveness. It will allow industrial partners to gain expertise in creating and optimizing video processing algorithms, and in tailoring them to address specific and diverse application needs. The project will combine a profound understanding of video processing algorithms with a matching design methodology that works smoothly with a multitude of embedded platforms, and focuses on scalability.” The developments have been tuned towards two real-life industry use cases, each of them having specific needs: eSATURNUS’ video-based platform for smart hospital rooms, and Televic’s video-based solutions for conferencing and education.

The outcomes

1. Real-time, low-complexity video correction algorithms operating in the compressed domain

As part of the HELP Video! project, the partners have developed real-time, low-complexity video stabilization and video rotation correction algorithms that exclusively operate in the compressed domain. The latter will allow the algorithms to run on low-power embedded devices that have limited resources for computation. The algorithms have been successfully implemented in two demonstrators.

Thomas Koninckx, project lead of the HELP Video! project: “eSATURNUS was specifically looking for algorithms allowing to compensate for rotation (e.g. arising from the manipulation of an endoscope inside a patient’s body) or vibration (e.g. arising from holding a handheld medical camera) during surgery. The solutions had to run on our low-power embedded devices that do not have all the resources to do complex calculations. The research partners demonstrated very clearly that using intermediate information like the motion vectors in the compressed domain can help to bootstrap more complex refinement algorithms. This leads to lower-power and more cost-effective video processing algorithms, when compared to a conventional approach.”

2. Reconfigurable and scalable

The proposed solution for video correction has the potential to scale towards the next-generation of resolutions, being ultra-high definition and 4K. This way, it has become a one-fits-all solution that can cover an entire range of performance points. 4K becoming ‘the new normal’ is also the main argument why this optimization is no longer optional for embedded applications.

The project partners also investigated how this video processing IP can easily be redirected towards other applications or other customers, in a reconfigurable way. A tool was developed that helps designers to develop a generic hardware implementation, and that can later automatically extract a different optimized implementation when resource requirements or development platform may change, without manual redesign. In order to evaluate the scalability of the implementation and SW/HW partitioning, a flexible PoC board was developed that matches the existing tool flow.

3. New methodology for enhanced video quality at low latency

The project consortium devised and implemented low-latency video processing algorithms with enhanced quality of experience. Filip Louagie: “For applications in e.g. the medical domain or in video conferencing, the perceived quality of the video is very important. Today, metrics such as peak-signal-to-noise ratio are typically used to judge video quality. However, this is not a good reflection of what we, as a human observer, perceive as a measure for quality. Therefore, the project partners defined a new quality metric, based on extended evaluation tests. This resulted in a new methodology to assess quality, which was used in a feedback loop to drive the decoder and improve video quality at very low latency. Importantly, the solution can be integrated in embedded hardware platforms where available bandwidth is typically restricted or varying.”

Download the leaflet