From JPEG to Bruegel
The move from analog to digital images – which started some three decades ago – was exciting but also brought along a number of challenges. How to store and share the images efficiently? How to support different media, bandwidths, or display qualities? How to annotate and secure images? How to allow for amateur and professional uses alike? And how to make best use of the ever-expanding possibilities of modern image sensors?
These questions were tackled by the Joint Photographic Experts Group (JPEG), the standardization committee that laid down the ways to encode, compress, process, and share digital images. Over the years, their work has been immensely influential and has resulted in a series of successful and widely adopted coding algorithms and file formats such as JPEG and its successor JPEG 2000, and more recently also JPEG XR (Extended Range, with support for e.g. high dynamic range images), JPSearch (standardizing image search and retrieval), JPEG XT (an extension implementing amongst others high dynamic range imagery while guaranteeing backward compatibility with legacy JPEG implementations) and JPEG XS (offering low latency, low complexity near-lossless coding).
Imec’s ETRO research group has had a key role in that effort. Today, Peter Schelkens is chairman of the JPEG Coding, Test & Quality subgroup, the group that is responsible for the specification of the coding standards, validation and quality assessment procedures. In line with that work, ETRO has built an extensive experience in user quality testing, helping to formulate standards, creating test suites, and setting up suitable laboratory infrastructure.
One recent and very visible result is the way Frederik Temmermans and the ETRO lab helped assemble high-resolution images of the paintings of Jan Van Eyck (c. 1390 – 1441 CE) and Pieter Bruegel (1525–1569 CE) and make them available online.
The Bruegel paintings belong to the Kunsthistorisches Museum of Vienna. They were photographed and scientifically analyzed, ready for the high-profile Bruegel exhibition running this winter in Vienna, the first time ever that such a major part of Bruegel’s work – 75% of all surviving works – is brought together. The images were made in the visible, infrared and X-ray domain. They were stitched together from many smaller high-resolution pictures, an endeavor in which ETRO was key. Finally, the images were put online by Universum Digitalis, an ETRO spinoff co-founded by Frederik Temmermans. On the websites, the users may zoom in down to the painting’s individual brush strokes.
Imagining new ways of imaging
The Van Eyck and Bruegel pictures are a culmination of what digital photography and JPEG file formats do so well: capturing scenes in a 2D representation, at a specific instance and with a fixed perspective and focus.
But today, technology is emerging to harvest much more information from a scene, such as depth, time-lapse, angle and/or hyperspectral information. And at the same time, researchers are looking to build displays that can render that extra information. Displays that show a much richer representation of the world, as if we would be looking at the real thing, and even walking around it.
So in 2014 the JPEG committee started work on these new imaging possibilities. The effort was named JPEG Pleno and is to become a framework to represent and exchange new imaging and rendering modalities. It will include new tools for improved compression, and functionality for image manipulation, metadata, image access and interaction, and privacy and security.
JEPG Pleno is primarily concerned with point cloud, light field, and holographic imaging.