Digital Picture Formats and Representations

This chapter describes the picture formats, processing techniques, and assessment methods that underpin the image and video coding process. It first shows how samples are represented in digital images and video signals, emphasizing the important aspects of color and spatio-temporal representations. It then examines how samples are grouped to form pictures and how pictures are grouped to form sequences of pictures or videos. The mapping of light levels to brightness as perceived by the human visual system is considered, showing how the signal from the camera must be transformed using a non-linear mapping (gamma correction) to ensure best use of the bits available. Next, it introduces the area of color spaces and color space transformation, showing how the separation of luma and chroma information can result in lower overall bit rates. Finally, it introduces the important topic of image and video quality assessment—both as a means of comparing the performance of different compression systems and as a means of controlling encoder modes during compression. Examples are provided throughout.