Fundamentals of machine vision

Automobiles may acquire a rich variety of relevant information from image data and its analysis using machine vision techniques. This chapter provides an overview on the principles underlying image formation and image analysis. The perspective projection model is formulated to describe the mapping of the 3D real world onto the 2D image plane with its intrinsic and extrinsic calibration parameters. Image analysis typically begins with the identification of features. These may describe locations of particular local intensity patterns in a single image, such as edges or corners, or may quantify the 2D displacement of corresponding pixels between two images acquired at different time instances or by a multicamera system. Such features can be used to reconstruct the 3D geometry of the real world using stereo vision, motion stereo, or multiview reconstruction. Temporal tracking using Bayesian filters and its variations not only improves accuracy but readily allows for information fusion with data of other sensors. The chapter closes with two application examples. The first addresses object detection and tracking using multiple image features. The second application focuses on intersection understanding illustrating the large potential of high-level scene interpretation through machine vision. The vast majority of creatures able to navigate through space strongly rely on its visual system for this task. In particular, it is well known that humans perceive about 90 % of the information required for driving visually. This allows the conclusion that sufficient information is available in the visual domain. Furthermore, it can be expected that vision-based driver assistance systems exhibit a fairly transparent behavior, e.g., drivers are prepared for cautious driving at decreased velocity under bad weather conditions. Finally, vision allows the perception of a multitude of different information relevant for vehicle control. Much of this information has been designed for visual perception and is hardly recognizable by any other technology, such as lane markings or traffic signs. Thus, it comes as no surprise that cameras are indispensable sensors in driver assistance and automated driving. A camera projects the three-dimensional (3D) real world onto a two-dimensional (2D) imager. Hence, image acquisition reduces the available information by an entire dimension. While 2D information suffices in several tasks, such as the classification of objects, the majority of driver assistance functions requires 3D perception of the vehicle’s environment. This holds particularly true for safety-critical functions including any kind of longitudinal and lateral control. Hence, the scope of image analysis techniques includes 3D reconstruction of the scene geometry and dynamics. Buoyed by the continuing decline in prices of camera and processing hardware on the one hand and the rich information extractable from image sequences on the other, image sensors are used in a constantly growing number of applications. Compared to the seamless perception performed by biological vision systems, machine vision is still in its infancy with application limited to narrowly defined domains. Even with prior domain knowledge, *Email: stiller@kit.edu Handbook of Driver Assistance Systems DOI 10.1007/978-3-319-09840-1_21-1 # Springer International Publishing Switzerland 2015