Vision and the graphical simulation of spatial structure

One important message of this paper is that vision research is highly relevant to 3D graphics technology and that modern electronic graphical systems can and soon will strongly stimulate the further development of vision science. First an outline is given of ecological optics, the discipline trying to describe the visual information available to an active (mobile, structure-seeking) observer. Whereas ecological optics describes the available visual structure, the observables, psychophysics studies what is actually 'picked-up', observed, by existing observers. Some examples are given of psychophysical studies inspired by ecological optics, made possible by modern computer graphics systems, and relevant to the development of graphics systems 'tuned' to human perceptual capacities. Flat displays simulating 3D layouts and objects have a kind of dual reality. The active observer can easily obtain information on flatness by moving the head or trying to undo motion blurring with pursuit eye movements (in cases where there is motion-blurring, as in films). On the other hand the display will also contain information on 3D-structures and specify depth. A brief discussion is given of some less generally known but powerful depth cues and their possible relation to internal models of solid shape (the visual potential). Finally I inquire into the possibilities of ultimately realising a graphics system providing the active observer with a simulated visual world of a high degree of visual realism. This requires, for example, a high resolution at the center of gaze, but a strong decrease of resolution is allowed with increasing eccentricity in the visual field. In using this fact to save pixels, and thus computations, one has to monitor the direction of the center of gaze and its changes during eye movements. The feedback of this information to the computer need not be more precise than the control systems inside the observer responsible for the movements. More in general, the limits of perception set limits to the required quality of the display system and the limits of organismic movement control set limits to the required quality of the interaction with the machine. These limits in relation to interactive 3D graphics simulation are explored and, where possible, quantified.