Optimal quantization of standard distribution functions

Abstract Quantizer design for minimizing the mean square error has been developed independently by Lloyd and Max. They have tabulated the output and decision levels for Gaussian distribution function based on both uniform and nonuniform quantization. Subsequently this design has been extended to other standard distribution functions such as Rayleigh and Laplacian. Preliminary investigation of error analysis in image processing has shown that image reconstruction based on minimum MSE is not optimal. Quantizer design based on minimization of powers of quantization error other than two (MSE) yields less degraded images. This has led to the necessity of developing quantization tables for most frequently used distribution functions based on mean fourth power error (MFPE) and mean sixth power error (MSPE). The later criteria result in better quantizer design in regions of large luminance changes, contours, edges, etc. leading to subjectively higher quality images. This research focusses on optimal quantizer design for minimizing MFPE and MSPE. Quantization ranges and output levels based on both uniform and nonuniform spacing for Gaussian, Rayleigh and Laplacian distributions are developed. The tables, developed by numerical techniques are based on normalized standard deviation. All the computations are implemented with double precision accuracy (64 bits) with an algorithmic error range of 10 −6 –10 −8 using IBM 370/155 digital computer. Plotting routines are implemented on Tektronix hardware (interactive graphics package)_using DEC-20 digital computer. Other relevant parameters such as distortion ratio (ratio of distortion of uniform to nonuniform quantizers), distortion and entropy as a function of quantization levels are illustrated. This tabular and graphical data will be useful in digital communications such as image processing.