The maximum entropy method

1. Introduction.- 1.1 What is the Maximum Entropy Method.- 1.2 Definition of Entropy.- 1.3 Rationale of the Maximum Entropy Method.- 1.4 Present and Future Research.- 2. Maximum Entropy Method MEM1 and Its Application in Spectral Analysis.- 2.1 Definition and Expressions of Entropy H1.- 2.1.1 Approach 1.- 2.1.2 Approach 2.- 2.1.3 Discussion.- 2.2 Formulation and Solution.- 2.2.1 Formulation.- 2.2.2 Solution.- 2.2.3 Discussion.- 2.3 Equivalents and Signal Model.- 2.3.1 ACF Extension Subject to the Nonnegativity Constraint.- 2.3.2 Principle of MCE.- 2.3.3 AR Process (Signal Model).- 2.3.4 Bayesian Method.- 2.3.5 Wiener Filter and Approximation Theoretic Approach.- 2.4 Algorithms and Numerical Example (Given ACF).- 2.4.1 Levinson's Recursion for 1-D Noiseless Data.- 2.4.2 Lim-Malik Algorithm for 2-D Noiseless Data.- 2.4.3 Wernecke-D'Addario Algorithm for 2-D Noisy Data.- 2.4.4 Numerical Example.- 2.5 Algorithms and Numerical Example (Given Time Series).- 2.5.1 Burg Algorithm.- 2.5.2 Marple Algorithm.- 2.5.3 Other Fast Algorithms.- 2.5.4 Numerical Example.- 2.6 Order Selection.- 2.6.1 FPE Criterion.- 2.6.2 AIC Criterion.- 2.6.3 Other Criteria.- 2.6.4 Summary.- 3. Maximum Entropy Method MEM2 and Its Application in Image Restoration.- 3.1 Definition and Expressions of Entropy H2.- 3.1.1 MLM.- 3.1.2 Direct Definition Method.- 3.1.3 Discussion.- 3.2 Formulation and Implicit Solution.- 3.2.1 Formulation.- 3.2.2 Implicit Solution.- 3.2.3 Iterative Algorithm.- 3.2.4 Discussion.- 3.3 Explicit Solution.- 3.3.1 Explicit Solution.- 3.3.2 Discussion.- 3.3.3 Examples.- 3.4 Equivalents and Signal Model.- 3.4.1 ACF Extension Subject to the Nonnegativity Constraint.- 3.4.2 Principle of MCE.- 3.4.3 Exponential Process (Signal Model).- 3.4.4 Bayesian Method.- 3.4.5 MLM.- 3.5 R - ? Procedure.- 3.5.1 Statements of the MEM2 Problem.- 3.5.2 R - ? Procedure.- 3.5.3 Example.- 3.6 Algorithms and Numerical Examples (I).- 3.6.1 Frieden Algorithm.- 3.6.2 Gull-Daniell Algorithm.- 3.6.3 Revised GD Algorithm.- 3.6.4 Simplified Newton-Raphson Algorithm.- 3.6.5 Numerical Example.- 3.7 Algorithms and Numerical Examples (II).- 3.7.1 Skilling-Bryan Algorithm.- 3.7.2 Differential Equation Approach.- 3.8 Algorithms and Numerical Examples (III).- 3.8.1 MEM/MemSys5 Package.- 3.8.2 MEM Task in IRAF.- 3.8.3 Restoration with Variable Resolution.- 3.8.4 Numerical Examples.- 3.8.5 Other Algorithms.- 4. Analysis and Comparison of the Maximum Entropy Method.- 4.1 Generalized MEM.- 4.1.1 Formulation of GMEM.- 4.1.2 "Entropy" Expressions in GMEM.- 4.1.3 Properties of GMEM.- 4.2 Expressions of Entropy.- 4.3 Solution's Properties.- 4.3.1 Existence.- 4.3.2 Uniqueness.- 4.3.3 Consistency.- 4.3.4 Statistical Properties.- 4.4 Resolution Enhancement and Data Extension (Experimental Results).- 4.4.1 Examples.- 4.4.2 Resolvability in 1-D Spectral Estimation.- 4.4.3 Resolvability in 2-D Spectral Estimation.- 4.4.4 Super resolut ion and Spectral Line Splitting.- 4.5 Resolution Enhancement and Data Extension (Theoretical Analysis).- 4.5.1 Data Extension in MEM1 and MEM2.- 4.5.2 Resolution Enhancement of MEM1 and MEM2.- 4.5.3 MEM1 and MEM2 Spectra at Low SNR.- 4.5.4 Line Splitting of MEM1.- 4.6 Peak Location and Relative Power Estimation (Experimental Results).- 4.6.1 Peak Location (Given ACF).- 4.6.2 Peak Location (Given Time Series).- 4.6.3 Relative Power Estimation (Given ACF).- 4.6.4 Summary and Comments.- 4.7 Peak Location and Relative Power Estimation (Theoretical Analysis).- 4.7.1 Interference Between Peaks Causes Peak Shifting.- 4.7.2 Explanation of the Peak Shifting in MEMI Spectra.- 4.7.3 Relative Power Estimation for MEMI.- 4.7.4 Summary for Sects. 4.4-4.7.- 4.8 Comments on the Three Schools of Thought on MEM.- 5. Applications of the Maximum Entropy Method in Mathematics and Physics.- 5.1 Solution of Moment Problems.- 5.1.1 General Theory.- 5.1.2 Numerical Methods.- 5.1.3 Noisy Moment Problems.- 5.1.4 Numerical Examples.- 5.2 Solution of Integral Equations.- 5.2.1 Conversion of Integral Equations to Moment Problems.- 5.2.2 Solution of Moment Problems by MEM.- 5.2.3 Numerical Examples.- 5.2.4 Discussion.- 5.3 Solution of Partial Differential Equations.- 5.3.1 Theory.- 5.3.2 Numerical Example.- 5.3.3 Discussion.- 5.4 Predictive Statistical Mechanics.- 5.4.1 Formulation and Solution.- 5.4.2 Useful Formulae.- 5.5 Distributions of Particles Among Energy Levels.- 5.5.1 Boltzmann Distribution.- 5.5.2 Fermi-Dirac and Bose-Einstein Distributions.- 5.6 Classical Statistical Ensembles.- 5.6.1 Microcanonical Ensemble.- 5.6.2 Canonical Ensemble.- 5.6.3 Grand Canonical Ensemble.- 5.7 Quantum Statistical Ensembles.- 5.7.1 Microcanonical Ensemble.- 5.7.2 Canonical Ensemble.- 5.7.3 Grand Canonical Ensemble.- Appendices.- A. Cepstral Analysis.- A.1 Cepstral Analysis System.- A.2 I/O Relationship.- A.3 Properties of the Complex Cepstrum.- A.4 I/O Relationship for Minimum-Phase Input.- B. Image Restoration.- B.1 Image Formation.- B.2 Image Restoration.- B.3 Relationship Between Image Restoration and Spectral Estimation.- References.