Moment information and entropy evaluation for probability densities
2012
Abstract How much information does the sequence of integer moments carry about the corresponding unknown absolutely continuous distribution? We prove that a reliable evaluation of the corresponding Shannon entropy can be done by exploiting some known theoretical results on the entropy convergence, uniquely involving exact moments without solving the underlying moment problem. All the procedure essentially rests on the solution of linear systems, with nearly singular matrices, and hence it requires both calculations in high precision and a pre-conditioning technique. Numerical examples are provided to support the theoretical results.
Keywords:
- Joint entropy
- Mathematical optimization
- Entropy in thermodynamics and information theory
- Maximum entropy probability distribution
- Joint quantum entropy
- Differential entropy
- Min entropy
- Principle of maximum entropy
- Mathematics
- Mathematical analysis
- Maximum entropy thermodynamics
- Entropy power inequality
- Rényi entropy
- Shannon's source coding theorem
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
32
References
21
Citations
NaN
KQI