Sparsity and shrinkage in predictive densityestimation

We develop new perspectives on the roles of sparsity and shrinkage in predictive density estimation under Kullback-Leibler loss. Our results explain and extend some recently observed information theoretic connections between predictive density estimation and the well-studied normal mean estimation problem. We find new phenomena in sparse minimax prediction which contrast with point estimation theory results and are explained by the new notion of risk diversification. We generalize these new uncertainty sharing ideas to address the nature of optimal shrinkage over unconstrained parameter spaces. Our density estimates can be used to construct competitively optimal probability forecasts and our results give some theoretical support to log-optimality based forecasting techniques used in the fields of weather forecasting, financial investments and sports betting. Motivational stories and examples from the world of sports, stock markets and wind speed profiles are used to suggest the scope of the theory developed in this thesis.