Prediction risk for the horseshoe regression

Predictive performance in shrinkage regression suffers from two major difficulties: (i) the amount of relative shrinkage is monotone in the singular values of the design matrix and (ii) the amount of shrinkage does not depend on the response variables. Both of these factors can translate to a poor prediction performance, the risk of which can be estimated unbiasedly using Stein's approach. We show that using a component-specific local shrinkage term that can be learned from the data under a suitable heavy-tailed prior, in combination with a global term providing shrinkage towards zero, can alleviate both these difficulties and consequently, can result in an improved risk for prediction. Demonstrations of improved prediction performance over competing approaches in a simulation study and in a pharmacogenomics data set confirm our theoretical findings.