Asymptotic properties on high-dimensional multivariate regression M-estimation

Abstract In this paper, we work on a general multivariate regression model under the regime that both p , the number of covariates, and n , the number of observations, are large with p ∕ n → κ ( 0 κ ∞ ) . Unlike previous works that focus on a sparse regression vector β , we consider a more interesting situation in which β is composed of two groups: components in group I are large while components in group II are small but possibly not zeros. This study aims to explore the asymptotic behavior of the ridge-regularized high-dimensional multivariate M-estimator of β in group II. By applying the double leave-one-out method, we successfully derive a nonlinear system comprised of two deterministic equations, which characterizes the risk behavior of the M-estimator. The system solution also enables us to yield asymptotic normality for each component of the M-estimator. Moreover, we present rigorous proofs to these approximations that plays a critical role in deriving the system. Finally, we perform experimental validations to demonstrate the performance of the proposed system.