Fully Bayesian Analysis of Relevance Vector Machine Classification with Probit Link Function for Imbalanced Data Problem

The original RVM classification model uses the logistic link function to build the likelihood function making the model hard to be conducted since the posterior of the weight parameter has no closed-form solution. This article proposes the probit link function approach instead of the logistic one for the likelihood function in the RVM classification model, namely PRVM (RVM with the probit link function). We show that the posterior of the weight parameter in PRVM follows the Multivariate Normal distribution and achieves a closed-form solution. A latent variable is needed in our algorithms to simplify the Bayesian computation greatly, and its conditional posterior follows a truncated Normal distribution. Compared with the original RVM classification model, our proposed one is a Fully Bayesian approach, and it has a more efficient computation process. For the prior structure, we first consider the Normal-Gamma independent prior to propose a Generic Bayesian PRVM algorithm. Furthermore, the Fully Bayesian PRVM algorithm with a hierarchical hyperprior structure is proposed, which improves the classification performance, especially in the imbalanced data problem.