Principles of Supervised Learning

The medical literature has many examples of predictive models based on multiple regression and logistic regression, collectively referred to as supervised machine learning. Both methods actually have fairly stringent “assumptions,” which are often not met. The advent of high-power computing has allowed the development of newer analytical methods for regression and classification which variably compensate for failed assumptions. Some of these methods “shrink” the input variables while others create new, independent variables and then apply more traditional methods. Support vector machines utilize a novel approach that is nonparametric and not model based, avoiding many of the limiting assumptions of traditional methods and also allowing nonlinearity. Quantitative measures of error, when applying these models to a validation dataset, such as mean square error, Akaike Information Criteria, and others, allow numerous models to be constructed and compared, picking the model that “works best.”