A risk-based comparison of classification systems
2009
Performancemeasures for families of classification system families that rely upon the analysis of receiver operating
characteristics (ROCs), such as area under the ROC curve (AUC), often fail to fully address the issue of risk,
especially for classification systems involving more than two classes. For the general case, we denote matrices
of class prevalence, costs, and class-conditional probabilities, and assume costs are subjectively fixed, acceptable
estimates for expected values of class-conditional probabilities exist, and mutual independence between a variable
in one such matrix and those of any other matrix. The ROC Risk Functional (RRF), valid for any finite number
of classes, has an associated parameter argument, which specifies a member of a family of classification systems,
and for which there is an associated classification system minimizing Bayes risk over the family. We typify
joint distributions for class prevalences over standard simplices by means of uniform and beta distributions, and
create a family of classification systems using actual data, testing independence assumptions under two such
class prevalence distributions. Examples are given where the risk is minimized under two different sets of costs.
Keywords:
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
35
References
0
Citations
NaN
KQI