Evaluation of Example-Based Measures for Multi-label Classification Performance

This work presents an analysis of six example-based metrics conventionally used to measure the classification performance in multi-label problems. roc curves are used for depicting the different trade-offs generated from each measure. The results show that measures diverge when performances decrease, which demonstrates the importance of selecting the right performance measure regarding to the application at hand. Hamming loss proved to be the wrong choice when sensitive classifiers are wanted, since it does not take into account the imbalance between classes. In turn, geometric mean showed a higher affinity to identify true positives. Additionally, the Matthews correlation coefficient and F-measure showed comparable results in most cases.