On the Surprising Efficiency of Committee-based Models.
2021
Committee-based models, i.e., model ensembles or cascades, are underexplored in recent work on developing efficient models. While committee-based models themselves are not new, there lacks a systematic understanding of their efficiency in comparison with single models. To fill this gap, we conduct a comprehensive analysis of the efficiency of committee-based models and find that committee-based models provide a complementary paradigm to achieve superior efficiency without tuning the architecture: a simple ensemble or cascade of existing networks can be considerably more efficient than state-of-the-art single models, even outperforming sophisticated neural architecture search methods. The superior efficiency of committee-based models holds true for several tasks, including image classification, video classification, and semantic segmentation, and various architecture families, such as EfficientNet, ResNet, MobileNetV2, and X3D.
Keywords:
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
57
References
4
Citations
NaN
KQI