Ensemble Effort Estimation: An updated and extended systematic literature review

Ensemble Effort Estimation (EEE) techniques combine several individual software estimation methods in order to address the weaknesses of individual methods for prediction tasks. A systematic review published in 2016 analyzed empirical studies on EEE techniques published between 2010 and (January) 2016. The research on EEE has continuously evolved over the past five years (2016–2020), generating new findings that should be aggregated to the existing body of evidence on the subject.The goal of this paper is to update the systematic review from 2016 with new findings from studies published between 2016 (full year) and 2020 (inclusive).To conduct our review update, we followed existing guidelines for updating systematic reviews in software engineering and other fields. We started with an appraisal of the background and methods of the 2016 review, which resulted in the updated review protocol used to conduct our study. We retrieved 3,682 papers using automatic searching techniques, from which we selected 30 papers for data extraction and analysis.Our findings reinforce the results of the previous review in that machine learning is still the technique most common to construct EEE and that the ensemble techniques have outperformed the individual models. We added new evidence showing that there is no clear superiority of an EEE model over the others. Also, we found that ensemble dynamic selection is still little used in Software Effort Estimation (SEE).This review adds new evidence about the use of EEE techniques in software development which reinforces previous findings and also shows research opportunities in constructing more effective EEE. Besides, ensemble dynamic selection appears as a promising area of research which still is underexplored.