Who are Vulnerability Reporters?: A Large-scale Empirical Study on FLOSS

(Background) Software vulnerabilities pose a serious threat to the security of computer systems. Hence, there is a constant race for defenders to find and patch them before attackers are able to exploit them. Measuring different aspects of this process is important in order to better understand it and improve the odds for defenders. (Aims) The human factor of the vulnerability discovery and patching process has received limited attention. Better knowledge of the characteristics of the people and organizations who discover and report security vulnerabilities can considerably enhance our understanding of the process, provide insights regarding the expended effort in vulnerability hunting, contribute to better security metrics, and help guide practical decisions regarding the strategy of projects to attract vulnerability researchers. (Method) In this paper, we present what is, to the best of our knowledge, the first large-scale empirical study on the people and organizations who report vulnerabilities in popular FLOSS projects. Collecting data from a multitude of publicly available sources (NVD, bug-tracking platforms, vendor advisories, source code repositories), we create a dataset of reporter information for 2193 unique reporting entities of 4756 CVEs affecting the Mozilla suite, Apache httpd, the PHP interpreter, and the Linux kernel. We use the dataset to investigate several aspects of the vulnerability discovery process, specifically regarding the distribution of contributions, their temporal characteristics, and the motivations of reporters. (Results) Among our results: around 80% of reports come from 20% of reporters; first time reporters are significant contributors to the yearly total in all 4 projects; productive reporters are specialized w.r.t. the project and vulnerability types; around half of all reports come from reporters acknowledging an affiliation. (Conclusions) Projects depend both on a core of dedicated and productive reporters, and on small contributions from a large number of community reporters. The generalized Pareto principle (the (1 - p)/p law) can be used as a metric for the concentration of contributions in the vulnerability-reporting ecosystem of a project.