Human Subject Evaluation of Computer-Security Training Recommender

Security breaches in software systems cause massive financial and reputation losses to organizations and put their customers at risk by having their confidential data stolen. Delivering proper software security training to software developers is key to prevent such breaches. Conventional training methods do not take into account the code written by developers. The recommender system described in this paper analyzes developer code for security vulnerabilities and recommends mitigation strategies specific to each developer based on the detected vulnerabilities. The system utilizes a public vulnerability repository as its knowledge base. Such mitigation strategies are platform independent, giving further strength to the utility of the system. This paper extends our previous work and describes a human subject evaluation conducted to determine the usefulness of the system. Our evaluation suggests that this system successfully retrieves relevant training articles from the public vulnerability repository and human subjects found these articles to be suitable for training. The recommender system was found to be as effective as a commercial tool.