Mining nested flow of dominant APIs for detecting android malware

Abstract According to the Kaspersky Lab threat report, mobile malware attacks almost doubled in 2018. A study conducted in 2018 by Accenture found malware attacks to be the most expensive to resolve. Android Operating System (OS) is the most dominating platform on mobile devices. This makes Android OS susceptible to malware attacks. We need to develop new techniques and methods to stop this influx of malware attacks. In this paper, we propose a novel technique named DroidDomTree that mines the dominance tree of API (Application programming interface) calls to find similar patterns in Android applications for detecting malware. Dominance is a transitive relation. A dominance tree of API calls highlights a strong flow of path and identifies the nesting structure of APIs and hence emphasizes the importance of certain APIs in an application. It also helps in finding modules and their interaction in an application. If a malicious module is embedded in an application, then this provides strong evidence that the application contains malware. We use these properties and develop a nested model of the dominance tree of API calls and a new scheme for assigning weights to each node in the dominance tree for efficient feature selection. During 10-fold cross-validation, with eight different classifiers using real malware Android applications, DroidDomTree achieved detection rates in the range of 98.1%–99.3% and false positive rates in the range of 1.7%–0.4%.