Presentation 4. App vetting systems: Issues and challenges

Increasingly, attention is being paid to security vulnerabilities of mobile apps, and with good reason. Such vulnerabilities, if exploited, could be used to wreak havoc on users by stealing their information or controlling their mobile device. Given the billions of mobile apps in use today, security breaches threaten to occur on a very large scale. Fortunately, efforts have been made to address these vulnerabilities by developing tools to test for and identify them. Today, there are dozens of tools available for testing one or more known vulnerabilities. Unfortunately, however, no single tool has shown the ability to identify all potential vulnerabilities and thus, efforts have also been made to leverage multiple tools in order to provide sufficient vulnerability-detection coverage. These efforts have recently led to the development of systems for managing the testing of apps via multiple tools and for supporting the decision-making process for approving or rejecting apps prior to deployment on a mobile device. We refer to such systems as app vetting systems. App vetting systems comprise a number of actors and components and support a workflow that entails (1) the submission of apps by clients including developers and app stores, (2) the testing and analyses of apps via multiple tools including remote tool services and human analysts, (3) the combining of tool results into a single vulnerability risk assessment and (4) the approval or rejection of apps by a decision maker. App vetting systems are aimed primarily at organizations that have a persistent need to analyze large numbers of apps. Although the general concept of an app vetting system is simple, numerous issues and challenges exist with respect to their design, deployment and usage. To better understand some of these issues and challenges it is best to examine each phase of the app vetting workflow. During the submission phase, for example, issues surrounding intellectual property arise due to the storage and decompiling of apps on the app vetting system as well as other components including remote tool services used by the app vetting system. Here, challenges exist in developing and enforcing policies and usage agreements across all components used by an app vetting system to protect against intellectual property violations. During the testing phase, an app vetting system will process an app using multiple test tools. Perhaps the most obvious challenge with regard to test tools is the difficultly in selecting the most cost-effective set of tools necessary to satisfy the organization's mobile app security requirements (if they exist). In addition, challenges exist in integrating heterogeneous test tools with an app vetting system, underscoring the need for APIs and tool wrappers to be developed. Because an app vetting system must also be able to automatically combine results from multiple, disparate tools into a single vulnerability risk assessment, work is needed to develop standardized reporting formats and risk assessment metrics as well as novel algorithms to support this capability. Another tool-related issue concerns licensing, particularly with respect to commercial tools. Often, commercial tool licenses will restrict the number of apps that can be tested (or the number of lines of code that can be examined) by the tool or provide unrestricted usage at a very high cost. Unfortunately, it is often difficult to know a priori how many apps (or lines of code) will need to be tested so a challenge exists in selecting the most appropriate licensing agreements. A related challenge is deciding who (organization or client) will pay for using a commercial tool. This decision will likely impact the design and implementation of the app vetting system to support the desired business model. Ultimately, an app vetting system is used to facilitate decision-making in approving or rejecting apps. In general, the approval or rejection of an app can be automated in cases where tools detect no or low-risk vulnerabilities or if one or more tools detect a severe or high-risk vulnerability, respectively. It is the cases in between no-/low-risk and severe/high-risk vulnerabilities, however, that complicate the approval or rejection process. Here, several challenges exist including comparing disparate vulnerability reports with the organization's app security requirements and identifying acceptable risk thresholds for approving or rejecting an app. Sometimes, human analysts are employed to provide additional insight in order to facilitate the approval/rejection decision-making process, but this potentially leads to inconsistent results. In addition, protocols and procedures for accepting or rejecting an app must be defined. For example, approving an app may require the app vetting system to digitally sign and return the app to the client. In other cases, rejecting an app might require working with a developer to address identified vulnerabilities; such interaction may be contentious if developers disagree with risk assessments derived by the app vetting system, particularly if those assessments are based on false positive tool results. This presentation discusses the issues and challenges surrounding app vetting systems and provides lessons learned during the development and deployment of an app vetting system for the DARPA TransApps program.