Barnum: Detecting Document Malware via Control Flow Anomalies in Hardware Traces

This paper proposes Barnum, an offline control flow attack detection system that applies deep learning on hardware execution traces to model a program’s behavior and detect control flow anomalies. Our implementation analyzes document readers to detect exploits and ABI abuse. Recent work has proposed using deep learning based control flow classification to build more robust and scalable detection systems. These proposals, however, were not evaluated against different kinds of control flow attacks, programs, and adversarial perturbations.