Detection of Intelligent Agent Behaviors Using Markov Chains

We consider the problem of detecting the behavior of intelligent agents operating in stochastic environments. In particular, we focus on a scenario where we are given two models for agent behaviors and we are interested in detecting whether one model appears within the other model. We use Markov chains to represent the behavioral models of the agents and we propose to extract the long-run probabilities as features that can be used to detect if one model is contained in the other. Results show that our approach is capable of detecting known strategies for agents interacting within classical games and to categorize malware behaviors.