Common Structures in Resource Management as Driver for Reinforcement Learning: A Survey and Research Tracks.

In the era of growing digitalization, dynamic resource management becomes one of the critical problems in many application fields where, due to the permanently evolving environment, the trade-off between cost and system performance needs to be continuously adapted. While traditional approaches based on prior system specification or model learning are challenged by the complexity and the dynamicity of these systems, a new paradigm of learning in interaction brings a strong promise - based on the toolset of model-free Reinforcement Learning (RL) and its great success stories in various domains. However, current RL methods still struggle to learn rapidly in incremental, online settings, which is a barrier to deal with many practical problems. To address the slow convergence issue, one approach consists in exploiting the system’s structural properties, instead of acting in full model-free mode. In this paper, we review the existing resource management systems and unveil their common structural properties. We propose a meta-model and discuss the tracks on how these properties can enhance general purpose RL algorithms.