Cognitive Cellular Networks: A Q-Learning Framework for Self-Organizing Networks

Self-organizing networks (SON) aim at simplifying network management (NM) and optimizing network capital and operational expenditure through automation. Most SON functions (SFs) are rule-based control structures, which evaluate metrics and decide actions based on a set of rules. These rigid structures are, however, very complex to design since rules must be derived for each SF in each possible scenario. In practice, rules only support generic behavior, which cannot respond to the specific scenarios in each network or cell. Moreover, SON coordination becomes very complicated with such varied control structures. In this paper, we propose to advance SON toward cognitive cellular networks (CCN) by adding cognition that enables the SFs to independently learn the required optimal configurations. We propose a generalized Q-learning framework for the CCN functions and show how the framework fits to a general SF control loop. We then apply this framework to two functions on mobility robustness optimization (MRO) and mobility load balancing (MLB). Our results show that the MRO function learns to optimize handover performance while the MLB function learns to distribute instantaneous load among cells.