Outage Analysis in Cognitive Radio Networks with Energy Harvesting and Q-Routing

The present work explores mitigation in end-to-end outage probability in an energy harvesting (EH) enabled cognitive radio networks (CRNs) through reinforcement learning (RL) based multi-hop Q-routing. The operation of CRN follows a frame structure that includes cooperative spectrum sensing (CSS), based on the decision of which, secondary transmit nodes either do EH from the primary user (PU) signal or make an opportunistic data transmission. In either operation, both PU reappearance and disappearance probabilities are considered in mathematical analysis. An optimization problem is formulated that minimizes the outage probability in the CRN under the constraints of primary user interference protection, individual secondary link throughput, and energy causality in EH. The closed-form expressions are derived to find the optimal values of sensing duration, secondary power allocation fraction on SS and data transmission. The present multi-hop CRN studies RL-based Q-routing in different network topologies and analyzes the worst-case runtime complexities. It is observed that the tree structure based network reduces the complexity of Q-routing over the other topologies and provides a significant gain on the outage. The present work improves the CRN outage performance by $11.01$ % and $39.89$ % over the existing techniques.