Privacy-preserving incentive mechanism for multi-leader multi-follower IoT-edge computing market: A reinforcement learning approach

Abstract Computation offloading is a promising solution for resource-limited IoT devices to accomplish computation-intensive tasks. In order to promote the service trading between edge computing service providers and IoT devices, a series of works have explored incentive mechanisms for IoT-edge computing. However, most traditional incentive mechanisms (such as stackelberg game-based approaches) expose privacy of participants. Moreover, the existing reinforcement learning-based incentive mechanisms do not consider the competition among multiple providers, which is not in line with reality. In this paper, taking privacy concern and competition among providers into consideration, we utilize reinforcement learning (RL) technique to design a privacy-preserving incentive mechanism for multiple providers and multiple IoT devices. Specifically, we model the pricing and demand problem of providers and IoT devices as a multi-leader multi-follower stackelberg game, in which the providers work as leaders to determine their prices first, and then the IoT devices determine their demands as followers. We prove the existence and uniqueness of the Nash equilibrium (NE) of this game. Due to privacy concern, providers and IoT devices are unwilling to disclose their own parameters, which makes the derivation of NE becoming a great challenge. To address this problem, a new RL-based pricing mechanism (RLPM) is proposed, which enables providers to learn their optimal pricing strategies without knowing private information of other participants. Finally, numerical simulations are conducted to illustrate the convergence and effectiveness of the RLPM compared with other existing algorithms.