A knowledge-enhanced contextual bandit approach for personalized recommendation in dynamic domains

Recently, contextual multiarmed bandits (CMAB)-based recommendation has shown promise for applications in dynamic domains such as news or short video recommendation, where items are changing over time. There are two key challenges in constructing a CMAB-based system. First, a user’s historical rating data are usually sparse, which restricts the precise representation of dynamic contextual information. Second, it is difficult to design a personalized recommendation policy because of the high diversity of users’ selection behaviors over time. Therefore, existing CMAB-based recommendation methods mainly focus on the improvement of recommendation accuracy. This leads to difficulty in capturing the dynamic personalized preferences of users. To overcome these limitations, we design a novel knowledge-enhanced contextual-bandit approach that combines the dynamic recommendation strategy of the bandit algorithm and the knowledge representation of the knowledge graph. We argue that a user’s selection history for items reflects his/her favorite attributes, which will be repeated in the future. Accordingly we propose the knowledge-enhanced CMAB model, which leverages the contextual information of both users and items obtained by knowledge graph-based embedding. We validate the performance of the proposed model using two public datasets. Experimental results show that our method outperforms state-of-the-art methods in terms of both recommendation accuracy and diversity.