Multi-hop reasoning over paths in temporal knowledge graphs using reinforcement learning

Abstract Knowledge graphs (KGs) are usually incomplete—many new facts can be inferred from KGs with existing information. In some traditional reasoning methods, temporal information is not taken into consideration, meaning that only triplets (head, relation, tail) are trained. In current dynamic knowledge graphs, it is a challenge to consider the temporal aspects of facts. Recent temporal reasoning methods embed temporal information into low-dimensional spaces. These methods mainly support implicitly reasoning, which means they cannot get the specific reasoning paths. These methods limit the accuracy of reasoning paths and ignore multiple explainable reasoning paths in temporal knowledge graphs (TKGs). To overcome this limitation, we propose a multi-hop reasoning model TPath in this paper. It is a reinforcement learning (RL) framework which can learn multi-hop reasoning paths and continuously adjust the reasoning paths in TKGs. More importantly, we add time vectors in reasoning paths, which further improve the accuracy of reasoning paths. Meanwhile, considering the diversity of temporal reasoning paths, we propose a new reward function. In TPath, the agent employs the Long Short-Term Memory networks (LSTM) to capture current observations from the environment, and it outputs action vectors (relation vectors and time vectors) to the environment through activation functions. Experimentally, our model outperforms other state-of-the-art reasoning methods in several aspects over two public temporal knowledge graph datasets.