Open Access

ARTICLE

Knowledge Reasoning Method Based on Deep Transfer Reinforcement Learning: DTRLpath

by Shiming Lin^1,2,3, Ling Ye², Yijie Zhuang¹, Lingyun Lu^2,*, Shaoqiu Zheng^2,*, Chenxi Huang¹, Ng Yin Kwee⁴

1 School of Informatics, Xiamen University, Xiamen, 361104, China
2 Key Lab of Information System Requirement, Nanjing Research Institute of Electronics Engineering, Nanjing, 210007, China
3 School of Information Engineering, Changji University, Changji, 831100, China
4 School of Mechanical & Aerospace Engineering, Nanyang Technological University, Singapore, 639798, Singapore

* Corresponding Authors: Lingyun Lu. Email: ; Shaoqiu Zheng. Email:

Computers, Materials & Continua 2024, 80(1), 299-317. https://doi.org/10.32604/cmc.2024.051379

Received 04 March 2024; Accepted 03 June 2024; Issue published 18 July 2024

Abstract

In recent years, with the continuous development of deep learning and knowledge graph reasoning methods, more and more researchers have shown great interest in improving knowledge graph reasoning methods by inferring missing facts through reasoning. By searching paths on the knowledge graph and making fact and link predictions based on these paths, deep learning-based Reinforcement Learning (RL) agents can demonstrate good performance and interpretability. Therefore, deep reinforcement learning-based knowledge reasoning methods have rapidly emerged in recent years and have become a hot research topic. However, even in a small and fixed knowledge graph reasoning action space, there are still a large number of invalid actions. It often leads to the interruption of RL agents’ wandering due to the selection of invalid actions, resulting in a significant decrease in the success rate of path mining. In order to improve the success rate of RL agents in the early stages of path search, this article proposes a knowledge reasoning method based on Deep Transfer Reinforcement Learning path (DTRLpath). Before supervised pre-training and retraining, a pre-task of searching for effective actions in a single step is added. The RL agent is first trained in the pre-task to improve its ability to search for effective actions. Then, the trained agent is transferred to the target reasoning task for path search training, which improves its success rate in searching for target task paths. Finally, based on the comparative experimental results on the FB15K-237 and NELL-995 datasets, it can be concluded that the proposed method significantly improves the success rate of path search and outperforms similar methods in most reasoning tasks.

---

Keywords

---