Open Access

ARTICLE

DCRL-KG: Distributed Multi-Modal Knowledge Graph Retrieval Platform Based on Collaborative Representation Learning

Leilei Li¹, Yansheng Fu², Dongjie Zhu^2,*, Xiaofang Li³, Yundong Sun², Jianrui Ding², Mingrui Wu², Ning Cao^4,*, Russell Higgs⁵

1 Artificial Intelligence Academy, Wuxi Vocational College of Science and Technology, Wuxi, 214068, China
2 School of Computer Science and Technology, Harbin Institute of Technology, Weihai, 204209, China
3 Department of Mathematics, Harbin Institute of Technology, Weihai, 264209, China
4 College of Information Engineering, Shandong Vocational and Technical University of International Studies, Rizhao, 276826, China
5 School of Mathematics and Statistics, University College Dublin, Dublin, D04 V1W8, Ireland

* Corresponding Authors: Dongjie Zhu. Email: ; Ning Cao. Email:

Intelligent Automation & Soft Computing 2023, 36(3), 3295-3307. https://doi.org/10.32604/iasc.2023.035257

Received 14 August 2022; Accepted 04 November 2022; Issue published 15 March 2023

Abstract

The knowledge graph with relational abundant information has been widely used as the basic data support for the retrieval platforms. Image and text descriptions added to the knowledge graph enrich the node information, which accounts for the advantage of the multi-modal knowledge graph. In the field of cross-modal retrieval platforms, multi-modal knowledge graphs can help to improve retrieval accuracy and efficiency because of the abundant relational information provided by knowledge graphs. The representation learning method is significant to the application of multi-modal knowledge graphs. This paper proposes a distributed collaborative vector retrieval platform (DCRL-KG) using the multimodal knowledge graph VisualSem as the foundation to achieve efficient and high-precision multimodal data retrieval. Firstly, use distributed technology to classify and store the data in the knowledge graph to improve retrieval efficiency. Secondly, this paper uses BabelNet to expand the knowledge graph through multiple filtering processes and increase the diversification of information. Finally, this paper builds a variety of retrieval models to achieve the fusion of retrieval results through linear combination methods to achieve high-precision language retrieval and image retrieval. The paper uses sentence retrieval and image retrieval experiments to prove that the platform can optimize the storage structure of the multi-modal knowledge graph and have good performance in multi-modal space.

---

Keywords

---