Open Access

ARTICLE

Cross-Modal Hashing Retrieval Based on Deep Residual Network

by Zhiyi Li^1,2,*, Xiaomian Xu², Du Zhang¹, Peng Zhang²

1 Faculty of Information Technology, Macau University of Science and Technology, Macau
2 School of Economics and Management, South China Normal University, Guangzhou, 510006, China

* Corresponding Author: Zhiyi Li. Email:

Computer Systems Science and Engineering 2021, 36(2), 383-405. https://doi.org/10.32604/csse.2021.014563

Received 28 September 2020; Accepted 06 November 2020; Issue published 05 January 2021

Abstract

In the era of big data rich in We Media, the single mode retrieval system has been unable to meet people’s demand for information retrieval. This paper proposes a new solution to the problem of feature extraction and unified mapping of different modes: A Cross-Modal Hashing retrieval algorithm based on Deep Residual Network (CMHR-DRN). The model construction is divided into two stages: The first stage is the feature extraction of different modal data, including the use of Deep Residual Network (DRN) to extract the image features, using the method of combining TF-IDF with the full connection network to extract the text features, and the obtained image and text features used as the input of the second stage. In the second stage, the image and text features are mapped into Hash functions by supervised learning, and the image and text features are mapped to the common binary Hamming space. In the process of mapping, the distance measurement of the original distance measurement and the common feature space are kept unchanged as far as possible to improve the accuracy of Cross-Modal Retrieval. In training the model, adaptive moment estimation (Adam) is used to calculate the adaptive learning rate of each parameter, and the stochastic gradient descent (SGD) is calculated to obtain the minimum loss function. The whole training process is completed on Caffe deep learning framework. Experiments show that the proposed algorithm CMHR-DRN based on Deep Residual Network has better retrieval performance and stronger advantages than other Cross-Modal algorithms CMFH, CMDN and CMSSH.

---

Keywords

---

Citations