Open Access

ARTICLE

Efficient User Identity Linkage Based on Aligned Multimodal Features and Temporal Correlation

by Jiaqi Gao¹, Kangfeng Zheng^1,*, Xiujuan Wang², Chunhua Wu¹, Bin Wu²

1 School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing, 100876, China
2 Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

* Corresponding Author: Kangfeng Zheng. Email:

Computers, Materials & Continua 2024, 81(1), 251-270. https://doi.org/10.32604/cmc.2024.055560

Received 01 July 2024; Accepted 13 August 2024; Issue published 15 October 2024

Abstract

User identity linkage (UIL) refers to identifying user accounts belonging to the same identity across different social media platforms. Most of the current research is based on text analysis, which fails to fully explore the rich image resources generated by users, and the existing attempts touch on the multimodal domain, but still face the challenge of semantic differences between text and images. Given this, we investigate the UIL task across different social media platforms based on multimodal user-generated contents (UGCs). We innovatively introduce the efficient user identity linkage via aligned multi-modal features and temporal correlation (EUIL) approach. The method first generates captions for user-posted images with the BLIP model, alleviating the problem of missing textual information. Subsequently, we extract aligned text and image features with the CLIP model, which closely aligns the two modalities and significantly reduces the semantic gap. Accordingly, we construct a set of adapter modules to integrate the multimodal features. Furthermore, we design a temporal weight assignment mechanism to incorporate the temporal dimension of user behavior. We evaluate the proposed scheme on the real-world social dataset TWIN, and the results show that our method reaches 86.39% accuracy, which demonstrates the excellence in handling multimodal data, and provides strong algorithmic support for UIL.

---

Keywords

---