Open Access

ARTICLE

Hash Table Assisted Efficient File Level De-Duplication Scheme in SD-IoV Assisted Sensing Devices

by Ghawar Said¹, Ata Ullah², Anwar Ghani^1,*, Muhammad Azeem¹, Khalid Yahya³, Muhammad Bilal⁴, Sayed Chhattan Shah^5,*

1 Department of Computer Science, International Islamic University, Islamabad, 44000, Pakistan
2 Department of Computer Science, National University of Modern Languages, Islamabad, 44000, Pakistan
3 Department of Electrical and Electronics Engineering, Nisantasi University, Istanbul, 34467, Turkey
4 Department of Computer Engineering, Hankuk University of Foreign Studies, Yongin-si, Gyeonggi-do, 17035, Korea
5 Department of Information and Communication Engineering, Hankuk University of Foreign Studies, Yongin-si, Gyeonggi-do, 17035, Korea

* Corresponding Authors: Anwar Ghani. Email: ; Sayed Chhattan Shah. Email:

Intelligent Automation & Soft Computing 2023, 38(1), 83-99. https://doi.org/10.32604/iasc.2023.036079

Received 16 September 2022; Accepted 02 March 2023; Issue published 26 January 2024

Abstract

The Internet of Things (IoT) and cloud technologies have encouraged massive data storage at central repositories. Software-defined networks (SDN) support the processing of data and restrict the transmission of duplicate values. It is necessary to use a data de-duplication mechanism to reduce communication costs and storage overhead. Existing State of the art schemes suffer from computational overhead due to deterministic or random tree-based tags generation which further increases as the file size grows. This paper presents an efficient file-level de-duplication scheme (EFDS) where the cost of creating tags is reduced by employing a hash table with key-value pair for each block of the file. Further, an algorithm for hash table-based duplicate block identification and storage (HDBIS) is presented based on fingerprints that maintain a linked list of similar duplicate blocks on the same index. Hash tables normally have a consistent time complexity for lookup, generating, and deleting stored data regardless of the input size. The experiential results show that the proposed EFDS scheme performs better compared to its counterparts.

---

Keywords

---