Open Access

ARTICLE

Enhanced Best Fit Algorithm for Merging Small Files

by Adnan Ali¹, Nada Masood Mirza^1,2, Mohamad Khairi Ishak^1,*

1 School of Electrical and Electronic Engineering, Universiti Sains Malaysia (USM), Nibong Tebal, Pulau Pinang, 14300, Malaysia
2 University College, United Arab Emirates University, Al Ain, UAE

* Corresponding Author: Mohamad Khairi Ishak. Email:

Computer Systems Science and Engineering 2023, 46(1), 913-928. https://doi.org/10.32604/csse.2023.036400

Received 29 September 2022; Accepted 13 November 2022; Issue published 20 January 2023

Abstract

In the Big Data era, numerous sources and environments generate massive amounts of data. This enormous amount of data necessitates specialized advanced tools and procedures that effectively evaluate the information and anticipate decisions for future changes. Hadoop is used to process this kind of data. It is known to handle vast volumes of data more efficiently than tiny amounts, which results in inefficiency in the framework. This study proposes a novel solution to the problem by applying the Enhanced Best Fit Merging algorithm (EBFM) that merges files depending on predefined parameters (type and size). Implementing this algorithm will ensure that the maximum amount of the block size and the generated file size will be in the same range. Its primary goal is to dynamically merge files with the stated criteria based on the file type to guarantee the efficacy and efficiency of the established system. This procedure takes place before the files are available for the Hadoop framework. Additionally, the files generated by the system are named with specific keywords to ensure there is no data loss (file overwrite). The proposed approach guarantees the generation of the fewest possible large files, which reduces the input/output memory burden and corresponds to the Hadoop framework’s effectiveness. The findings show that the proposed technique enhances the framework’s performance by approximately 64% while comparing all other potential performance-impairing variables. The proposed approach is implementable in any environment that uses the Hadoop framework, not limited to smart cities, real-time data analysis, etc.

---

Keywords

---