Open Access
ARTICLE
Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches
Bao Rong Chang1, Hsiu-Fen Tsai2,*, Yu-Chieh Lin1
1
Department of Computer Science and Information Engineering, National University of Kaohsiung, Kaohsiung, Taiwan
2
Department of Fragrance and Cosmetic Science, Kaohsiung Medical University, Kaohsiung, Taiwan
* Corresponding Author: Hsiu-Fen Tsai. Email:
(This article belongs to this Special Issue: Hybrid Intelligent Methods for Forecasting in Resources and Energy Field)
Computer Modeling in Engineering & Sciences 2023, 134(2), 783-815. https://doi.org/10.32604/cmes.2022.020128
Received 05 November 2021; Accepted 28 March 2022; Issue published 31 August 2022
Abstract
Big data analytics in business intelligence do not provide effective data retrieval methods and job scheduling that
will cause execution inefficiency and low system throughput. This paper aims to enhance the capability of data
retrieval and job scheduling to speed up the operation of big data analytics to overcome inefficiency and low
throughput problems. First, integrating stacked sparse autoencoder and Elasticsearch indexing explored fast data
searching and distributed indexing, which reduces the search scope of the database and dramatically speeds up
data searching. Next, exploiting a deep neural network to predict the approximate execution time of a job gives
prioritized job scheduling based on the shortest job first, which reduces the average waiting time of job execution.
As a result, the proposed data retrieval approach outperforms the previous method using a deep autoencoder and
Solr indexing, significantly improving the speed of data retrieval up to 53% and increasing system throughput by
53%. On the other hand, the proposed job scheduling algorithm defeats both first-in-first-out and memory-sensitive
heterogeneous early finish time scheduling algorithms, effectively shortening the average waiting time up to 5% and
average weighted turnaround time by 19%, respectively.
Keywords
Cite This Article
Chang, B. R., Tsai, H., Lin, Y. (2023). Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches.
CMES-Computer Modeling in Engineering & Sciences, 134(2), 783–815.