Open Access
ARTICLE
A Data-Aware Remote Procedure Call Method for Big Data Systems
Jin Wang1,2, Yaqiong Yang1, Jingyu Zhang1,3,*, Xiaofeng Yu4, Osama Alfarraj5, Amr Tolba5,6
1 School of Computer & Communication Engineering, Changsha University of Science & Technology, Changsha, 410000, China
2 School of Information Science and Engineering, Fujian University of Technology, Fujian, 350000, China
3 School of Systems Engineering, National University of Defense Technology, Changsha, 410000, China
4 School of Business, Nanjing University, Nanjing, 210093, China
5 Computer Science Department, Community College, King Saud University, Riyadh, 11437, Saudi Arabia
6 Mathematics and Computer Science Department, Faculty of Science, Menoufia University, Shebin-El-kom, 32511, Egypt
* Corresponding Author: Jingyu Zhang. Email:
Computer Systems Science and Engineering 2020, 35(6), 523-532. https://doi.org/10.32604/csse.2020.35.523
Abstract
In recent years, big data has been one of the hottest development directions in the information field. With the development of artificial intelligence technology, mobile
smart terminals and high-bandwidth wireless Internet, various types of data are increasing exponentially. Huge amounts of data contain a lot of potential value, therefore
how to effectively store and process data efficiently becomes very important. Hadoop Distributed File System (HDFS) has emerged as a typical representative of dataintensive distributed big data file systems, and it has features such as high fault tolerance, high throughput, and can be deployed on low-cost hardwares. HDFS nodes
communicate with each other to make the big data systems work properly, using the Remote Procedure Call (RPC) mechanism. However, the RPC in HDFS is still not
good enough to work better in terms of network throughput and abnormal response. This paper presents an optimization method to improve the performance of HDFS.
The proposed method dynamically adjusts the RPC configurations between NameNode and DataNodes by sensing the data characters that stored in DataNodes. This
method can effectively reduce the NameNode processing pressure, and improve the network throughput generated by the information transmission between NameNode
and DataNodes. It can also reduce the abnormal response time of the whole system. Finally, the extensive experiments show the effectiveness and efficiency of our
proposed method.
Keywords
Cite This Article
J. Wang, Y. Yang, J. Zhang, X. Yu, O. Alfarraj
et al., "A data-aware remote procedure call method for big data systems,"
Computer Systems Science and Engineering, vol. 35, no.6, pp. 523–532, 2020. https://doi.org/10.32604/csse.2020.35.523
Citations