Open Access

ARTICLE

SMK-means: An Improved Mini Batch K-means Algorithm Based on Mapreduce with Big Data

Bo Xiao¹, Zhen Wang², Qi Liu^3,*, Xiaodong Liu³

1 Jiangsu Key Laboratory of Atmospheric Environment Monitoring and Pollution Control, Jiangsu Collaborative Innovation Centre of Atmospheric Environment and Equipment Technology, School of Environmental Science and Engineering, Nanjing University of Information Science and Technology, 219 Ningliu Road, Nanjing, 210044, China.
2 School of Computer and Software, Nanjing University of Information Science and Technology, 219 Ningliu Road, Nanjing, 210044, China.
3 School of Computing, Edinburgh Napier University, 10 Colinton Road, Edinburgh, EH10 5DT, UK

* Corresponding Author: Qi Liu. Email: .

Computers, Materials & Continua 2018, 56(3), 365-379. https://doi.org/10.3970/cmc.2018.01830

Abstract

In recent years, the rapid development of big data technology has also been favored by more and more scholars. Massive data storage and calculation problems have also been solved. At the same time, outlier detection problems in mass data have also come along with it. Therefore, more research work has been devoted to the problem of outlier detection in big data. However, the existing available methods have high computation time, the improved algorithm of outlier detection is presented, which has higher performance to detect outlier. In this paper, an improved algorithm is proposed. The SMK-means is a fusion algorithm which is achieved by Mini Batch K-means based on simulated annealing algorithm for anomalous detection of massive household electricity data, which can give the number of clusters and reduce the number of iterations and improve the accuracy of clustering. In this paper, several experiments are performed to compare and analyze multiple performances of the algorithm. Through analysis, we know that the proposed algorithm is superior to the existing algorithms

---

Keywords

---