Vol.69, No.1, 2021, pp.695-711, doi:10.32604/cmc.2021.017574
OPEN ACCESS
ARTICLE
Abnormal Event Correlation and Detection Based on Network Big Data Analysis
  • Zhichao Hu1, Xiangzhan Yu1,*, Jiantao Shi1, Lin Ye1,2
1 School of Cyberspace Science, Harbin Institute of Technology, Harbin, 150001, China
2 Department of Computer and Information Science, Temple University, Philadelphia, 42101, USA
* Corresponding Author: Xiangzhan Yu. Email:
Received 03 February 2021; Accepted 29 March 2021; Issue published 04 June 2021
Abstract
With the continuous development of network technology, various large-scale cyber-attacks continue to emerge. These attacks pose a severe threat to the security of systems, networks, and data. Therefore, how to mine attack patterns from massive data and detect attacks are urgent problems. In this paper, an approach for attack mining and detection is proposed that performs tasks of alarm correlation, false-positive elimination, attack mining, and attack prediction. Based on the idea of CluStream, the proposed approach implements a flow clustering method and a two-step algorithm that guarantees efficient streaming and clustering. The context of an alarm in the attack chain is analyzed and the LightGBM method is used to perform false-positive recognition with high accuracy. To accelerate the search for the filtered alarm sequence data to mine attack patterns, the PrefixSpan algorithm is also updated in the store strategy. The updated PrefixSpan increases the processing efficiency and achieves a better result than the original one in experiments. With Bayesian theory, the transition probability for the sequence pattern string is calculated and the alarm transition probability table constructed to draw the attack graph. Finally, a long-short-term memory network and embedding word-vector method are used to perform online prediction. Results of numerical experiments show that the method proposed in this paper has a strong practical value for attack detection and prediction.
Keywords
Attack scene; false positive; alarm correlation; sequence mining; multi-step attack
Cite This Article
Z. Hu, X. Yu, J. Shi and L. Ye, "Abnormal event correlation and detection based on network big data analysis," Computers, Materials & Continua, vol. 69, no.1, pp. 695–711, 2021.
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.