Open Access
ARTICLE
An Efficient Way to Parse Logs Automatically for Multiline Events
Mingguang Yu1,2, Xia Zhang1,2,*
1 School of Computer Science and Engineering, Northeastern University, Shenyang, 110169, China
2 Neusoft Corporation, Shenyang, 110179, China
* Corresponding Author: Xia Zhang. Email:
Computer Systems Science and Engineering 2023, 46(3), 2975-2994. https://doi.org/10.32604/csse.2023.037505
Received 06 November 2022; Accepted 06 January 2023; Issue published 03 April 2023
Abstract
In order to obtain information or discover knowledge from system logs, the first step is to perform log parsing, whereby unstructured raw logs can be transformed into a sequence of structured events. Although comprehensive studies on log parsing have been conducted in recent years, most assume that one event object corresponds to a single-line message. However, in a growing number of scenarios, one event object spans multiple lines in the log, for which parsing methods toward single-line events are not applicable. In order to address this problem, this paper proposes an automated log parsing method for multiline events (LPME). LPME finds multiline event objects via iterative scanning, driven by a set of heuristic rules derived from practice. The advantage of LPME is that it proposes a cohesion-based evaluation method for multiline events and a bottom-up search approach that eliminates the process of enumerating all combinations. We analyze the algorithmic complexity of LPME and validate it on four datasets from different backgrounds. Evaluations show that the actual time complexity of LPME parsing for multiline events is close to the constant time, which enables it to handle large-scale sample inputs. On the experimental datasets, the performance of LPME achieves 1.0 for recall, and the precision is generally higher than 0.9, which demonstrates the effectiveness of the proposed LPME.
Keywords
Cite This Article
APA Style
Yu, M., Zhang, X. (2023). An efficient way to parse logs automatically for multiline events. Computer Systems Science and Engineering, 46(3), 2975-2994. https://doi.org/10.32604/csse.2023.037505
Vancouver Style
Yu M, Zhang X. An efficient way to parse logs automatically for multiline events. Comput Syst Sci Eng. 2023;46(3):2975-2994 https://doi.org/10.32604/csse.2023.037505
IEEE Style
M. Yu and X. Zhang, “An Efficient Way to Parse Logs Automatically for Multiline Events,” Comput. Syst. Sci. Eng., vol. 46, no. 3, pp. 2975-2994, 2023. https://doi.org/10.32604/csse.2023.037505
Copyright © 2023 The Author(s). Published by Tech Science Press.
This work is licensed under a Creative
Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.