Open Access

ARTICLE

JShellDetector: A Java Fileless Webshell Detector Based on Program Analysis

Xuyan Song, Yiting Qin, Xinyao Liu, Baojiang Cui^*, Junsong Fu

School of Cyber Security, Beijing University of Posts and Telecommunications, Beijing, 100876, China

* Corresponding Author: Baojiang Cui. Email:

Computers, Materials & Continua 2023, 75(1), 2061-2078. https://doi.org/10.32604/cmc.2023.034505

Received 19 July 2022; Accepted 22 September 2022; Issue published 06 February 2023

Abstract

Fileless webshell attacks against Java web applications have become more frequent in recent years as Java has gained market share. Webshell is a malicious script that can remotely execute commands and invade servers. It is widely used in attacks against web applications. In contrast to traditional file-based webshells, fileless webshells leave no traces on the hard drive, which means they are invisible to most antivirus software. To make matters worse, although there are some studies on fileless webshells, almost all of them are aimed at web applications developed in the PHP language. The complex mechanism of Java makes researchers face more challenges. To mitigate this attack, this paper proposes JShellDetector, a fileless webshell detector for Java web applications based on program analysis. JShellDetector uses method probes to capture dynamic characteristics of web applications in the Java Virtual Machine (JVM). When a suspicious class tries to call a specific sensitive method, JShellDetector catches it and converts it from the JVM to a bytecode file. Then, JShellDetector builds a Jimple-based control flow graph and processes it using taint analysis techniques. A suspicious class is considered malicious if there is a valid path from sources to sinks. To demonstrate the effectiveness of the proposed approach, we manually collect 35 test cases (all open source on GitHub) and test JShellDetector and only two other Java fileless webshell detection tools. The experimental results show that the detection rate of JShellDetector reaches 77.1%, which is about 11% higher than the other two tools.

---

Keywords

---