A Survey on Stock Market Manipulation Detectors Using Artificial Intelligence

1  Introduction

A well-regulated financial market is an important foundation for any country that wishes to have a thriving economy. It facilitates effective flows of investments in various forms, including equities, currency exchanges, and bonds between potential sellers and buyers [1]. Some of the roles played by the financial market are to provide liquidity to assets, direct savings to more productive uses, and reduce the cost of transactions. In general, financial markets can be classified into primary and secondary markets [2]. The first category is the market in which securities are created, while the second category is the market in which securities are traded. There are a several types of financial markets, but among them, the stock market is considered the most important market that reflects the health of a country’s economy. Put simply, the stock market is a marketplace in which investors can sell and buy the shares of publicly listed companies and other securities. Even in a small country like Malaysia, there are almost 1000 listed companies on the Bursa Malaysia exchange, which processes millions of transactions every year [3]. Due to the large trading volume, all the buying and selling activities of shares must be regulated to facilitate fair transactions.

A fair stock exchange is very important in order to attract foreign investment that will energize a local economy [4,5]. Apart from that, a fair stock exchange will also ensure the sustainability of the marketplace in which traders will keep buying and selling company shares. It is also the reason the stock exchange of any country embroiled in war will not command the trust of foreign investors. Therefore, it is the role of regulators to come out with a set of rules and impose them effectively so that all transactions are valid and fair. However, there will always be a certain group of traders, who will try to artificially inflate or deflate a stock price for their benefit [6]. These rogue traders aim to disrupt the efficiency of the marketplace by manipulating the fairness of stock prices. These manipulation activities can be executed by a single investor or a group of investors in a collusion scheme. In general, temporary distortions of a stock price will bring profit to manipulators, as they will sell the shares at a relatively higher price or buy shares at a relatively lower price than the fair value. A good systematic review of stock market manipulation can be found in [7], whereby our paper focuses more on the methods and algorithms that are divided into conventional machine learning and deep learning approaches.

In the digital era of stock trading, the size of trading volume per day can exceed more than 1 million transactions, particularly for large stock exchanges such as New York Stock Exchange, London Stock Exchange, and Shanghai Stock Exchange [8]. Therefore, regulators need to be prepared to monitor these transactions effectively and efficiently to spot any irregularities or anomalies that may be caused by manipulation activities. However, due to the large number of transactions, it is not effective to use human operators to skim through all the transactions manually. It is not an effective solution, whereby humans are prone to fatigue when dealing with repetitive tasks, especially in the case of long working hours [9]. Apart from that, the efficiency will also be low and hence motivates the regulators to adopt automation processes. In the early days, multivariate statistical analysis and conventional machine learning methods were used to identify manipulation patterns. Normally, these approaches use a shallow machine learning procedure, in which much of the process requires good pre-processing and feature selection methods [10]. In these crucial steps, a cleaned set of inputs is passed through a popular machine learning classifier or clustering method, depending on the final decision-making task. Apart from that, if the input feature size is too large, it needs to be trimmed through dimensionality reduction methods. In general, detectors that are based on artificial intelligence (AI), specifically deep learning technology have surpassed the average human capability in many fields, including disaster prediction [11], intelligent disease diagnosis systems [12], agriculture yield estimation [13], security alert system [14] and many more. Usually, the deep learning approach, which has gained more traction recently, involves a deeper and more complex network devised to extract the manipulation patterns. This approach usually employs an end-to-end procedure, in which feature selection and decision-making components are linked together, and trained to obtain the optimal set of parameters [15]. A large set of input features is also possible, as such the network performance will typically improve if more training data is supplied. This advancement has also been motivated by the adoption of graphics processing units that enable parallel computation to be performed effectively in processing deep networks.

This review paper is organized into six sections that start with a brief introductory section. Section 2 then provides brief taxonomy of the manipulation cases, followed by Sections 3 and 4 that discuss in detail the previous works on intelligent stock market manipulation detectors using supervised and unsupervised conventional machine learning approaches, respectively. Section 5 discusses the state-of-the-art deep learning approach that mainly applied a recurrent-based network to identify the manipulation cases. Finally, a concise summary of the review findings and a few future work suggestions are provided in the last section. The general flow of this paper is shown in Fig. 1.

Figure 1: The overall paper structure

2  Stock Market Manipulation: The Taxonomy and Categories

In general, stock market manipulation is defined as an illegal practice that attempts to raise or lower stock prices by masking it as an active trade [16,17]. Nevertheless, there is no universally accepted definition of stock market manipulation due to the unclear legal meanings and inconsistent terms that are used in economics and finance literature. For example, Goldwasser [18] stated that the US Securities and Exchange Act 1934, the Australian Corporations Law, and the European Union (EU) Market Abuse Regulation (MAR) generally forbid market manipulation by providing numerous provisions to prevent it. However, these regulations failed to clearly describe stock market manipulation. He further argued that the role of determining the true definition of manipulation cases has largely shifted towards the definition of a country’s court, which is based on the nature of each case. As per the statutory law of the U.S, it is illegal to sell or purchase stocks via a deceptive or manipulative contrivance or tool.

The Australian statutory law (Corporations Act 2011-Sect 1041A) on the other hand, forbids transactions that include or possibly include the impact of producing fake security prices. Whereas the statutory law of the EU (Regulation [EU] No 596/2014 on market abuse) defines market manipulation as “Entering into a transaction or behavior that gives or is likely to give false or misleading signals as to the supply of/demand for/price of a financial instrument; secures or is likely to secure the price of a financial instrument at an abnormal or artificial level”. In Malaysia, stock market manipulation is defined under Section 176 of the Capital Markets and Services Act (CMSA) 2007 as “the act of transacting in the securities of a company that will have or is likely to have the effect of raising or lowering or maintaining the price of the company’s securities on a stock market, with the intention of inducing other persons to purchase or subscribe for the company’s securities”.

Since statute laws do not consistently describe stock market manipulation, it is also essential to investigate the definition contained in economics and laws literature. By departing from conventional legal interpretations, Fischel et al. [19] claimed that stock market manipulation is an ambiguous term that shapes the foundation of criminal proceedings. They indicated that the absence of an objective definition of manipulation suggests that it can only be described as a deceptive effort to change the prices of stocks. Therefore, they stated that trading should never be forbidden as manipulative; instead, only fictitious trading such as trades where the seller and buyer are the same person, and therefore, dispersing misleading information should be treated as fraudulent. Their rationale was that: 1) it is unlikely that a strictly trade-based manipulation case would succeed and 2) rules prohibiting manipulation may deter legitimate trading.

Contrary to Fischel et al. [19], Thel [20] claimed that it is easy to perform manipulation cases. According to him, any transaction that is carried out with the intention of decreasing or increasing a security’s price can be defined as manipulation. This assumes that manipulators may sometimes control prices through trading and thereby earn profits through pre-existing price-dependent contracts and by forcing other stock market players to execute exchanges at the manipulated market price. Another study by Jarrow [21] defines manipulation as a trading technique that produces real positive wealth at a zero level of risk. Cherian et al. [22], on the other hand, describe manipulation as someone trading in such a way that affects a stock price to their benefit. From a different perspective, Kyle et al. [23] suggested that trading methods can only be categorized as unlawful market manipulations if they impair market efficiency by reducing liquidity and diminishing price accuracies. According to them, a trading technique is not explicitly harmful to society until both requirements are met; thus, it must not be forbidden.

While exploring stock market manipulation in both developed and emerging stock markets, Baker et al. [24] explained that stock market manipulation involves various aspects, such as trading frequency, trading volume, order size, and stock price fraction magnitude. Their argument is further supported by Fox et al. [25] who categorized stock market manipulation into two broad categories: open market or misstatement manipulations. Open market manipulation occurs via fictitious trading, while misstatement manipulation is an information-based manipulation whereby a trader creates a materially false statement to push down a stock price. Both types of manipulations can drive the average sale price to exceed the average purchase price.

Based on the above discussion, this review proposes that the most comprehensive definition of stock market manipulation, covering all of its aspects, is based on the definition of Allen et al. [26]. They divided stock price manipulations into three broad categories namely 1) action-based, 2) information-based and 3) trade-based manipulations, which will be discussed in detail in the following subsections. According to them, action-based manipulation is a situation in which the deliberate action of any party in a stock market affects the value of stocks. Information-based manipulation, on the other hand, involves the spread of rumors or false information in a market to manipulate stock prices. Finally, trade-based or trading-induced manipulation involves artificial increments or decrements in stock prices through fictitious trading by stock market intermediaries. Fig. 2 shows the main categories of stock market manipulation, especially for trade-based schemes.

Figure 2: The main categories of stock market manipulation schemes

2.1 Action-based Manipulation Scheme

Action-based manipulation occurs when someone acts or takes any action with the intention of influencing the movement of stock prices. For example, a company’s director may close a company branch to drive down the stock price. Although action-based manipulation has not been explicitly examined in the prior literature, it can also be treated as a form of information-based manipulation if the manipulator’s actions produce a fake impression, which is identical to fake information. Therefore, many of the results surrounding information-based manipulations are also applicable to action-based manipulations.

2.2 Information-based Manipulation Scheme

As discussed earlier, information-based manipulation refers to any type of manipulation through the spreading of deceitful information or statements with the intention of manipulating stock prices. Many studies have explored this type of manipulation. For example, Vila [27] employed game theory to create a simple case in which manipulators short-sell a stock by releasing false and harmful stock-related information and then recovered their position when the price was low. Similarly, Bagnoli et al. [28] examined a framework in which manipulators announced a fake takeover bid to push up a stock price. The effectiveness of both strategies depends on the integrity of the information disseminated by manipulators. If market participants recognize fake information in repeated trials of such games, then the manipulators will soon be debunked, and their manipulation technique will no longer be lucrative.

To address the credibility issue in recurring games, Benabou et al. [29] and Bommel [30] proposed a framework that incorporates inaccurate information to impact a share price. In Benabou et al. model [29], noisy insider information limits the capacity of investors to confirm the validity of available information. Thus, manipulators, such as company executives with insider information, stock analysts, or journalists may influence a market price over a prolonged span of time without sacrificing credibility by combining true facts with false information in a piece of evidence that they issue. From a different perspective, Bommel [30] developed a framework in which informed investors regulate stock prices by disseminating inaccurate rumors. Rumors are insightful in an equilibrium state; thus, leading rational profit-maximizing players to trade based on them. Since rumors are unreliable, stock markets sometimes overshoot, which enables informed manipulators to make gains not only from their own superior information but also by trading based on the overshoot prices of the market. Likewise, Eren et al. [31] applied Bommel’s framework to demonstrate that a hype-and-dump form of exploitation enhances market trading volume and depth but reduces the efficiency of markets.

2.3 Trade-based Manipulation Scheme

Trade-based or trading-induced manipulation is another type of manipulation that is more difficult to detect, and it is the focus of this review. Trade-based manipulation involves an artificial increment or decrement of stock prices through fictitious trading by traders or stock market intermediaries [32]. The most popular form of this manipulation scheme is “pump & dump” which is mainly prevalent in emerging stock markets. In a pump & dump manipulation scheme, manipulators take a long position in stocks, inflate stock prices by increasing the liquidity of traded stocks, and finally reverse their position at an inflated price to gain huge profits. The pumping period of stocks can take a few hours to many years, during which several manipulators can adopt additional techniques, such as wash sales and pooling of stocks [33].

Another form of trade-induced manipulation is dump-and-pump, also known as “bear raids”. This involves manipulators selling or short-selling stocks and driving prices downward by forcing others to sell and finally cover their positions again at depressed prices. A common characteristic of pump & dump manipulation is that manipulators profit directly from market manipulation by exploiting investors who purchase stocks at higher prices and later sell them at reduced prices. Market manipulation is a worldwide phenomenon incurred in both developed as well as developing markets. Aggarwal et al. [34] examined the pump & dump cases of manipulation collected from the US Securities and Exchange Commission (SEC) litigation releases using daily data. They highlighted 142 cases of manipulation and acquired the data of 51 manipulation cases over the period from 1990 to 2001. From their observations, the minimum manipulation duration is 2 days, the median duration is 202 days, and the maximum duration is 373 days, showing the varying characteristics of pump & dump manipulations. They reported that stock prices generally rise during the manipulation period and decrease during the post-manipulation period, which also increases stock price volatility. Their sample manipulation cases involved mostly illiquid stocks, while most of the manipulations were conducted by informed insider traders such as substantial shareholders, management, and brokers or market-makers.

Comerton-Forde et al. [35] conducted a study based on a sample of 184 cases of closing stock price manipulation that were highlighted by the SEC over the period between 1997 and 2009. Based on their sample, the manipulation was mainly conducted by brokers, a company’s top management, substantial shareholders, and fund managers. They revealed that cases of closing stock price manipulations were primarily linked to large hikes in day-end profits, large trading volumes, wider spreads, and subsequent reversals of returns. On the basis of these results, they built a closing stock price manipulation index that can examine manipulations in multiple markets at various times of interest for which prosecution data is not available.

Leuz et al. [36] examined 421 cases of the pump & dump manipulations over the period 2002 to 2015 and provided evidence on the participation rate, size of investment, average losses, and characteristics of those who took part in these schemes. Their results suggest that the participation of individual investors is quite common in these manipulation schemes, which involve an average loss of about 30% for such investors. Moreover, the study highlighted several categories of investors who intentionally take part in these manipulation schemes based on their superior information. The existing literature also shows that emerging markets are particularly vulnerable to manipulative tactics. There is an abundance of manipulation cases in Asian stock markets. The pump & dump strategy is considered a common manipulation tactic in the Indian stock market. But, the absence of a strong regulatory framework in such markets indicates that most of these manipulative tactics are hard to capture, and thus, there is relatively limited empirical evidence about them. The impact of pump & dump manipulations has been captured in rare empirical research by Bommel [30]. They revealed that in the Pakistani stock market, when brokers traded on their own behalf, they collected about 50% to 90% higher earnings compared to retail investors. They further asserted that brokers had achieved these higher earning differentials through “pump & dump” manipulative schemes.

Huang et al. [37] observed that due to insufficient regulation, ineffective monitoring, and institutional flaws, the Taiwanese equity market was plagued by market manipulation, insider trading, IPO fraud, and other stock irregularities. This paved the way for the trading of power money (trust money) and attracted prospective criminals, who then defrauded most retail investors. In a similar vein, Wirama et al. [38] examined incidents of market manipulation in the Indonesian stock market. They revealed that traders purposefully spread rumors about the market to draw in customers. Market makers can affect stock prices by posting unfounded information if they have more cash and greater access to media and information. Initially, the rumors create significant anomalous returns for the manipulators, but over time, the effectiveness of manipulation falls. Similarly, Kan [39] highlighted numerous incidents of stock market manipulation in the Malaysian equity market and narrated the salient features of these manipulations. The study analyzed 33 cases of stock market manipulations between 2005 and 2015 and concluded that these manipulations were mainly conducted by those in privileged positions with easy access to information. Moreover, greed, dishonesty, self-interest, and ethical failure were found to be the prominent reasons for stock market manipulations. Overall, price manipulation by several stock market intermediaries is detrimental to the increase in market capitalization as individual investors are reluctant to invest in these markets.

2.4 Early Experimental Researches on Stock Market Manipulation

Through an unconventional field experiment, Hanson et al. [40] carried out a seminal laboratory experiment on stock price manipulations. It involved twelve investors who transacted cash and stock in a digital limit orders stock exchange. Half of the investors were offered cash rewards during the manipulation experiment to influence stock prices during trading. Their key result suggests that the manipulators were unable to distort price accuracy, as their actions were counteracted by other traders. In an identical experimental market study, Comerton-Forde et al. [41] demonstrated that if investors are granted rewards to manipulate closing prices, both the liquidity and price accuracy of a market will be affected. These findings indicate that incentives are low likelihood key determinants in instigating manipulated trading.

It is important to investigate how market manipulation affects liquidity and market efficiency. While numerous prior studies have investigated prosecuted manipulation instances to evaluate the ex-post impact of market manipulation, the findings conclude that it is hard and challenging to predict the likelihood of someone manipulating a stock market. Will the likelihood or possibility of manipulation deter the participation of investors in stock markets, thus making a market less efficient and less liquid? Or does a stock market become efficient due to investors becoming better informed? The challenge arises from the absence of a counterfactual manipulation-free market in the real world to serve as the comparison standard. Many of the drawbacks of empirical approaches can be solved by experimental research because an experimenter can monitor and regulate information, as well as reward and punish fundamental asset values. At the same time, an experimenter is also able to resolve the problems created by the inadequate rate of manipulation detection and identification. In this case, the key advantage of this opportunity is the ability to design the experimental environment in a reasonably realistic manner such that the findings are scientifically relevant and provide real markets with meaningful insights. Despite this, experimental research is more successful compared to empirical research, but it is underutilized in enhancing knowledge about market manipulation.

In summary, the concept of market manipulation encompasses a wide range of frequently varying tactics that typically include the intention of producing a false appearance of a security’s price or trading operation. As technology becomes more advanced, experimental research to detect manipulation cases can now utilize more advanced techniques such as complex data analytic methods [42,43]. However, advanced data analytics and artificial intelligence-based strategies for trading behavior analysis have not yet been extensively reviewed. Therefore, this review will fill the gap in the literature and attempt to analyze the state-of-the-art techniques for detecting manipulation cases through conventional and deep machine learning approaches. Fig. 3 shows the standard algorithm workflow for both conventional machine learning and deep learning approaches.

Figure 3: The standard algorithm workflow for both conventional and deep learning approaches

3  Artificial Intelligence in Finance

Many financial firms with more than 5000 employees have implemented basic AI, mainly to predict revenues and detect anomalous transactions [44]. Some of the financial sectors such as online banking, market analysis, property management, corporate finance, and insurance have used advanced AI in managing their operations. In [45], Rong et al. analyzed product reviews on an e-commerce platform using sentiment analysis. The authors used the reviewed texts as input features, which were then converted into word vectors before passing them to a compact CNN model. The model consisted of one convolutional layer and three fully connected layers. A flatten operator was used to map 2-dimensional feature maps into 1-dimensional feature vectors for classification purposes. Using a similar CNN model, Zhu et al. [46] analyzed consumer credit scores for the Chinese financial market. The time-series input was transformed into a 2-dimensional matrix, which will be the input to two convolutional layers networks. The classification layers derived flattened vector input from the convolutional layers, which are embedded with DropOut units to reduce the likelihood of overfitting. Muranda et al. [47] faced a challenge in balancing the amount of data among the classes for their fraudulent insurance claim detector. They aimed to detect fraudulent claims, which in general have a low likelihood of occurring, and hence, they need to augment the training data to better fit the proposed model. They implemented two approaches of over-sampling, which are synthetic minority oversampling and adaptive synthetic techniques. They found out that the proposed sampling methods managed to train the neural network model with 100% accuracy. Generative adversarial network (GAN) was also used in financial-based prediction, especially for forecasting the price. The research by Hsieh et al. [48] stated that they used long short-term memory (LSTM), arranged in GAN format to predict the sale of houses in the USA. They used basic house information that includes floor areas, number of rooms, age of the house, and localization as the input to single- and two-layer LSTM networks.

Rather than using house specifications as the only input, the work in [49] utilized additional text descriptions of the houses to predict the best rental price. They explored the basic unit of LSTM, bi-directional LSTM, and self-attention network in processing all input data. The house location is clustered first to group them into general classes, which will be combined with the text-based descriptors to determine the optimal rental price. A gated recurrent unit (GRU) was also used to map the time-series patterns for the finance applications, instead of LSTM. In [50], GRU becomes the backbone used to optimize the policy for financial portfolio management. The stacked GRU was modeled using deep reinforcement learning by providing a set of rewards and punishments that optimized the proposed portfolio goals. The network policy was trained based on Q-learning, which produced good results for both dynamic and policy gradient techniques. Apart from that, deep reinforcement learning was also applied to commodity and foreign exchange markets. Usha et al. [51] implemented the model for forecasting the price of gold, silver, and oil, as well as predicting the exchange rate for US dollars. The model depends only on price and date information to perform short-term forecasting, as more information is required for long-term forecasting. The agents were trained for 800 epochs with a windowing size of 10 overlapping sets of data using a low learning rate value of 0.001. The work in [52] also discussed exchange rate prediction using the LSTM network to model the input-output relationship rather than reinforcement learning. They used a more comprehensive set of input features that include exchange date, time, open, close, volume, low and high prices. The proposed model of two LSTM stacks with two fully connected layers produced better forecasting results compared to the conventional approaches of autoregressive with moving average and support vector regressors. Therefore, AI has been successfully used in many applications of intelligent finance systems.

4  Stock Market Manipulation Detection: Supervised Conventional Machine Learning Approach

One of the basic methods of modeling manipulation cases is by deriving multivariate statistical representations to model the incidence. Zare et al. [53] developed two modeling schemes to analyze the possibility of manipulated and non-manipulated stock prices. They used tick stock data that were divided into six sections per day from the Tehran Stock Exchange. The models were applied to each section and hence, a maximum of six detection points were sampled per day. The methods produced two resultant output graphs representing sellers’ and buyers’ behaviors. Each of the seller’s and buyer’s outputs contained three classes of information, which were then modeled using the Gaussian distribution. The two graphs were compared, and the skewness feature was used to determine the possibility of manipulated cases. The respective mean and variance for each Gaussian distribution were statistically derived from the training data. Their results show that the skewness of manipulated cases is generally higher compared to non-manipulated cases. Rather than using simple statistical modeling, most of the existing methods devised to detect manipulation cases use a machine learning approach, whereby the modeling parameters are optimally trained to represent the respective class. In [54], the authors analyzed manipulation cases that involve four popular stocks on the NASDAQ Stock Market: Apple, Google, Intel, and Microsoft. They devised two machine learning methods, support vector machine (SVM) and k-nearest Neighbour (kNN) to identify trade-based manipulation cases due to spoof trading and quote stuffing. They selected three inputs; order price, volume, and trading time, which were then transformed into logarithmic representation. The transformation allowed the features to obtain well-spaced input to better represent manipulation cases. Based on their results, the manipulated cases tend to show outlier features, which is more obvious if logarithmic representation is used. Synthesized manipulation cases were also augmented during the training process to balance the number of cases between normal and manipulated cases.

A set of more comprehensive machine learning methods were explored in [55] by comparing six classifiers that included conditional inference trees, random forest (RF), naive Bayes (NB), SVM, artificial neural networks (ANN), and kNN. They discussed four types of market anomalies, which are high-yield investment fraud, broker embezzlement, late-day trading, and market manipulation. They also increased the number of manipulated cases of Dow Jones Industrial companies to 33 cases from 2003. They also improved the logarithmic representation by introducing an additional log ratio input of stock prices. Finally, they applied a more advanced oversampling method using SMOTEBoost to create synthetic data based on clustered information. The generated samples balanced out the minority class, which applies to all tested methods. The best method according to their findings was NB with an F2 Score of 53% using tick stock data price. One of the crucial steps in machine learning is feature reduction, whereby a smaller representative set of features can better represent the manipulation cases rather than the whole input features. The primary reason for this improvement can be attributed to noise removal from certain features, that can make it more difficult to identify manipulation cases.

Similar to previous findings, Uslu et al. [56] found that NB also produced the best manipulation detection performance. Even though they used a slightly smaller number of cases with only 22 manipulation incidences, they used a more comprehensive set of inputs with 20 features, including price volatility, transaction volume of increasing price, and number of active buying transactions. Daily data from 2010 to 2015, which has been verified by the Capital Market Board of Turkey, were extracted from Borsa Istanbul. The results proved that NB produced better accuracy, precision, and specificity compared to the second-best classifier, SVM. Contrary to these findings, the work in [57] found out that SVM produced the best detection rate, which was tested using Borsa Istanbul data. However, the number of input features was much less with only three types of information, which are average daily return, average change in trading volume, and average volatility. The authors proposed a unique data division between normal and manipulation cases, whereby the normal data were also taken from the same company but extracted before and after the manipulation period. Usually, other researchers have typically used different company stock data to represent normal cases to avoid interfering patterns that may affect the learning process of the classifier. Only the days that have been confirmed as manipulation cases are flagged as high, while the rest are flagged as low. They reported that SVM with RBF kernel produced the best detection performance based on 277 manipulation cases from 1995 to 2004 compared to ANN, LR, and discriminant analysis classifiers.

Apart from NB and SVM, the researchers in [58] and [59] found that the kNN classifier produced the best detection rate. This is in line with the use of different stock data and input features to the classifier that has resulted in different performance findings. The work in [58] used 64 manipulation cases, confirmed by the China Securities Regulatory Commission to build their intelligent detection system. They analyzed daily and tick data, but they found that tick data were too random and concluded that daily data were the more suitable input features. The features used were opening price, highest price, lowest price, closing price, and trading volume, which resulted in 99% accuracy. In addition to the kNN classifier, they analyzed six other classifiers, which are SVM, decision tree (DT), linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), LR, and ANN classifiers, whereby LR produced the lowest accuracy at 89%. On the other hand, if tick data were used as the input, the classification performance was only around 50%, which is similar to random guess performance. The same findings were concluded in [59], which reported that the kNN classifier produced the best detection rate when the system was tested on Moscow Exchange data. They also argued that pump & dump is the earliest form of manipulation type, whereby pinging and spoof trading are the more recent schemes. They also divided the evolution of manipulation methods into three phases, which are modeling based on raw data, multivariate statistical analysis, and machine learning. Even though the paper was published in 2018, they excluded recent research trend based on deep learning, which has proven to better capture complex relationships in manipulation patterns. Another critical weakness of their method is the use of only two types of input features, which are 1) changes in daily trading volume and 2) the ratio of opening to closing price to train their kNN classifier, which is not informative for detecting more advanced manipulation schemes.

The research conducted in [60] used principal component analysis (PCA) to select the dominant input features to identify manipulation cases. The tick data of the bid and ask prices of 15 manipulation cases were extracted from the US Stock Exchange, and used as the training data to develop the optimized classifier. Rather than using raw data in the temporal space, the data were converted into frequency representation using a continuous wavelet transform. A total of seven transformed high-frequency data was capped, and became the input to the classifier. PCA was then applied to reduce the features by selecting between the top-1 and top-5 features. They utilized three classifiers, which are XGboost, kNN, and SVM to experiment on three input configurations; 1) raw temporal input, 2) frequency input and 3) frequency with PCA reduction. The best overall performance was obtained by using a kNN classifier with frequency input features that were selected by using the PCA method. Rizvi et al. [61] then applied the same early setup used in the previous work, but with a minor modification of periodicity removal to highlight unique patterns on the tick stock data. However, they used full synthetic data to validate the proposed method, in which the manipulation cases were artificially injected into the normal data taken from the NASDAQ Stock Exchange Market. The synthetic data were generated to mimic two types of manipulation types, which are spoof trading and pump & dump schemes. As a set of full fake data was used, the results do not reflect the real performance of the tested classifiers. They also introduced an autoencoder (AE) network, trained by using kernel density estimation to prove that their methods worked better than the standard kNN, SVM, K-means, and PCA-based clustering classifiers. A slight difference can also be observed in the frequency transformation method used in their study, whereby they used the discrete version of the wavelet transform, but retained the same feature reduction strategy.

Apart from US-based stock exchange analysis, India-based data have also been analyzed in several studies. It is worth noting that stock market regulations between countries are different, with different sets of rules and standards, even though the same manipulation scheme is referred to, which makes training a machine learning model using composite data from various countries less feasible. In [62], Thoppan developed an optimized standard machine learning classifier to detect manipulation cases using daily stock data. Three input features were selected that include closing price, trading volume, and interday price variance, which were optimized for discriminant analysis (DA), ANN-genetic algorithm (ANN-GA), and SVM classifiers. The hyperparameters for the ANN were optimized using GA, while the optimal kernel for SVM was based on the radial basis function (RBF). The data comprised relatively low manipulation cases from 2003 until 2009 with only 15 cases taken from the Indian Equity Exchange that had been confirmed by the Securities and Exchange Board of India. Their findings show that the best classifier was SVM with a 93% true positive detection, followed by DA with 81% and ANN-GA with 80%. The research in [63] also found a similar performance level for the ANN classifier with 88% accuracy in detecting manipulation cases. However, the authors argued that good detection performance can only be achieved for pump & dump manipulation schemes, while the detection rate for spoof trading manipulation schemes is very poor. This is because the pattern for pump & dump schemes can be easily captured due to sudden trading volume fluctuations, which are not clearly visible in spoof trading cases.

Another research by Sridhar et al. [64] introduced an ensemble approach to further improve the detection rate. The previously discussed methods did not apply any ensemble method that is known to be able to improve classification performance, but at the expense of computational time. The increment in computational burden comes from the need to optimize and execute more than one inference process. Nevertheless, they proved that an ensemble of five ANN works better compared to the standard kNN, linear regression (LR), LDA, QDA, SVM, and NB classifiers. They tested their findings using daily stock data from the Bombay Stock Exchange, which has been verified as manipulated stock cases by the Securities and Exchange Board of India. The three manipulation types that were of interest in their research were price manipulation, volume manipulation, and trade reversal. Since their method required a different selection of input for each model, the original set of features that was chosen was relatively large with 20 features that included the spread of close-open price, lowest price, closing price, and total turnover. By using a data division of (60:15:25) between (training:validation:testing), the ensemble ANN produced 96% accuracy, followed by kNN, which is the second best classifier with only 88% accuracy. The same ensemble approach was used in [65], in which multiple models were designed for each manipulation case. Some of the models were dedicated to different tasks, including price change, bilateral trade, and trade basket ratio, which were then combined to produce anomaly detection to sift out the early candidates for manipulation cases. They used the Tehran Stock Exchange data from 2012 to 2017 that had been confirmed by the Securities and Exchange Organization of Iran as the manipulated cases. Similar to the previous findings, the pump & dump tactic was one of the simplest manipulation schemes, whereby intraday data alone is enough to automatically detect the incidence. Each of the models was combined using uniform weights through a logistic regression method.

Normally, training a conventional machine learning classifier requires an iterative update of the parameters. The length of the training process depends heavily on the starting point during the initialization stage, whether it is close to the optimal points or not, which may result in a short or long training period, respectively. Therefore, Sridhar et al. [66] explored the use of an extreme learning machine approach in designing their manipulation detector. Basically, the basis of the network was similar to a one-hidden-layer ANN architecture, but the parameters were optimized in a single step using nonlinear random projection transformation rather than an iterative process. To verify their proposed method, the authors extracted 16 manipulation cases of volume and price manipulation schemes from the Bombay Stock Exchange. The same input features used in [64] were applied but with the addition of MinMax scaling with different dataset divisions of (70:30) between the (training:testing) data. The resultant true positive rate detections were more than 0.8 for both cases, but the training period was much shorter compared to the standard ANN classifier.

Apart from a standard classification approach to detect manipulation cases, a non-supervised machine learning technique has also been used to create an artificial intelligence bot (AI-Bot) to perform manipulative trading schemes [67]. An AI-Bot was trained to produce an optimal investment strategy, with the baseline to represent the trading dynamic. This AI-Bot was based on a genetic algorithm that utilized 10000 simulation agents to produce the optimal trading strategy. Interestingly, the authors concluded that the AI-Bot suggested that the optimal scheme should be constructed using a manipulative strategy, which indirectly can be used to detect this illegal activity. They simulated eleven synthetic test cases, and most of them produced certain manipulative-based strategies to produce an optimal return on investment. However, the dynamic nature of stock markets makes this strategy questionable since traders continuously adapt to new rules and regulations, which are frequently updated by regulators when a loophole is found. Apart from that, another AI-Bot, known as Bot2Stock was developed to inject malware programs into potential victims to execute manipulative trades [68]. The rogue traders’ modus operandi is to hack the victim’s email or brokerage account and then, deliver a short trade attack that lasts less than a minute. Normally, the victims do not immediately realize that these intelligent agents have attempted manipulative trades because of the short period attacks. The authors revealed that Bot2Stock yielded a return of investment of 2.8% using 1000 simulation agents just by using the pump & dump manipulation strategy. Furthermore, each of the agents was not required to be aware of other agents’ activities, making them independent of each other and reducing the risk of collusion detection. The authors also highlighted that their research aimed to raise awareness on the importance of sound cybersecurity systems with two main suggestions: two-factor authentication and a better alert trigger mechanism. Table 1 summarizes the supervised conventional machine learning methods used in stock market manipulation detection.

5  Stock Market Manipulation Detection: Unsupervised Conventional Machine Learning Approach

Contrary to the supervised method, which aims to classify whether a trading activity is a manipulated case or not, the unsupervised method aims to detect manipulation cases through a clustering approach. No ground truth labeling is provided during the training phase, and any detected anomalous trading activity is considered as a potential manipulation case. Therefore, normal cases are clustered as one group, and any large deviation from this group is treated as the manipulated case. In [69], the researchers collected daily stock data from the Australian Stock Exchange (2009–2013) to test 11 clustering methods to identify manipulation cases. They used a set of five input features, including closing price, transaction volume, transaction total worth, lowest price, and highest price, which is the same input set for all tested clustering methods. They found that 25% of the data fell under the anomaly group, whereby the best-reported methods according to the precision and F1-Score metrics were the local outlier factor (LOF) and clustering-based multivariate Gaussian outlier score (CMGOS) methods. However, the third best clustering method, which is connectivity-based outlier factor (COF) technique reported a less than 0.2 F1-Score, which was far less than the two best methods. The large performance gap between the clustering methods may have been caused by unoptimized training procedures that led to suboptimal parameter settings being used during the testing phase. For a binary classification problem, the formula for F1-Score is shown below, whereby Tp is the true positive, Fp is the false positive and Fn is the false negative.

F1−Score=2Tp2Tp+Fp+Fn(1)

Another method proposed in [70] used contextual anomaly detection to identify the possibility of manipulation cases in stock trading activities. Two contexts were considered either to find the similarity to the neighborhood data or to find the similarity of the time series pattern to the rest of the historical data. Stock prices were organized in the form of a sliding window, whereby the centroid of the windowing data was used to group them into the two contexts. A simple threshold method was then used to determine the most similar pattern whether they fell under a normal or anomalous group. Their findings show that the proposed method performed significantly better with a 0.3 recall rate, compared to the benchmark methods of kNN and random walk clustering methods that produced less than 0.1 recall rates. Since the anomalous behavior of these two stock exchanges is relatively different due to different regulations, making it hard to train a model using a combined dataset of stock exchanges. However, Al-Thani et al. [71] used both US and Qatar stock exchange data to train a nonsupervised manipulation case detector. They argued that, as manipulation schemes keep continuously evolving, the ground truth label cannot be provided for most cases, particularly for new manipulation schemes. Therefore, the unsupervised method was chosen to detect anomaly cases regardless of the type of manipulation scheme. Stock prices were smoothed using a moving average to reduce fluctuation noise. The same contextual anomaly detector as proposed in the previous work was used to group the data points. Besides that, correlation measure was also applied to find similar patterns among companies in the same sectors, which is useful in determining the normal group.

Instead of using a simple correlation measure to relate the manipulation activities of various companies, the work in [72] detected collusion among traders in order to manipulate a stock price. This type of manipulation scheme is harder to detect as the system needs to identify the trader’s group first before any of their trading activities can be scrutinized. Therefore, an unsupervised clustering method has been used to group the traders as there is no ground truth can be inferred accurately for this case. By default, a group of traders working together will surely push up the numbers of trading order or volume compared to the normal pattern. Yet, it is hard to distinguish between the real pattern and the manipulated pattern because there will always be many traders who buy and sell stocks by following the mass trend. Hence, the collusion incidence needs to be identified first, before the group trading activity can be clustered together. Due to the difficulty in obtaining the ground truth, simulated data was used to verify the proposed method. They proposed two-stage methods, which are to cluster the traders first before the activities were clustered. To find possible relations between the traders, graph mining through spectral clustering was used to find the traders’ groups. Once the group has been identified, the LOF clustering method was used to detect anomaly cases by capping a distance threshold between stock data points and the normal group of data. It is also important to note that stock prices can be easily misled by the execution of collective activities by various traders, whereby it will be considered a normal occurrence if only a single trader’s point of view is considered.

6  Stock Market Manipulation Detection: Deep Machine Learning Approach

The primary factors that distinguish the deep machine learning approach from the conventional machine learning approach are the depth and complexity of a network. Hence, a deep network, which is normally deeper, allows it to map complex relationships between input and output. Moreover, it is usually packaged as end-to-end networks that cover both feature extraction and decision-making modules. Consequently, the deep machine learning approach also needs a good computational platform, which is usually in the form of parallel processing that is frequently realized through a graphic processing unit. One of the most basic deep machine learning architectures is a 1-dimensional convolutional neural network (CNN), which was used in [73] to detect stock manipulation cases. An autoencoder with one layer encoder and one layer decoder was devised to detect non-linear patterns, whereas normal patterns are usually linear in nature. Data from Standard and Poor’s 500 from the US stock market were used to verify the proposed architecture, and the results were benchmarked with three conventional approaches, kNN, SVM with Gaussian kernel, and isolated forest classifiers. Five input features from daily stock data were used as the input to the autoencoder, including closing price, opening price, highest price, lowest price, and stock volatility. No normalization or logarithmic representation was applied, as they affect the training process due to different price baselines among the various stocks. However, even without applying any preprocessing steps, the findings proved that an autoencoder still produces the best detection rate compared to the other conventional classifiers. Generally, CNN is more popular in extracting spatial features as implemented in biomedical imaging [74], video surveillance [75], self-driving cars [76] and intelligent agricultural systems [77]. On the other hand, a recurrent neural network (RNN) is designed to better extract the temporal relationship between the input and output, making it more suitable for stock market analysis. In [78], Sridhar and Mootha went a step further by introducing an ensemble of classifiers using an RNN as the base. They tested their proposed ensemble classifier by using daily stock data from the Bombay Stock Exchange. The RNNs were tasked with extracting the features, which were then passed to a dense connection to produce the right class prediction, either a normal or manipulated case.

However, the most used deep architecture for detecting manipulation cases in stock markets is the more advanced variant of vanilla RNN, which is the long short-term memory (LSTM) networks. This architecture allows a network to capture both the long- and short-term relationships between input and output. The short-term relationships are stored using four-gate schemes that allow a network to capture any short-period manipulation scheme. The research in [79] used the simplest form of LSTM, which is just one layer to extract long and short-term dependencies to identify manipulated cases. They dedicated the network to a single type of manipulation case, an insider trading scheme. The daily stock data from 2006 to 2016 were taken from the US stock exchange with a total of 74 manipulation cases. By using a window size equivalent to 10, the proposed LSTM-based classifier produced a detection accuracy of 84%.

In [80], a deeper LSTM network was utilized to identify manipulation cases in cryptocurrency exchanges. The network was tested on data from CoinMarketCap, which happened to hide the identity of the owners. In addition to historical stock price information, blockchain information was also used to identify collusion possibilities among traders. The author tested two configurations of the manipulation detectors whether with or without the sentiment analysis module. Without the help of sentiment analysis information, their 3-layer LSTM did not perform really well compared to the seasonal auto regressive integrated moving average with exogenous factors (SARIMAX) classifier, which is a conventional machine learning method that was found to be effective in identifying manipulation cases. Then, the input of the sentiment analysis system was added to increase detection accuracy using various analyzers, including SVM, deep neural decision forest, RF, and 5-layer CNN classifiers. By using a 3-layer LSTM, Leangarun et al. [43] developed a generative adversarial network (GAN) to detect stock manipulation cases. The unique aspect of this adversarial approach was the two competing networks that aimed to detect and produce fake incidents, which refers to manipulation incidents in this case. The better a network in producing synthetic manipulation cases, the better it produces the ideal detector to identify manipulation cases. Both the generator and discriminator networks used 3-layer LSTM as the basis, and 47 manipulation cases from Thailand Stock Exchange were used as the training and testing data. Limit order book data were used as the input to the network, classified as level II information by the authors.

The authors then enhanced the design of their detector by introducing an improved version of the LSTM-GAN and a new LSTM-AE network [81]. The same limit order book data, containing a set of 15 features were used as the input to the network. All features were first normalized according to the z-score methodology. In the LSTM-AE, a 4-layer LSTM was used as the basis for the encoder and decoder modules. The number of hidden units for the encoder module is 64, 32, 16, and 8, while it is the opposite order for the decoder module, with 8, 16, 32, and 64 hidden units. For the LSTM-GAN, both the discriminator and generator modules were appended with two dense layers each. The authors took an extra precautionary step in selecting normal class stocks because some of them could have contained manipulated cases that had been miss identified by the regulators. Hence, only stocks that had been certified by the Good Corporate Governance list from 2004 until 2016 were used as the training and testing data. Besides that, the authors also ensured a balanced data representation from large, medium, and small-capital companies to capture unique patterns that could vary due to company size. Their results show that the LSTM-AE performed better than the LSTM-GAN in detecting manipulation cases with an accuracy of 0.6839 compared to 0.5640. The same findings were reported in [82], in which vanilla RNN was used to construct the GAN modules. However, the authors stressed that the proposed method was useful in identifying a potential future manipulation scheme through its unique generator module. The entire space of the latent variable can be explored to generate possible cases for future reference. It is worth highlighting that a GAN is normally optimized by using a contrastive or triplet loss function rather than a standard cross-entropy loss function.

In any RNN, the network flow can be unidirectional as such the features flow from the input to the output, or it can be bidirectional as such the flow can be both forward and backward between the two layers. Tallboys et al. [83] used a bidirectional LSTM to construct their LSTM-GAN architecture to detect stock manipulation cases. Data from five US companies were used to validate the proposed method, which has been vetted by US SEC as belonging to the manipulation cases. For the discriminator module, only a single bidirectional LSTM was used with 100 hidden units, while for the generator module, a 2-layer bidirectional LSTM was used in both the encoder and decoder parts of the networks with a total of 64 hidden units per layer. However, their results highlight that a standard 2-layer LSTM with a dense connection performed better in distinguishing normal cases from manipulated cases.

Another variant of the LSTM is the stacked self-attention network, also known as the transformer. This model was utilized in [84] to detect manipulation cases in the Thailand Stock Exchange. However synthetic manipulation cases were added to the data pool to enrich the training samples that were originally composed of six cases from 2004 to 2016. A pump & dump manipulation scheme was injected into the normal data, as it is the most commonly executed fraud strategy. A 60:20:20 data division scheme was used to arrange the dataset which comprised 15 tick data features. The best results were returned by a stack of eight transformers feature extractors with an average accuracy of 91%. The output features from the transformer were down-pooled by using a global pooling operator, which was then fed into the dense connections with a 40% dropout unit. Aside from RNN variants, reinforcement learning was optimized to detect manipulation cases [85]. The authors specifically designed the method for identifying manipulation cases caused by spoof and pinging trading. The learning process adopted the full and partial observability of the Markov decision process. Besides, the reward and punishment of the model were assumed to be stationary or independent of time. The model was trained with the objective of maximizing portfolio growth to mimic a manipulator’s underlying actions. This selection was made due to the contrast in portfolio growth patterns between an honest and a rogue trader. In general, the complexity of the base CNN model is higher compared to the base RNN model. Apart from that, model complexity will increase proportionally to the number of stacked networks for both CNN and RNN models. Furthermore, a gated recurrent unit is a more complex model compared to the LSTM, while LSTM is a more complex model compared to the vanilla RNN. On top of that, the complexity of the deep model will also increase if bidirectional network flow is implemented, instead of unidirectional flow. Table 2 summarizes the deep machine learning methods used in stock market manipulation detection.

7  Challenges and Opportunities

In conclusion, AI has been successfully applied to detect and identify manipulation cases in the stock market. Most approaches have utilized tick data in their experiments, rather than daily trading data. In general, the price pattern of tick data is more likely to be influenced by outliers compared to daily trading data. Apart from that, most of the existing methods were based on the traditional machine learning paradigm, which requires a model designer to select the optimal set of features as the input variables. SVM classifier is found the most popular method to detect manipulation cases, whether it has been used as the proposed method or for benchmarking purposes. Besides that, it is also interesting to note that several studies have reported that the simplest tested classifier produces the best detection rate. This finding may corroborate the low numbers of testing data in verifying algorithm performance. There are several challenges faced by researchers in producing a better-automated algorithm for stock market manipulation detection, including:

1.    The rules and regulations for each country are different, and hence, the definition of stock manipulation has slight differences, especially in the thresholds used to define the manipulation schemes. Besides that, some countries are able to detect manipulation schemes in a shorter period compared to others, which makes some of the shorter-period manipulation activities left undetected.

2.    There is no publicly available online dataset that can be used as the general benchmark since most researchers have used their own country’s stock data to verify their proposed methods. Apart from that, there is no standard database that lists down all the manipulation cases that are publicly accessible by the researchers.

3.    Most of the normal cases that represent the non-manipulated cases were used without implementing stringent due diligence. There is a possibility of manipulated cases that have been labeled as non-manipulated cases due to miss detection of the incident. Contrary to that, some of the manipulated cases were sourced from a verified source, and this stringent approach should also be applied to the non-manipulated cases.

Besides that, there are several areas or opportunities in which current intelligent stock manipulation detection systems could be further improved:

1.    Implement a deep learning analysis network that includes a dense feedforward model, which would be useful in cases of insufficient training data. A dense feedforward connection can map a more complex relationship between input and output to extract meaningful patterns of existing and new manipulation strategies. Two popular techniques can be explored using a skip connection, which is based on an addition operator, or using a dense feedforward connection, which is based on a concatenation operator.

2.    Utilize an advanced attention mechanism to better extract complex patterns in stock historical data. The most popular variation in the attention mechanism is stacked self-attention, which is famously known as the transformer. This network is expected to allocate more weights to a certain part of the data to better extract manipulation patterns. In some cases, manipulation cases occur only for a short duration, while in others, they may occur over several days. Hence, by having a better-designed transformer network, these unique temporal patterns can be better captured to identify manipulation cases.

3.    Use integrated stock data from multiple countries to enrich the training data. However, the selected countries must have similar regulation standards, whereby different countries have different sets of laws. Furthermore, loopholes in certain countries’ regulations may lead to some manipulation cases being undetected, which would lead to false pattern learning by intelligent systems. Therefore, careful vetting procedures must be implemented to reduce the possibility of a noisy integrated dataset.

Funding Statement: This work was supported in part by the RHB-UKM Endowment Fund through Dana Endowmen RHB-UKM under Grant RHB-UKM-2021-001, and in part by the Universiti Kebangsaan Malaysia through the Dana Padanan Kolaborasi under Grant DPK-2021-012.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.