Wrapper Based Linear Discriminant Analysis (LDA) for Intrusion Detection in IIoT

1  Introduction

Intrusion has become a severe issue in the provision of security for network traffic. A single instance of intrusion can hack, steal or even damage data from both computer and network systems, and could potentially harm its hardware. In addition, intrusion can create huge losses in the field of IIoT. Consequently, the processes of detecting and preventing intrusions have become vital in an IIoT environment [1]. Since all industries have been connecting to one another, sharing valuable information in an intelligent way requires the Industrial Internet of Things (IIoT). Through the IIoT platform, many industries operate in various domains such as a smart city, smart power management, automation, logistics, supply chain process, and health care [2]. Through different sensor signals, an IIoT communicates and maintains the established infrastructure, sharing valuable information through a larger network. The aim of the IIoT is to facilitate the intelligent production and delivery of goods by establishing in smart companies effective communication between customers and business dealers [3]. Industry 4.0 has focused on the implementation of an optimized problem for utilizing smart devices through data-driven services. The resulting intelligent activities achieve efficient work by seamlessly connecting together various smart devices. In order to carry out that process, a reliable uninterruptable network connecting the smart devices is required. Through this network connection, cyber-attacks might occur threatening the smart industries in various ways. Hackers also try to get sensible information in the IIoT environment. Therefore, an industrial control system is needed to control the intelligent activities of smart devices [4]. Detecting the vulnerabilities in the traffic of the network in IIoT is called an Intrusion detection system. To provide security in IIoT, the initial step is to identify the hackers, network traffic, security logs, collect information from the system point of view to check for the damages in the network [5].

The process of preventing hackers in IIoT needs customized solutions with smart devices connected to the network. Its applications need deep learning or machine learning techniques for improving the efficiency and low-cost maintenance in the IIoT field. Moreover, various research works have been explored in accordance with IDS in IIoT. The main drawbacks of the existing techniques are time consumption and inefficiency. To overcome these issues, this paper describes the pre-processing technique and implements sterilization and normalization. In order to improve the accuracy, it has adopted wrapper-based direct LDA, and for classification, optimized Extreme Learning Machine (ELM) with Particle swarm optimization (PSO) is applied for relevant features. In the research, it is tried in responding to the following research questions comprehensively:

Q1: How does the proposed feature selection algorithm wrapper-based direct LDA technique outperform after comparing it with the existing work?

Q2: How far the optimized algorithm of Particle Swarm Optimization (PSO) with Extreme Learning Machine (ELM) are the efficient ones?

For intrusion detection systems, the machine learning algorithm of the Naive Bayes algorithm (NB) is used to detect intrusion [6]. This algorithm is utilized for reducing the feature of the original data feature. NB is used for intrusion detection in the aspects of the accuracy and reducing the features.

The Logistic Regression algorithm (LR) was implemented in the evaluation of mapping features of data. In this LR method, it selected 23 features to create a new dataset [7]. The main contributions of this research work are listed below.

1.    Implementation of machine learning-based IDS model for different industrial IIoT applications.

2.    Use of optimized ELM with PSO for classifying the intrusions.

3.    Selection of relevant features using wrapper-based direct LDA.

The research work has been organized as follows: Section 2 elaborates the review of the literature, Section 3 describes the selection of features using wrapper based direct LDA, Section 4 discusses the feature classification, Section 5 shows the evaluation of results and Section 6 presents the conclusion about the research work with future predictions.

2  Review of Literature

The main aim of IDS is to detect, observe and prevent the intruder’s attack in IIoT or interconnected network or node. It acts as a protector and also it safeguards the network node or IIoT from intruders [8]. Sensor nodes are used to identify malicious activities. Similarly, IDS can monitor the actions of the user and determine the behavior of hackers as well as a malicious network node. In the paper [9], different classifier algorithms were implemented with the NSL-KDD data set. It analyzed the protocols by using the WEKA tool for detecting the intruders in the network. For improving the accuracy, dimensionality reduction-based Chronic Fatigue Syndrome (CFS) was used. IDS system monitored the actions of the user to check whether the attack was normal or malicious behavior. In that regard, three types of IDS were used. They were Host-based IDS System (HIDS), Anomaly-based IDS System (AIDS), Network-based IDS System (NIDS) [10].

In the paper [11], the NB15 dataset had introduced a new hybrid model for IDS and monitored each attack using Support Vector Machine (SVM) and Genetic Algorithm (GA). Least Square Support Vector Machine (LS-SVM) based IDS was proposed in the paper [12]. Feature selection-based information concept was used for handling linear and non-linear features of correlation. They used various datasets of KDD cup 99, NSL-KDD, and Kyoto 2006+. For feature reduction in the data set of NB15, Random Forest (RF) was implemented in order to monitor the attack of “Fuzzers”. False Alarm Rate (FAR) and non-detection of data were performed [13]. Deep learning methods had also been implemented for the detection of the multiclass approach based on anomaly detection. In that case, Convolutional Neural Network (CNN) was used in the form of 8 × 8 images which were to be entered into CNN layers for the classification of accuracy [14]. Tab. 1 shows the survey of the existing algorithm in IDS.

3  Proposed Wrapper Based LDA Methodology

In the anomaly detection of IIoT using IDS, dual feature selection of LDA and Principle Component Analysis (PCA) with wrapper-based algorithm were used in the training process. This work was implemented using four phases such as pre-processing, feature selection, feature classification with the optimized algorithm, and ensembled the processing. Initially, the data set was categorized into two sets such as training and testing with the ratio of 8:2. For the sake of improving the accuracy level, pre-processing was needed in the aspect of balancing the data and handling the missing values and irrelevant values. Fig. 1 shows the architecture of the proposed work.

Figure 1: Architecture of proposed work

In this work, sterilization and normalization were included in the pre-processing phase. In the sterilization, the noise values were replaced with null or infinity symbols. In the normalization, all feature values were brought to the same scale. In the proposed work, wrapper-based feature selection and deep feed-forward neural network model were considered. This procedure was used to reduce the number of features for further proceeding. Artificial neural network, filtered-based SVM, RF, and Decision Tree (DT) were used in the classifiers.

3.1 Pre-Processing

Due to the large volume of traffic in the network to detect the intruders, the raw input data from the dataset are found to be intricated to process with all the features. These raw data may consist of either numeric or non-numeric data. These input data are needed to be pre-processed using numericalization and normalization.

3.1.1 Numericalization

For handling the noise values, numericalization is used, which replaces the values of noise win th infinity symbols of mean values or zero values.

3.2 2 Normalization

In the dataset, the input data attributes are in unstructured format such as numeric or non-numeric data. To consider all input data attributes in the uniformity scale, normalization is used. The normalization procedure is defined by min-max procedure, Within the range interval of [−3,3]

Y=−3<min(xmax,max(x,xmin))<3 (1)

Standardization is represented as

xi=Y−μσ (2)

where, mean (μ ) is defined as:

μ=1N∑i=1N⁡(Yi) (3)

The standard deviation σ is defined as:

σ=1N∑i=1N⁡(Yi−μ)2 (4)

After pre-processing, each record in the NSL-KDD dataset contains 100 dimensions.

3.3 Feature Selection using Proposed Wrapper Based Direct LDA Model

In the feature selection process, from the input features d of data set, the relevant features r are considered for getting more optimized result i.e., r<d. This work has implemented the wrapper method of bidirectional elimination with direct LDA (Linear Discriminant Analysis) for feature selection. The bidirectional elimination wrapper method is similar to the forward selection technique. The steps involved in bi-directional elimination are as follows:

In wrapper methods, the feature selection process is based on a Direct Linear Discriminant Analysis (DLDA) algorithm. The steps involved in the wrapper method contain three techniques such as forward selection in which the p-value is less than the threshold value and is added. The next technique is backward elimination from the features in the dataset, in which the p-value is greater than the threshold value and is removed from the dataset. The procedures involved in DLDA are explained below:

The wrapper method is implemented with Direct Linear Discriminantt Analysis (DLDA). An elegant dimensionality reduction is an essential one for making a feature in the database. This reduction in dimensionality is used to reduce the features of input data into low dimensionality of feature space.

In the DLDA algorithm, replacement of null spaces by its zeeigenvaluesues and generation of eigen vectors using null space of he features in the data set are carried out. The optimal discriminant matrix Q which satisfies:

QTdswA=Id,QTdsvaluesQ=Λ (5)

Here Id is the identity matrix and Λ is the diagonal matrix with elements in decreasing order. Further dsvalues , dsw are between class matrix and within class matrix of the dataset.

3.4 Feature Classification

From the feature selection process, the optimal relevant features are selected for classifying the intrusive type of data and normal data. This method is called an intrusive analytical engine. Feature classification is an important stage for detecting the malicious attacks in IIoT. This paper has proposed Extreme Learning Machine (ELM) with Particle Swarm Optimization (PSO) method for detecting the intrusion in IIoT IDS.

3.4.1 Extreme Learning Machine Model (ELM)

The basic concept of ELM is based on single or multiple hidden layer feed forward neural network [25]. The structure of ELM consists of input, one or more multiple layers and output layer. In order to get an accurate and effective detection of intrusion, weight adjustment between input and hidden layer is needed. The comparison and weight adjustment are based on the Single Layer Feed Forwards (SLFN) Neural Network. Fig. 2 shows the architecture of ELM based SLFN.

Figure 2: ELM for IDS

Fig. 2 shows the choice of M sample relevant features from the data set using Algorithm 2 and the select (ri, qi), where ri = [pi1, pi2,…pin]T is the ith data with n various features and qi = [qi1, qi2,… qin]T describes the labels of pi with n hidden neurons and it is defined as:

∑n=1k⁡βih(wnpi+cn)=αi,i=1,2,…,M (6)

where, wn=[wn1,wn2,wn3,…,wnk]T is the weight vector with ith neuron in the hidden layer with corresponding input nodes. βi=[βi1,βi2,…,βin]T are the weight vector values of ith neuron in the hidden layer with corresponding output nodes. cn is the threshold value of ith neuron in the hidden layer. αi=[αi1,αi2,…,αin]T is the ith output neuron and h is the activation function.

3.4.2 Particle Swarm Optimization (PSO)

The optimized algorithm of particle swarm optimization is an intelligence concept based on the global search system [25–27]. It is also called a reasonable algorithm because of its classifying detection of intrusion activities, computational reasonability, and global search [28]. Fig. 3 shows the PSO steps for classifying the intrusion activities in IIoT.

Figure 3: PSO for classifying IDS

By the selected features, PSO uses the principal space for classifying the intrusion activities in IIoT. In the PSO technique, space particles’ population which is called as swarm is used for representing the solutions. For classifying the malicious attack in IIoT, the search space particles of 1 or 0 are randomly generated. If the space component in the principal is 1, the particle is chosen as no intrusive activities and if it is 0, the intrusive activities are ignored. In order to make it more powerful, the principal searching space is randomly traversed and optimal classification of IDS in IIoT is obtained. Velocity and position in the search space should be updated, and the position of the particle is given as:

pi={pi1,pi2,….,piD} (7)

vexi={vei1,vei2,….,veiD} (8)

Here D indicates particle search space. Position and velocity for search space are calculated as:

veidt+1=w∗veidt+c1∗ri1∗(pbid−xidt)+c2∗ri2∗(pgbd−xidt) (9)

Here D is the dimension and t represents the number of iterations for the search space. W is the value of weight and c1 and c2 are the constant values, ri1 and ri2 are randomly distributed values of 0 and 1. pbid and pgbd are dimension space. Values of location for each particle should be updated. The process is repeated until it reaches the maximum iteration or satisfies the fitness values. The algorithmic steps of PSO is given below:

Algorithm 3 shows the dimensionality reduction of features which has improved the accuracy of classification in exploring the malicious attacks. At the same time, it has produced accurate and efficient results in detecting the intruders around IIoT [29–31].

4  Results and Discussion

This section discusses the analysis of the experimental study based on the proposed techniques in the selection of features and classifications of IDS.

4.1 Dataset Description

The dataset contains both normal and intruders’ data in IIoT using NSL-KDD, [29]. The NSL-KDD dataset contains unbalanced distribution of data with large number of repeated data. The experiment is carried out by using two sets of data such as training and testing data set from NSL-KDD data set. There are 41 features which are divided into three types of features namely, symbolic features, binary features and continuous features. Specifically, the training data set contains 22 attacks and the testing data set contains 37 various attacks [32]. There are five classes in the data set. They are Probe, Normal, User to Root (U2R), Remote to Local (R2L), and Denial of Service (DoS) [33]. IDS attacks in IIoT with a detailed description in terms of training, testing data are mentioned in Tab. 2.

4.2 Features Selection using Wrapper Based Direct LDA Model (WDLDA)

The NSL-KDD dataset consists of 41 features which are represented in Tab. 3. The proposed approach of the feature selection called WDLDA which has selected five relevant features from the feature set and it has improved the accuracy in the classification. This proposed approach is compared to the other existing feature selection techniques which is given in Tab. 4. And the selected features along with the name of the proposed techniques are given in Tab. 5.

4.3 Evaluation of Performance of Metric Measures

The experimental results of the proposed work are given using metric measures.

Detection Accuracy (DA):

It is the ratio of exact matching detections over the total number of instances in IDS evaluations.

DA=TP+TNTP+FP+TN+FN (10)

Error Rate (ER):

It measures the ratio of incorrect detections of malware attacks over the total number of cases. It is also called as misclassification of error.

ER=FP+FNTP+FP+TN+FN (11)

Mathews Correlation Coefficient (MCC)

It is a correlation between predicted output with the real data.

MCC=(TP.TN)−(FP.FN)(TP+FP).(TP+FN).(TN+FP).(TN+FN) (12)

4.4 Performance Evaluation of Feature Selection using NSL-KDD Data Set

The performance of IDS is based on the real input features and the selected features are represented in Tab. 6. The real 41 features, normalized 41 input features and feature selection using WDLDA selected 5 features are given. The Tab. 6. shows the results which has proved the important of two steps in the process of normalization. It is used to avoid the traffic in the network data. The selection of feature process also overcomes the overfitting issue by enhancing the overall IDS performance in order to improve the classification in an accurate way. It has decreased the rate of error and the time for detection and also minimized the computational time complexity.

The performance of the selected features in the proposed work is compared with the existing techniques on IDS such as standard Flexible Mutual Information-based Feature Selection (FMIFS), Flexible Linear Correlation Coefficient based Feature Selection FLCFS [34] and Synthetic Minority Oversampling Technique-ENN [35]. The experimented results are shown in Tab. 7.

From the Tab. 7, it is clear that the proposed WDLDA feature selection has produced better performance when compared to the other techniques. The use of selected features have improved the classification in an accurate way so as to protect the IIoT from the attackers.

4.5 Performance Evaluation of Proposed WDLDAIDS with Existing IDS Systems

For proving the proposed work WDLDA based IDS system, it is compared with the existing IDS systems such as DM-Naive Bayes (DMNB) [36], Deep Belief Network (DBN)-SVM [37], Particle Swarm Optimization (PSOM) [38] and Synthetic Minority Oversampling technique-ENN. The evaluated results are shown in Tab. 8.

From the experimental results, the Proposed WDLDA -ELM based Particle Swarm Optimizer (PSO) in IIoT IDS has obtained accuracy rate which is high and low of FPR rate. The proposed system obtains 97.71% of accuracy rate on the basis of detecting the intruders and 0.11 of False Positive Rate. The WDLDA-ELMPSOIDS gets high accuracy because of the optimization process. Fig. 4 shows the Receiver Operating Characteristic Curve (ROC) curve for the analysis of accuracy rate of feature selection and elimination of redundant features from the data set. This is compared with the other existing techniques of DM-Naive Bayes (DMNB), Deep Belief Network (DBN)-SVM, Particle Swarm Optimization (PSOM) and Synthetic Minority Oversampling Technique-ENN.

Figure 4: Elimination of recursive features

From the analysis of Fig. 4, it is observed that the proposed work has produced a significant improvement in the accuracy rate by selecting the features using WDLDA. Fig. 5 shows the false positive rate and true positive rate of the proposed work WDLDA-ELMPSO that is compared with the other exiting techniques.

Figure 5: ROC for FPR & TPR

Intrusion Detection System (IDS) approach is used to identify the malicious intruder of information in IIoT. The proposed work has produced the better performance in the aspects of sensitivity and specificity and also produced exact classification in an accurate way, minimizing the rate of error, and reducing the computational time complexity. Additionally in this work, the optimization algorithm is included so as to increase the accurate classification.

5  Conclusion

An improved wrapper-based feature selection with direct LDA has been proposed in this work for IDS. The usage of IIoT has increased and detected the malicious attacker in an efficient way. Handling the large number of features from the data set gives an inaccurate classification. Therefore, the proposed work WDLDA-ELMPSO is implemented with reduction of features using optimized deep learning algorithm which prevents the intruders in the traffic of network data. The advantage of the proposed work has high detection rate in accuracy. Also, it needs less computation and training time. In the analysis of the result, the proposed work WDLDA-ELMPSO has produced a better result when compared to the existing system. From the results, it is concluded that the accuracy is recorded as 97.71% and the false positive rate is recorded as 0.11 respectively. In future work, the usage of IIoT scheme could be increased and so it is necessary to provide detection of intruders in the traffic of the network by the improved version of various deep learning algorithms.

Funding Statement: The authors received no specific funding for this study.

Conflicts of Interest: The authors declare no conflict of interest regarding the publication of the paper.