A Deep Learning Approach for Detecting Covid-19 Using the Chest X-Ray Images

1  Introduction

The rapid and accurate detection of Covid-19 has been one of the most important/challenging problems for the scientists since the beginning of the pandemic in 2020. Among various solutions/tests, the Polymerase Chain Reaction (PCR) test has been the most-used method for detecting the Covid-19 as a world-wide approach. PCR test though has shown as an arduous test, requiring considerable time to get a result, and needing complicated procedure with the short supply kits. Contrarily, not only the X-ray images are highly attainable but also the scans are considerably low cost [1–6].

The exigency of building a real-time Covid-19 indicator that accurately detects the coronavirus is growing day-by-day. As analyzing the X-ray images are driven from the image processing topic, deep learning would be a promising candidate to do on that score. Therefore, our aim is to provide accurate X-ray images classifier based on deep convolutional neural network (DCNN) to efficiently detect the COVID-19 cases. According to the current literature, a few researches have been done in this specific area since the beginning of the pandemic [7–15]. Regardless of deep learning’s extraordinary ability to solve a wide range of complex problems, optimally updating the weights and biases of the fully-connected neural network (training procedure) is one of the most challenging issues that creates so much potential to improve the performance of the classifier [16–21]. There are a few algorithms which have been frequently used to train the fully-connected in the deep architecture e.g., gradient descent (GD) [22,23], conjugate gradient (CG) [24–26], arbitrary Lagrangian–Eulerian (ALE) [27–29], Hessian-Free optimization (HFO) [30], and Krylov subspace descent (KSD) [31].

1.1 Related Works

It is known that GD-based algorithms for training purposes have relatively lower implementation cost and more rapid execution time [32]. However, these algorithms require various tuning parameters (which should be set manually) to improve the entire performance of the system. Furthermore, the GD-based training algorithms is sequential in essence, thus, parallelizing them during the hardware implementations, for example, on microcontrollers [33] or field programmable gate arrays (FPGAs) would be so challenging. In addition, GD-based algorithms are comparatively slow, regardless of their stable behavior when they are using to train a fully-connected [34]. Therefore, GD-based algorithms require a considerable computational capacity and large random access memory (RAM) resources [35]. A little after, HFO has been used to train the deep architectures especially for deep auto-encoders. It has been shown that this algorithm has a better performance in pre-training and optimizing of deep auto-encoders in comparison with the proposed algorithm by Hinton and Salakhutdinov. It is worth mentioning that, KSD has a lower implementation complexity comparing to HFO while having more robust performance. In addition, the further studies have shown that KSD has a higher classification accuracy and convergence rate than HFO. Nevertheless, KSD needs larger storage space than HFO [30].

Although meta-heuristic-based approaches have played an important role to find the best solutions for a wide range of optimization and complicated problems, the efforts regarding optimizing the deep architectures using the meta-heuristic algorithms are still on the very beginning of their path. The proposed scheme in [36] was the first effort that employed the genetic algorithm (GA) to train a DCNN. This effort had benefited the advantage of different operators in GA (e.g., selection, crossover and mutation) to train the deep architecture by considering the weights and biases of each network as a chromosome. Furthermore, in the re-mixture stage, only the weights and biases of the first and third convolution layers have been rectified. Rosa et al. [37] proposed a harmony search (HS)-based approach for optimizing the DCNN parameters. The authors in [38] have presented a scheme based on progressive unsupervised learning (PUL) for tuning the parameters of pre-trained DCNN. They have shown that their approach can be efficiently implemented and can also be used as an optimized baseline for unsupervised learning. This approach can add a new feature as a selection option between the clustering and fine-tuning phases when the results of the clustering phase is noisy.

The authors in [39] have proposed a GA-based automatic DCNN for optimal image classification. The automatic characteristic of the presented GA is its most important feature in which there is no need to know additional information about the trained DCNN. Though, the main weakness of this approach is the training algorithm turns slow when the deep structure is large. Because the GA’s chromosomes will be considerably large consequently. The authors in [40] have also used the advantage of social-spider optimization (SSO) algorithm to train a deep adaptive neuro fuzzy inference system (ANFIS) for predicting the biochar yields [41]; though, the proposed scheme has faced the ill-conditioning issue. More training algorithms have also been proposed in recent years to better the performance of deep architectures [42–45]. Though, the mentioned schemes lack in providing low computational cost and unreliability when the inputs of the deep architecture are high-dimension and a large set of images.

1.2 Paper Contributions

According to the mentioned drawbacks and weaknesses of the proposed schemes in the literature, we aim to develop a novel meta-heuristic-based approach to optimally training a DCNN to accurately detect the coronavirus by using the COVIDetectioNet dataset [46]. Furthermore, we will update the last layer in the deep architecture of a fully-connected neural network by replacing it with a new fully-connected neural network trained by a novel MHMABC algorithm. The contributions of this paper are as follows:

•   We develop a meta-heuristic algorithm named MHMABC in which the migration operator of the biogeography-based optimization (BBO) is used to improve the exploitation ability of ABC algorithm for the first time.

•   We evaluate the performance of MHMABC algorithm using some benchmark datasets [46].

•   We optimally improve the performance of the DCNN by updating its optimization parameters using the proposed MHMABC algorithm.

•   By using the proposed deep architecture to analyze the X-ray images, we propose an accurate scheme over PCR to detect the Covid-19 cases.

The reminder of this paper is organized as follows. Section 2 represents the proposed MHMABC algorithm. Section 3 describes the developed deep architecture trained by MHMABC algorithm. Section 4 evaluates the performance of MHMABC algorithm on some well-known benchmark datasets. In Section 5, we use the proposed deep architecture to analyze the chest X-ray images by using the COVIDetectioNet dataset [46], and finally, we draw a conclusion of this paper in Section 6.

2  The Proposed MHMABC Algorithm

Artificial bee colony (ABC) is a swarm-based meta-heuristic algorithm that first introduced by Karaboga [47] in 2005. ABC algorithm was inspired by the intelligent search behavior of honey bees. In ABC algorithm, a colony consists of three types of artificial bees:

•   Scout bees: Solutions that are randomly generated to discover new spaces are called scout bees. Scout bees are responsible for exploring the search space.

•   Employed bees: A number of scout bees with good fitness function become employed bees. Employed bees are responsible for advertising quality food sources.

•   Onlooker bees: The onlooker bees are responsible for searching the neighborhood for employed bees. Onlooker bees receive information about food sources and search around these sources. The role of these bees is both exploitation and exploration of algorithm.

In ABC algorithm, scout bees discover a population of initial solution vectors randomly and repeatedly improve them by onlooker and employed bees (using neighbor search method to move towards better solutions while eliminating poor solutions). In general, ABC algorithm uses two main methods (neighbor search and random search) to get the optimal answer: Random search by scout and onlooker bees and neighbor search by employed and onlooker bees. In ABC algorithm, each candidate answer indicates the position of food source, and the quality of the nectar is used as a fitness function. In this algorithm, first, all initial populations are explored by scout bees. Scout bees with best fitness functions are selected as the employed bees. Employed bees exploit the solution positions and then onlooker bees are created. The higher the quality of the employed bee, the more onlooker bees will be created around it. The onlooker bee selects new food position (based on the information of the employed bee) and exploit around these positions. Next, random scout bees are created to find new random food positions. ABC algorithm can be formulated as Eqs. (1)–(3) [46].

Pi=fiti∑n=1SN fitn(1)

Vij=Xij+φij(Xij−Xkj)(2)

XLj=Xminj+rand(0,1)(Xmaxj−Xminj)(3)

where,Pi = Probability of selecting employed bees by onlooker bees, fiti = Fitness function of the ith solution, Vij = Onlooker bee, XLj = Scout bees, Xminj = Low limit of search space, Xmaxj = High limit of search space, SN = Number of employed bees, I ∈{1, 2, …, SN}, j = Dimension ∈ {1, 2, …, D}, k = Onlooker bee number, φij is the random number ∈[0,1], and L = Scout bee number. The weaknesses of the standard ABC algorithm include the lack of exploitation, as well as becoming stuck in local minimums. In ABC algorithm, only neighborhood searches is done around the employed bees. The onlooker bees select new food positions and exploit around these positions. The main purpose of this operator is to improve the fitness function of solutions by neighborhood searches around the employed bee. If the onlooker bees, in addition to neighborhood searches around the employed bee, move towards the several best positions, this movement will cause a variety of good solutions and find other parts of the search space. Thus, a better local/global search is performed in the ABC algorithm, as well as creating variety in the solutions, the convergence rate of the algorithm will also increase. In this paper, a new multi-habitat migration is proposed to improve the exploitation and exploration of ABC algorithm. To develop proposed MHMABC algorithm, the migration operator of the BBO algorithm [48] is used to improve the exploitation of the ABC algorithm. Multi-habitat migration is also proposed to improve the exploration of the algorithm. The migration operation occurs when a guest habitat with an emigration rate sends its species to a host habitat with an immigration rate. In this paper the exponential-logarithmic migration model is used [15]. Emigration rate (μk(j)) and immigration rate (λk(j)) are defined as functions of the number of their species as Eqs. (4) and (5):

{μk(j)=E×Ln(k(j)N+1)k(j)≤N3μk(j)=E×exp(k(j)N−1)k(j)>N3(4)

{λk(j)=I×exp⁡(−k(j)N)k(j)≤N3λk(j)=I×Ln(2−k(j)N)k(j)>N3(5)

where, k(j) represents the rank of the species (based on fitness function) living in the jth habitat, N represents population size, E and I display the highest rates of emigration and immigration, respectively. In a migration model of a standard BBO algorithm, only two habitats are combined, so the habitat produced is not significantly different from the parent’s habitats. Another disadvantage of the migration model of the standard BBO algorithm is that only the space between two habitats is searched (which is a small part of the search space). In this paper, a new multi-habitat migration is proposed as a novel operator to improve the exploitation and exploration of ABC algorithm. Fig. 1 shows the multi-habitat migration and neighborhood search of the MHMABC algorithm. The value of mean square error (MSE) is the same as the fitness function, and the lower the MSE, the better the solution. Fig. 2 shows the flowchart of the MPMABC algorithm. The MPMABC algorithm combines local search (by onlooker and employed bees and migration model) with global search methods (by scouts and onlookers bees and multi-parent migration), and seeks to balance the exploration and exploitation of algorithm.

Figure 1: An example of neighborhood search and multi-habitat migration in MHMABC algorithm

Figure 2: The flowchart of the MHMABC algorithm

3  Training Deep Architecture with MHMABC Algorithm

The main architecture of a DCNN is based on four layers: convolution, pooling, activation function, and fully connected layers. The role of the convolution layer is to put a convolution filter to the input data in order to extract the features by which a set of values will be multiplied with the input data. Next, a kernel passes over an image many times when the weights are multiplied with the input data. The kernel move from top to bottom and right to left, in order to cover all data before a mathematical operation be performed as a next step. The overall inputs will be summed and then generate a distinctive value per every kernel position. Rectified linear unit (ReLu) will act as a nonlinear activation function in the next step where the mapped data produced by the convolution operation will be passed through. ReLu finally replaces the negative values with zeros.

The pooling layer is responsible for reducing the size of the input data and hold the data which are the most worthy. As an example, consider a sample group of four pixels, the max pooling here means that the most valuable pixels which will be held. Pooling layer has an important role in deep learning by decreasing the computational cost balancing the total over fitting. After the convolution and pooling processes are done, the fully connected layer will act which mostly consists of multi-layer perceptron (MLP) neural network. One of the most important contributions of our work is to optimally update the parameters of the fully-connected in the proposed deep architecture using the MHMABC algorithm. To do on that score, we suppose the weights and biases of the MLP neural network as the optimization parameters for the MHMABC algorithm. MHMABC algorithm optimizes the weights and biases of the used DCNN to better classify the Covid-19 X-ray images.

We name the proposed approach MHMABC-DCNN. Fig. 3 shows the definition of a bee in MHMABC algorithm. The fitness function of algorithm (mean square error (MSE)) can be calculated as Eq. (6), where n is the total number of sample, yi is the system output, and di shows the desire value.

Fitnessfunction(MSE)=1n∑i=1n(yi−di)2(6)

Figure 3: An overview of bees in MHMABC algorithm

4  MHMABC Algorithm Performance Evaluation

In this section, the performance of the proposed MHMABC algorithm on different benchmark functions (21 competitive benchmark functions) is evaluated. To evaluate the performance of MHMABC algorithm, three well-known and two novel algorithms called GA, ABC, BBO, capuchin search algorithm (CapSA) [49], and chimp optimization algorithm (ChOA) [50] are used. Tab. 1 shows the calibration parameters of all algorithms.

4.1 Competitive Test Functions

In this section, 16 multimodal and 5 unimodal functions are used to evaluate the performance of the proposed algorithm. Tab. 2 shows the details of these benchmark functions (see [51] for more details). Furthermore, Tab. 3 shows the results of algorithms in this benchmark functions. All the algorithms have been implemented 30 times for each problem. Tab. 3 shows best value of the fitness function (Best), the standard deviation of 30 run (StdDev), and the average runtime of the algorithms (AvgTime). As presented in Tab. 3, the results of the MHMABC algorithm are better than the other algorithms. MHMABC in 21 (from 24 function) test functions (87.5%) has achieved the best value of the “Best”. CapSA in 12 functions (50%), ChOA in 10 functions (41.66%), BBO in 6 functions (25%), ABC in 5 functions (20.83%), and GA in 3 functions (12.50%) have achieved the best value of “Best”.

Also, the MHMABC algorithm is better than others in terms of AvgTime (Fig. 4) and SD. As shown in Fig. 4, the total “Average Time” of the MHMABC is less than other algorithms. The standard deviation (SD) values presented in Tab. 3 indicate that MHMABC has the highest rank of repeatability compared to other algorithms. According to the results, in most of the benchmark problems, the MHMABC produce results with the least amount of SD. Thus, the MHMABC is a very stable search algorithm.

Figure 4: Total AvgTime of algorithms

4.2 Pressure Vessel System Benchmark

In this section, the performance of the proposed MHMABC and other algorithms on the pressure vessel system benchmark are investigated, which can be formulated as Eqs. (7)–(12).

f(X)=0.6224x1x3x4+1.7781x2x32+3.1661x12x4+19.84x12x3(7)

Subjected to

g1(X)=−x1+0.0193x3≤0(8)

g2(X)=−x2+0.00954x3≤0(9)

g3(X)=−πx32x4−43πx33+129600≤0(10)

g4(X)=x4−240≤0(11)

0≤x1,x2≤100,10≤x3,x4≤200(12)

where, x1 = the shell thickness (Ts), x2 = The head thickness (Th), x3 = The radius of the cylindrical shell (R), and x4 = The length of the shell (L). Tab. 4 indicates the results of the different algorithms in pressure vessel system. The best value of the cost function obtained by the MHMABC algorithm is 5918.2586 $, which is better than the other algorithms. In addition, MHMABC performs better than other algorithms in the mean fitness and standard deviation. MHMABC is close to its highest accuracy in iterations = 50, While other algorithms do not have good accuracy. With increasing iteration, MHMABC has achieved high stability and high convergence speed. As shown in Tab. 4, the total “Average Time” of the MHMABC is less than other algorithms for the vessel system benchmark problem.

5  Simulation Results for Covid-19 Detection

In this part, the performance of proposed evolutionary deep architecture called MHMABC-DCNN and the other are evaluated for COVID-19 X-ray images classification. For this comparison, the accuracy, sensitivity, and specificity analysis are used. These three criteria are obtained from the confusion matrix and can be calculated as Eqs. (13)–(15) [52].

Sensitivity=TPTP+FN(13)

Specificity=TNTN+FP(14)

Accuracy=TP+TNTP+FN+FP+TN(15)

where TP = True positive, FN = False negative, TN = True negative, and P = False positive. In order to evaluate the performance of MHMABC-DCNN, we use the advantage of COVID X-ray-5 k [46] dataset, which consists of 2084 and 3100 training and test images, respectively. According to the radiologist advices, we only use the anterior–posterior Covid-19 X-ray images in this dataset since the lateral images have not been appreciate for detection goals. Furthermore, we have ignored using images which lacked minimum range of signs visibility of COVID-19. Therefore, we have only consider 184 images out of the 203 images, and removed 19 images whose signs of COVID-19 were not clear enough. Next, among the 184 remained images, we have choose 84 and 100 photos for the training and test sets, respectively. In order to increase the number of COVID-19 samples to 420, we have used the augmentation techniques. We have also used the benefit of ChexPert dataset [53] (as a supplementary dataset) since the number of non-COVID photos was not enough in the Covid-chestxray-dataset. The ChexPert dataset consists of 224,316 chest X-ray images obtained from 65,240 patients. Finally, we have considered 2000 non-COVID images for the training set and 3000 non-COVID images for the training set and test set. Fig. 5 depicts six stochastic sample images from the COVID-Xray-5k dataset, including two COVID-19 and four standard samples.

Tab. 5 shows the accuracy, specificity, and sensitivity of different deep learning architectures for COVID-19 X-ray images classification. According to Tab. 5, MHMABC-DCNN shows the best results in training and validation datasets. MHMABC-DCNN achieved 99.32% and 98.32% of accuracy in the test and train datasets, respectively. MHMABC-DCNN also achieved 99.63% and 98.82% of sensitivity in the test and train datasets, respectively. The results of evolutionary deep learning architectures in the test dataset show that deep learning architectures are well trained using meta-heuristic algorithms. Because accuracy, specificity, and sensitivity of different deep learning architectures in the test and train datasets are highly stable. As shown in Tab. 5, the “Average Time” of the MHMABC-DCNN is less than other architectures for COVID-19 X-ray images classification. Fig. 6 shows the radar plot of the architectures in both dataset. As can be seen, the accuracy, sensitivity, and specificity of the MHMABC-DCNN is better than other architectures.

Figure 5: Six stochastic sample images from the COVID-X-ray-5 k dataset

Figure 6: The radar plot of the machine learning approaches

Tab. 6 also demonstrates the comparison of the different deep architectures in MSE criteria. It can be seen that the proposed deep MHMABC-DCNN architecture has a fewer MSE than the other approaches. Therefore, the proposed MHMABC-DCNN has been useful to solve this problem. The rank of the algorithms in this comparison are: MHMABC-DCNN, CapSA-DCNN, ChOA-DCNN, BBO-DCNN, ABC-DCNN, GA-DCNN, DCNN, and MLPNN, respectively. To have a more comprehensive comparison, the convergence curve of the architectures has been shown in Fig. 7. As shown in this figure, MHMABC-DCNN, CapCA-DCNN, and ChOA-DCNN architectures converge faster than others. All architectures have been trained by meta-heuristic algorithms show higher convergence rates.

Figure 7: The convergence curve of the architectures

6  Conclusion

This paper proposed a real-time Covid-19 detector based on a novel deep learning architecture. In order to overcome the drawbacks of the PCR tests, we took the advantage of using chest X-ray images to better diagnose positive Covid-19 cases. After that, we proposed a novel algorithm named MHMABC to improve the exploitation and exploration of the ABC algorithm. We then optimally train the fully connected in our deep architecture by using the proposed MHMABC algorithm. The experiment results on Covid-19 X-ray image dataset showed that the proposed scheme (MHMABC-DCNN) could obtain better accuracy and convergence rate while reducing the execution cost. Furthermore, the performance evaluating on some well-known benchmark datasets showed that the proposed MHMABC algorithm outperformed the state-of-the-art algorithms.

Funding Statement: This work has been supported in part by the Institute of Information and Communications Technology Planning and Evaluation (IITP) under the High-Potential Individuals Global Training Program under Grant 2021-0-01532 (50%), and in part by the National Research Foundation of Korea (NRF) under Grant 2020R1A2B5B01002145 (50%), all funded by the Korean Government through Ministry of Science and ICT (MSIT).

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.