Coronavirus 19 (COVID-19) can cause severe pneumonia that may be fatal. Correct diagnosis is essential. Computed tomography (CT) usefully detects symptoms of COVID-19 infection. In this retrospective study, we present an improved framework for detection of COVID-19 infection on CT images; the steps include pre-processing, segmentation, feature extraction/fusion/selection, and classification. In the pre-processing phase, a Gabor wavelet filter is applied to enhance image intensities. A marker-based, watershed controlled approach with thresholding is used to isolate the lung region. In the segmentation phase, COVID-19 lesions are segmented using an encoder-/decoder-based deep learning model in which deepLabv3 serves as the bottleneck and mobilenetv2 as the classification head. DeepLabv3 is an effective decoder that helps to refine segmentation of lesion boundaries. The model was trained using fine-tuned hyperparameters selected after extensive experimentation. Subsequently, the Gray Level Co-occurrence Matrix (GLCM) features and statistical features including circularity, area, and perimeters were computed for each segmented image. The computed features were serially fused and the best features (those that were optimally discriminatory) selected using a Genetic Algorithm (GA) for classification. The performance of the method was evaluated using two benchmark datasets: The COVID-19 Segmentation and the POF Hospital datasets. The results were better than those of existing methods.
SARS-CoV-2 (also known as COVID-19) is a novel coronavirus. The disease caused by the virus has been declared a pandemic by the World Health Organization (WHO) and has spread to more than 170 countries worldwide. Globally, more than 10 million people have been infected. Recently, the WHO has stated that 33,842,281 COVID-19 cases have been confirmed worldwide, as have 1,010,634 deaths. In Pakistan, COVID-19 is spreading rapidly; a recent report described 312,263 confirmed cases, 6,479 deaths, and 296,881 recoveries.
The treatment of such patients in special care units requires early detection of COVID-19 to increase the survival rate. Separation of healthy people from affected patients is the prime objective; this is possible only if diagnosis is early. The diagnostic technique using throat swabs is 30% to 60% accurate, which means that undiagnosed patients may commonly infect healthy people [
Pre-processing is performed using a Gabor filter to enhance image intensities, and a marker controller watershed with thresholding is used to segment the actual lung region.
The lesion is segmented using a three-dimensional (3D) semantic segmentation model. In this model, the deeplabv3 network serves as the bottleneck of the moblenetv2 module. The combination of these convolutional neural networks accurately segments the infected lung region.
The GLCM and statistical features are computed from the segmented regions, and then fused and optimized by the Genetic Algorithm (GA) used for classification.
The structure of the manuscript is as follows: in Section 2, related work is described; in Section 3, our work is explained; in Section 4, the findings and discussion appear; and, in Section 5, conclusions are finally drawn.
Computerized imaging techniques are important when seeking to diagnose COVID-19 infection early. Computed tomography (CT) of the chest is preferred to X-ray [
Artificial intelligence methods play important roles when learning the patterns of many CT images and then making predictions based on those patterns [
Refs. | Year | Machine/deep learning methodologies | Dataset | Results | |
---|---|---|---|---|---|
[ |
2020 | AI tools for the analysis of CT images | 157 patients data | 0.996 AUC, 98.2 sensitivity, 92.2 specificity | |
[ |
2020 | ResNet18, ResNet50, SqueezeNet, and DenseNet-121 | 5,000 Chest X-rays | 98% sensitivity, 90% specificity | |
[ |
2018 | Pre-trained models | 34,006 CT images | 0.830 precision, 0.967 sensitivity, 0.004 specificity | |
[ |
2018 | Bayesian convolutional neural networks (BCNN) | 5941 Chest X-rays | 80% accuracy | |
[ |
2020 | Confidence aware anomaly detection (CAAD) model | 5,977 Chest X-rays | AUC of 83.61% |
The proposed model features three major phases:
In Phase 1, a Gabor filter with fine-tuned parameters is used in the pre-processing phase to improve image quality.
In Phase 2, the lung region is segmented using a marker-controlled watershed method. Deeplav3 serves as the bottleneck of the mobilenetv2 module used to segment the actual lesions of COVID-19.
In Phase 3, geometrical and statistical features are computed from each segmented image. The extracted characteristics are serially fused and, on addition of a GA, redundant features are eliminated. Each vector of optimized functionality is forwarded to the COVID-19 classifiers.
We applied the Gabor filter (a linear filter devised by Dennis Gabor) to CT images when analyzing textural patterns that contain rich information related to pathology. The filter analyzes the specific frequency contents in certain image directions. In the spatial field, a 2D Gabor filter with a Gaussian kernel is modulated via a plane sinusoidal wave. Given the convolution multiplication property, the Fourier transform of the impulse response (the Gabor filter) is convolved with the Fourier transform of the harmonic (sinusoidal) function and a Gaussian function. A Gabor filter is mathematically expressed as:
where B and C denote normalizing factors and f represents frequency. The Gabor-filtered images and the original CT images are illustrated in
A marker-controlled watershed approach with thresholding was then applied to the Gabor-filtered images to segment the actual lung surface, as shown in
where
We propose a new semantic segmentation model in which deeplabv3 serves as the bottleneck of mobilenetv2. There are 186 layers, thus 1 input layer; 70 convolutional layers; two 2D cropping layers; 59 batch norm layers; 43 ReLU layers; eight addition layers; and single depth concatenation, softmax, and pixel classification layers. The primary goal of the model is to assign semantic class labels pixel-by-pixel in the input image. The model is fine-tuned using the learning parameters shown in
Implementation software | Segmentation model | Size of input | Optimization | Size of mini-batch | Learning rate | Number of epochs |
---|---|---|---|---|---|---|
Matlab 2020Ra | Deeplabv3 and | Sgdm | 64 | 0.001 | 30 | |
Mobilenetv2 |
The proposed segmentation outcomes are illustrated in
The GLCM features extracted from the segmented region (
The GLCM of an image
Here, n is the sum of elements in G, and
Energy is used to measure local textural homogeneity within an image.
Entropy is used to measures the randomness.
where r denotes the number of intensity levels
Variance is utilized to evaluate the distribution of intensity values.
Homogeneity
Cluster prominence (
Cluster shade
Dissimilarity (
The three statistical features of length
In this study, GLCM features were computed in terms of mean, range, and variance and later fused with statistical features such as area, perimeter, and circularity. The length of each fused feature vector is
Iterations | 30 |
---|---|
Size of population | 10 |
Crossover % | 0.5 |
Parents | |
Mutation % | 0.3 |
Total number of the mutation | |
Selected pressure ( |
8 |
The vector of the resulting features was transferred to an SVM [
In this work, two types of experiments were performed. The first evaluated segmentation method performance using the ground truth annotations and the second explored the classification of CT images.
We used two datasets to validate our proposed approach: The POF Hospital dataset and the COVID-19 segmentation dataset. The POF hospital CT dataset contains data on 100 confirmed COVID-19 patients (2,879 positive CT images of 4,089 CT images). The COVID-19 segmentation dataset consists of the CT imaging data on 40 patients (each of 100 axial slices) and ground truth images. The classification data contain 373 positive and 456 negative slices [
In this experiment, semantic segmentation was performed using a group of similar pixels as revealed by their class labels. The prediction was performed at the pixel level; each image pixel was classified based on the corresponding class label. The results of the proposed segmentation method are compared pixel-by-pixel with the ground truth images in
The segmentation technique performance was validated in terms of various measures such as accuracy (mean and global), Intersection Over Union (IoU) (mean and weighted), and Boundary Contour Matching (BF) scores mathematically expressed as follows:
The IoU is also termed the Jaccard index, and computes the proportions of correctly classified pixels among the predicted and ground truth pixels.
The IoU is computed in terms of mean and weighted values. The mean IoU measures the average IoU of all classes in a particular image. The average IoU of a certain class is weighted by the number of pixels in that class. Th boundary F1 scores compute the proportions of class boundaries that align with the actual boundaries.
The results of the proposed segmentation method are listed in
Accuracy (Mean) | Accuracy (Global) | IoU (Mean) | IoU (Weighted) | BFscore (Mean) |
---|---|---|---|---|
0.97 | 0.89 | 0.90 | 0.96 | 0.98 |
The extracted GLCM and statistical features were serially fused. The GA chose the informative features. The resultant optimal feature vectors were fed to the support vector machine (SVM), extreme learning machine (ELM), and the ensemble (optimized). The training and testing images were validated using holdouts of 0.7 and 0.5. A summary appears in
Datasets | Cross-validation approach | Training and testing images |
---|---|---|
COVID-19 segmentation dataset | 0.5 hold out | 2044 training, 2044 testing |
0.7 hold out | 2862 training, 1226 testing | |
POF dataset | 0.5 hold out | 414 training, 414 testing |
0.7 hold out | 580 training, 248 testing |
We used the MATLAB 2020Ra Toolbox with the Nvidia Graphic 740k card to run all tests. The results of classification are presented by the two different separation criteria (0.5 and 0.7 holdout validations) in
The results of the classification are listed in
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 0.96 | 0.98 | 98.37 |
COVID-19 | 0.99 | 0.98 | 98.37 |
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 0.83 | 0.82 | 93.27 |
COVID-19 | 0.96 | 0.96 | 93.27 |
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 0.89 | 0.97 | 97.54 |
COVID-19 | 0.99 | 0.98 | 97.54 |
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 0.79 | 0.88 | 90.39 |
COVID-19 | 0.95 | 0.91 | 90.39 |
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 0.98 | 0.94 | 98.49 |
COVID-19 | 0.99 | 1.0 | 98.49 |
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 0.99 | 0.98 | 99.58 |
COVID-19 | 1.0 | 1.0 | 99.58 |
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 0.97 | 0.94 | 98.62 |
COVID-19 | 0.99 | 0.99 | 98.62 |
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 0.94 | 0.97 | 98.59 |
COVID-19 | 0.99 | 0.99 | 98.59 |
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 0.89 | 0.91 | 96.74 |
COVID-19 | 0.98 | 0.98 | 96.74 |
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 0.83 | 0.97 | 96.33 |
COVID-19 | 0.99 | 0.96 | 96.33 |
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 0.91 | 0.88 | 96.71 |
COVID-19 | 0.98 | 0.98 | 96.71 |
Classes | Sensitivity | Precision | Accuracy (%) |
---|---|---|---|
Non-COVID-19 | 1.0 | 0.97 | 99.53 |
COVID-19 | 0.99 | 1.0 | 99.53 |
The classification results show that the 0.5 holdout cross-validation method was associated with accuracies of 98.37% on the ELM, 93.27% on the SVM, and 97.54% on the optimized ensemble. The experimental evaluation thus showed that the ELM outperformed the other benchmark classifiers.
Similarly, the experimental evaluation using 0.7 holdout cross-validation yielded accuracies of 90.39% on the ELM, 98.49% on the SVM, and 99.58% on the optimized ensemble. Hence, the SVM exhibited the best accuracy.
The classification accuracies (50:50) of the COVID-19 segmentation dataset were 98.62% on the ELM, 98.59% on the SVM, and 96.74% on the optimized ensemble; the ELM was best.
The classification accuracies of positive/negative images (70/30) were 96.33% on the ELM, 96.71% on the SVM, and 99.53% on the optimized ensemble; the latter was thus best.
The classification results of the COVID-19 segmentation dataset were compared with a recent work in terms of accuracy; the existing method achieved 89% accuracy [
Ref. | Year | Accuracy (%) |
---|---|---|
[ |
2020 | 89 |
Proposed method | 99.53 |
We used a modified deep learning model to diagnose COVID-19 infection. CT images are noisy and such infections are hard to detect; denoising of CT images is a challenging task. We used a Gabor wavelet filter to remove noise and a marker-controlled watershed method to eliminate non-lung regions including background pixels.
Segmentation is an intricate task because lesions may be irregular in terms of both shape and size. Therefore, we used deeplabv3 as the bottleneck and mobilenetv2 as the classification head for segmentation. As accurate classification depends on the feature vectors used; we extracted GLCM features including the mean, range, and variance, and statistical features, and fused them serially.
We validated the proposed approach using two different training and testing criteria (holdouts of 0.5 and 0.7). The proposed approach yielded consistent results, confirming that it is novel. The approach accurately segmented/classified COVID-19 features and will help radiologists in the present pandemic situation. In future, our approach will be deployed in hospitals to analyze images automatically and predict either a normal or COVID-19-infected lung.