Multi-Level Deep Generative Adversarial Networks for Brain Tumor Classification on Magnetic Resonance Images

1  Introduction

In the current age, many computer vision and image processing techniques have substantially influenced medicinal image examination and diagnosis to achieve the results of competent medical experts [1,2]. The development of computer-aided systems and deep learning techniques in the medical field has become the most prominent research area to meet the current age challenges in different medical fields. Although medical imaging data is increasing day by day, with an increasing number of robust systems, newly developed techniques are the main backbone to executing the success of different deep neural networks [3].

Brain tumors (BT) occur due to brain cells’ hysterical and atypical evolution [4]. There are many types of brain tumors, but the most common is glioma tumor (GT). The GT occurs due to glial cells in the central nervous system responsible for handling the nutritional elements of a human brain [5]. Moreover, meningioma, the word originated from three membranes surrounding the brain and spinal cord. It originates in these membranes [6]. The pituitary is a small gland located at the back of the nose. Its irregular growth affects other brain glands and many functions in the body [7].

Additionally, medical imaging-related applications are increasing daily to facilitate the medical experts [8]. The convolutional neural networks (CNNs) have been applied to increase radiology practice efficiency, elastic registration between 3D MRI images, separation of brain tumors, neuropsychiatric illnesses, cancer in the breast, and medically ultrasound-based images [9].

The classification of brain tumorous MRI is the latest research topic centered on many deep learning techniques for qualitative analysis of brain tumors [10]. According to medical experts, brain tumor is one of the dangerous diseases that leads to the maximum number of deaths of all ages [11]. That’s why the accurate diagnosis process of BT during the curing process is crucial and needs a maximum level of specialization for medical experts to complete the diagnosis process. So, a Generative Adversarial Network (GAN) is prosed to handle this challenge.

On the other hand, the proposed technique known as GAN is the well-known machine learning technique consisting of two neural networks contended with each other to be most accurate in prediction results [12]. The GAN network consists of the generator model and discriminator model. The generator is the CNN, and the discriminator is the de-convolutional neural network. The generator’s function is to create the output artificially, while the discriminator is used to verify the artificially created output by the generator. GAN creates its training data to generate the higher quality output for text to image translation, image editing, creating and recreating with higher resolution, and 3-D object creation. Conversely, the discriminator model grows into a better network to verify the artificially created output [13].

Identifying the desired end product and collecting an original training dataset depends on those parameters is the initial phase in creating a GAN network. The achieved information known as random input is fed into the generator till it reaches a baseline level of resultant output. After that, these features are fed up with discriminators with authentic data points. The discriminator analyses the authenticity of each image data and produces a probability between 1 and 0 for true and fake correlates [14]. The achievement of these results is manual-based checking, and the method is repetitive till the preferred outcomes are achieved, as shown in Fig. 1.

Figure 1: General GAN network architecture

As a contribution to our proposed work for brain tumor classification, this research presented a new strategy known as the GAN network. The contributions of the proposed work are given below:

•   The GAN network created artificial medical image data to improve the learning ability of physicians and other computer-aided systems for the generation of augmented data.

•   GAN network initially goes through a training process on different MRI images to generate realistic images of brain tumors that are entirely altered from the original ones.

The remaining paper structure is presented as the second part presents the interrelated work, the third part explains the overall methodology, fourth one describes the experimental-based results and discussions, and the last section indicates the conclusion of the whole research work.

2  Related Work

Along with other traditional methods [15], CNNs [16,17] have a prominent role in medicinal image-based analysis, including brain segmentation of magnetic resonance images (MRIs) [18]. MR imaging consists of harmful ionizing radiations, but these radiations do not affect the patients and provide information related to the brain tumor’s size, shape, type, and position. Glioma and meningioma are the two hazardous types of brain tumors (BTs) detected by MRI scans. If these BTs are not detected initially, they can be very dangerous and lead to death [19].

Glioma is the famous type of BTs in humans worldwide [20]. The famous institute, known as the World Health Organization (WHO), categorizes the tumor into four grades. Grade I and II is the lower level of brain tumor known as meningioma, and grade III and IV is identified as a more severe type of tumor-like glioma [21]. More or less 20% is the meningioma type of brain tumor and treated as slowest in growth. The initial stage diagnosis of this type of tumor can save many people’s lives [22].

The interpretation of MRIs by trained medical experts of brain tumorous disease is a considerably more time-consuming, sensitive, and complicated job. In many cases, the brain tumor size is slightly different in color concentration, form, and surface [23]. Noisy images and fewer concentration factors for the trained medical expert are the other two issues for misinterpretation of brain tumors. Therefore the correct decision about the diagnosis of BT is a much more challenging task for the trained neurologists and surgeons [24].

On the other hand, the author applied three classifiers known as CNN, random forest classifier, and fully connected neural network for the brain tumor analysis in [25]. The best accuracy obtained by CNN is 90.26%. In this technique, the CNN consists of different parameters such as convolutional, fully connected, and pooling layers for the identification process [26]. Moreover, a capsule net was used to organize three brain tumors fed with MRI images. The classification accuracy obtained is 86.56% with the segmentation process and 72% with new brain tumor MRIs [27].

A Generative Adversarial Networks (GANs) was presented in the paper [28], which contains two nets first one is a generator known as a convolutional neural network, and the second one is a discriminator famous as a deconvolutional neural network. The generator attempts to yield actual output created on real distributed data to step out of the discriminator model. It is used to differentiate the actual data as generated by the generator. Moreover, the generator produces more real output and discriminator tries to create the most realistic copies of accurate data during the training process [13].

In another work, GAN was used to produce realistic images of faces, buildings, and other places that were not easily detected by human eyes [29]. GAN was used for the semantic BT segmentation extensively [30]. Consequently, the segmented process is used as a generator, and the discriminator is used to construct the additional GANs [31]. The discriminator delivers the fractional reality of the data without labels in the unsupervised learning process [32]. In another research, deep CNN used a deep convolution-based GAN for both generator and discriminator models.

Most of the researchers used data augmentation techniques [33] to enhance the quantity of the dataset. These techniques based on the GAN network create realistic data but almost new sample images to enhance the performance [28]. These newly created data images cover the place of accurate data, which is used to increase the output accuracy [12]. The GAN network is also used for classification facilitation [34,35], object detection [36,37], and separation [38] to handle the deficiency of trainable data images.

Additionally, adversarial learning is also used for medicinal imaging for genuine retinal and computed tomography image creation to enhance the output values [39,40]. A recent study shows a valuable performance for liver classification by using CNN based technique for GAN training data [41]. However, GAN-based MRI image generation is not so far appropriate because of low contrast MRI images and intra-classification variations. In another work, the author generated 64 × 64/128 × 128 MRIs by using conventional GANs even medical experts could not distinguish between real and synthetic images [42].

Recently, many researchers in medical imaging have begun to use the GAN network to generate the super-resolution images, anomaly detection, and estimation of computed tomography (CT) images from the related MR images [43]. Additionally, the generation of medical images from GAN as the realistic data augmentation GAN-based model is one of the best solutions for the training of physicians for MRI brain tumor images [44].

3  Methodology

The methodology section consists of the overall proposed methodology. A GAN is proposed in this work, which comprises two models known as generator and discriminator. The details of the applied methodology are given below.

3.1 Generator Model

The first model is recognized as a generator. It is also famous as ordinary CNN. The generator model creates a sample in the domain from a fixed-length random vector as input. The Gaussian distribution randomly draws vector space and seals it to the generative process. During the training process, the vectors in the vector space correspond to the respective points in the domain space and form the distribution map. After that, it creates the random latent space variables which we cannot directly observe.

During the example of GAN, the generator allots sense to topics in a specified latent space, letting new facts pinched from the same space be delivered as the input and applied to yield new and various output samples. It attempts to yield actual output based on accurate spread data to mislead the discriminator. It is used to differentiate the actual data produced by the generator, as shown in Fig. 2a.

Figure 2: Representation of generator model (a) and discriminator model view (b)

The generator model behaves like an ordinary CNN. The presented CNN model have own architecture and working environment. The generator takes input as random sample and produces new samples. This workflow uses seventeen layers to generate the generator model. The first layer is the input and image size is known by setting the channel’s weight, height, and size. The kernel size, the network’s learning rate, and stride are indicated using the convolution layer as shown in Eq. (3).

For the swiftness of the network, a batch normalization layer is fed to the framework. On the other hand, an activation function is added in the form of the rectified linear unit (ReLU) as presented in Eq. (1). More input is distributed into numerous pooling sections, and the max-pooling layer calculates the peak value for every region. The overall structure of the used layers is presented in Tab. 1 and Fig. 3 using various constraints and kernel dimensions that vary from 4 to 16, and the learning rate value is 0.001 with an epoch value of 50. Subsequently, the fully connected layer pools all the yields of earlier layers; in other words, it chains all the learned characteristics of the layers to acknowledge the enormous configurations as revealed in Eq. (2). Additionally, the softmax layer works as an activation function for normalization. By applying the possibilities, the classification layer categorizes it into stated classes.

f(a) = {0, for  b <0b,for  b≥0  (1)

Figure 3: GAN model workflow

Here, f (a) is the ReLU layer functionality and b the target value as revealed in Eq. (1).

fc=rcF(fc−1)=((fc−1))twic+ac (2)

In Eq. (2), fc show the output of fully connected net, F is the number of filters r the input, w weight and a indicate the constant.

c =(rd+2pa− fd)str +1 (3)

Here Eq. (3) is the convolutional layer c, rd is the dimension of input,  pa shows padding,  fd indicate the filter dimensions and str represent the stride.

3.2 Discriminator Model

The second part of the GAN model is known as the discriminator model. The function of the discriminator is used to verify the artificially created output by the generator. The discriminator model inputs a domain sample and forecasts a true or false binary class label .The real-world example is taken from the trainable dataset. The generator model only generates examples. A normal classification model is used as a discriminator. The discriminator model is discarded after training because the generator indicate the interesting space. As it has learned to abstract features from examples in the problematic domain, the generator can be repurposed in some cases. On the other hand, transfer learning programs can use similar or equal input data for all feature extraction layers, as shown in Fig. 2b.

It behaves like an ordinary deconvolutional neural network. This model applied overall ten layers of CNN with various constraints and filters for the classification purpose. The number one layer is input layer having restrictions as weight, height, and size of the channel. In the convolutional layer number of filters, size and stride are defined, while the kernel values are specified from 4 to 16. Moreover, the learning rate for the training process is 0.001, with an epoch value of 40. The comprehensive layers explanation is given in Tab. 2, and workflow is also revealed in Fig. 3.

3.3 Overall Training Process

The workflow in Fig. 3 displays the complete training of the GAN network. At the initial level pre-trained CNN model is used as a discriminator in a GAN to distinguish real MRIs produced by the generative model from real ones. The discriminator can extract and learn the features of MRIs in this way. After that, pre-trained CNN is combined for brain tumor brain tumor classification. During the classification module, the last fully connected net is replaced with a softmax layer to enhance classification accuracy. GAN was accomplished to handle 8 to 64 batches for each brain tumor image. Every batch contains two pixel-wise mini-batch images. The first few images are from datasets of actual MRIs, while a few images are from a generator model with a randomized vector as input from a specific latent space. In this way, the generator generates some sample images. Additionally, the discriminator gets the real image as input, passes it through the deconvolutional model, and creates binary classification for real and fake images in the training and testing process as revealed in Fig. 3.

4  Results and Discussion

All the models in this research have been implemented by using Python (3.6.6) with supportive libraries as NumPy array (np), Keras models (2.2.4), layers and utils, matplotlib, sklearn utils and metrics, TensorFlow models, pandas, and seaborn. These libraries support the development of machine-learning applications. The proposed model was calculated on the system with Core i9 11th generation, Nvidia graphic card 6 GB, and 16 GB of RAM.

4.1 Dataset Details

The freely accessible CE-MRI dataset (https://figshare.com/articles/braintumor dataset/1512427) has been used. It consists of 2-dimensional images with a large slice gape. The dataset was collected from 2005 to 2020 from different hospitals in China. The dataset consists of four tumor classes: glioma, meningioma, pituitary, and no tumor [45], as revealed in Fig. 4. A glioma tumor is a common brain tumor that originates in the glial tissues surrounded by neurons [46].

Figure 4: Sample images of CE-MRI dataset

In contrast, meningioma is a tumor that arises from meninges tissues surrounding the brain and spinal cardinal system [47]. A pituitary tumor is due to the abnormal growth of pituitary glands in the back of the nose [48]. The size of each image in the dataset is 512 × 512 pixels, as shown in Fig. 2. The CE-MRI dataset was separated into training 70%, validation 15%, and testing 15%. The description of dataset is given below in Tab. 3.

4.2 Preprocessing

The images in the dataset are 2-dimensional with 512 × 512 pixels, as shown in Fig. 4. The dataset is checked with duplication, missing values, label name, and extension during the cleaning. Moreover, all the images are made noise-free using a histogram equalizer. In this work, the images are directly fed to the convolutional neural network, and the kernel is applied to resize the images. The results are highly dependent on these values. However, these values are not fixed and vary according to the image pixel sizes. To remove these intensity variations, normalization is used. So before giving values to the CNN model, all the values are normalized, having the same size range. Now the size of the images is 224 × 224 after normalization and resizing. In this way, the training process speeds up by resizing images and requires less memory.

4.3 Statistical Evaluations

The proposed GAN network for brain tumor classification and detection is calculated with the help of different arithmetical calculations specified underneath Eqs. (4)–(7). The correctly classified images are known tp, negatively true classified images are denoted as tn, while fp are the positively incorrect classified images. Moreover, fn represents the total number of negatively classified images. The statistical equations are given below:

Precision=tp(tp+fp) (4)

Recall=tp(tp+fn) (5)

F1−score=2(Sensitivity ×Precision)Sensitivity + Precision   (6)

Accuracy=(tp+tn)(tn+tp+fp+fn) (7)

4.4 Discussion

After completion of the training and testing process, the GAN network creates some fake images to enhance the dataset to increase the training of the model, as shown in Figs. 5A, 5B, 5C and 5D. Fig. 7 shows the GAN model confusion matrix for the four-class classification of the test dataset. The test dataset consists of four tumor classes glioma, meningioma, pituitary, and no tumor. On the other hand Fig. 6 shows the training process confusion Metrix. The numbers in the boxes of the confusion matrix show the overall number of images used for the classification purpose. Fig. 12 displays the graphical demonstration of accuracy and loss for training and validation. The statistical values are also shown in Tab. 2 with average precision: 0.92, recall: 0.93, F1-score: 0.93 and accuracy 0.98 values.

Figure 5: Fake data generation by GAN network (A) 8 batches (B) 16 batches (C) 32 batches (D) 64 batches

Figure 6: GAN model four-class classification confusion matrix for the training dataset

Figure 7: GAN model four-class classification confusion matrix for the test dataset

Figs. 8, 9, 10 and 11 shows the graphical demonstration using the GAN network’s 8, 16, 32, and 64 mini-batches. The y-axis depicts the loss, and the x-axis represents the GAN network simulation batch numbers. The blue line indicates the generator model evaluation. The orange line represents the discriminator model values and overlaps for some values.

Figure 8: Graphical demonstration of 8 batches loss for GAN network

Figure 9: Graphical demonstration of 16 batches loss for GAN network

Figure 10: Graphical demonstration of 32 batches loss for GAN network

Figure 11: Graphical demonstration of 64 batches loss for GAN network

Figure 12: Graphical demonstration of training and validation accuracy and loss

During the simulation process, the batch number increases up to 17500 with almost 5.5 percent loss for 8 batch size simulation, as shown in Fig. 8. On the other hand, mini-batch 16 decreases to 8000 with almost 6, as shown in Fig. 9. Additionally for mini-batch 32, the batch number values decrease to 4000 with a loss value of almost 6, as shown in Fig. 10. Moreover, the values for mini-batch 64 decrease upto 2000 with almost 6.5 loss as shown in Fig. 11.

Tab. 4 displays the statistical values of the GAN model after the simulation with generator and discriminator models. The average statistical values are given in Tab. 4 as precision: 0.92, recall: 0.93, F1-score: 0.93 and accuracy 0.98 values.

The Tab. 5 compares the accuracy values for all the existing model values. The proposed GAN architecture shows the best accuracy of 0.96% due to its generator and discriminator structures. The other existing approaches show the lowest accuracy. The accuracy value for the GAN proposed model is the highest one as 0.96 shown in Tab. 4.

5  Conclusion

We have proposed a deep learning method known as Generative Adversarial Network (GAN) in this research. It consists of two parts, famous as generator and discriminator. First, a pre-trained CNN model is applied in the discriminator model to extract the robust features and learn the basic structures of MRI images in CNN layers. Then a profound CNN model is applied to differentiate between four tumor classes. The publically available CE-MRI images dataset is used to evaluate the GAN model. It achieved an accuracy of 96%, which is the highest one as compared to the existing techniques. In medical terms, the proposed model would help the generation of real medical image data to maximize the training of the medical experts for the acute prediction of medical diseases.

Funding Statement: Authors would like to acknowledge the support of the Deputy for Research and Innovation-Ministry of Education, Kingdom of Saudi Arabia for funding this research through a project (NU/IFC/ENT/01/014) under the institutional funding committee at Najran University, Kingdom of Saudi Arabia.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.