Arithmetic Optimization with Ensemble Deep Transfer Learning Based Melanoma Classification

1  Introduction

Melanoma has been classified as one of the serious skin cancer types and is ranked at 9th position amongst the common cancer types. Every year, more than 132,000 cases are diagnosed across the globe [1]. According to the report by The American Cancer institution in the year 2019, 1,92,310 patients are affected with melanoma in U.S. In the last few years, the prevalence rate of melanoma is increasing with more number of patients getting infected with the disease alike other cancer types. A minor operation may incline the possibility of recovery after early detection of melanoma [2]. Dermoscopy is the commonly used imaging technique used for diagnosing melanoma. It expands the skin surface affected by cancer and its structure can be easily seen by dermatologists for investigation [3]. But discrimination of skin lesions, by dermatologists, from dermoscopic images is subjective and a time-consuming process. Further, the accuracy of the diagnoses primarily depends upon the expertise. Therefore, non-experienced dermatologists cannot make judgments with accuracy [4].

On the other hand, numerous problems arise in case of automatic detection of melanoma. At first, skin lesion has great intra-class variations and inter-class similarities in texture, color, and shape; different types of skin cancers possess high visual similarity [5]. Secondly, the region of skin lesions largely differ in dermoscopic images while the borders between normal skin and skin lesions remain unclear in certain images. Next, artifacts like rulers, texture, and hair are highly challenging to recognize in dermoscopic images in case of changes in melanoma. Likewise, many other factors increase the challenges involved in automated detection of melanoma from dermoscopic images [6]. So, there is an urgent need to develop an automated and non-subjective detection technique. This might help dermatologists to make accurate diagnoses. Such challenges inspire the researchers to propose new algorithms for diagnosis and visualization of melanoma. Computer-Aided Diagnosis (CAD) schemes assist in the diagnosis of melanoma. CAD system offers a user-friendly environment for inexperienced dermatologists [7]. The evidence produced by CAD diagnosis mechanism is utilized as a second opinion for melanoma diagnosis. In order to detect a skin lesion at early stages and resolve the abovementioned complexity, extensive studies have been conducted earlier with the help of Computer Vision (CV) algorithm [8]. The classification method varies such as Decision Trees (DT), Support Vector Machines (SVM), and Artificial Neural Networks (ANN). Conventional techniques are parametric in nature and need trainable datasets to achieve normal distribution. However, skin cancer dataset is uncontrivable [9]. Every lesion contains a distinct pattern; therefore, this technique is not sufficient. For this reason, DL technique is powerful in skin image classification as it can help the dermatologists to diagnose lesions with high accuracy [10].

The current study develops an Intelligent Arithmetic Optimization with Ensemble Deep Transfer Learning Based Melanoma Classification (IAOEDTT-MC) model. Initially, the proposed IAOEDTT-MC model applies image preprocessing at initial stage with the help of Gabor Filtering (GF) technique. In addition, U-Net segmentation approach is employed to segment the lesion regions in dermoscopic images. Besides, an ensemble of DL models including ResNet50 and ElasticNet models is applied. Moreover, AO algorithm with Gated Recurrent Unit (GRU) method is employed for both identification and classification of melanoma. The proposed IAOEDTT-MC approach was experimentally validated with the help of benchmark datasets and the results were examined under several measures.

2  Literature Review

In literature [11], the authors proposed an automatic skin lesion classification technique. In this technique, Transfer Learning (TL) and pretrained Deep Learning (DL) networks were used. Apart from data augmentation and fine-tuning, TL model is also employed for AlexNet by replacing the final layer with softmax function to categorize three distinct types of lesions such as atypical nevus, common nevus, and melanoma. In Li et al. [12], two DL techniques were proposed to address three major tasks that evolve in the region of skin cancer image processing such as lesion segmentation, lesion dermoscopic feature extraction, and lesion classification. A DL architecture, comprised of two Fully Convolutional Residual Network (FCRN), was presented to concurrently produce coarse classification and segmentation results. Lesion Index Calculation Unit (LICU) was designed earlier to refine the coarse classification result by evaluating distance heat map. A direct CNN was introduced in this study for dermoscopic feature extraction tasks.

The authors in the study conducted earlier [13] developed a DL-based technique that overcomes the limitation in automated detection and segmentation of melanoma from dermoscopic images. An encoder-decoder network, with decoder and encoder subnetworks, was developed in this study. This network is interconnected with a sequence of skip pathways that carries the semantic level of encoder feature map close to the decoder feature map for feature extraction and effectual learning. The scheme applies multi-stage-and-scale method and uses softmax classification for pixel-wise classification of melanoma lesions. Kaur et al. [14] developed an automated melanoma classification method based on Deep CNN (DCNN) for accurate classification of malevolent vs. benevolent melanoma. DCNN architecture can be well planned by organizing more than one layer that is accountable for deriving lower to higher-level features of the skin image in an exclusive manner. Other vital conditions in DCNN structure includes the selection of sizes and multiple filters that employ optimizing hyperparameters, proper DL layers, and selection of in-depth network.

Thapar et al. [15] suggested a reliable technique for skin lesion detection with the help of dermoscopic images. This aim of this technique is to enhance the visual observation and diagnostic ability of healthcare professionals in terms of differentiating benign from malignant lesions. Swarm Intelligence (SI) algorithm was utilized in this study for segmentation of Region of Interest (RoI) from dermoscopic images. Further, Speeded-Up Robust Feature (SURF) was applied for feature extraction of RoI which is determined as a better segmentation outcome, attained by Grasshopper Optimization Algorithm (GOA). Banerjee et al. [16] proposed a DL-based ‘You Only Look Once (YOLO)’ technique based on the application of DCNN to diagnose melanoma from digital and dermoscopic images. This technique also aimed at providing fast and accurate output compared to traditional CNN. With respect to position of the recognized object in a cell, this method forecasted the class confidence score and bounding box of the detected object. In literature [17], the authors addressed the problem of efficient usage of feature set extracted from DL model which is pre-trained on ImageNet.

3  The Proposed Model

In this study, a novel IAOEDTT-MC technique has been developed for identification and classification of melanoma on dermoscopic images. Initially, the proposed IAOEDTT-MC model applies image pre-processing using GF technique. Followed by, U-Net segmentation approach is employed to segment the lesion regions in dermoscopic images. Furthermore, an ensemble of DL models including ResNet50 and ElasticNet models is applied. At last, AO algorithm with GRU approach is employed for the identification and classification of melanoma. Fig. 1 depicts the overall process involved in IAOEDTT-MC approach.

Figure 1: Block diagram of IAOEDTT-MC approach

3.1 Image Pre-Processing

GF is a band-pass filter that is effectively executed for CV application and variations in image processing. Two-dimensional Gabor function is an orientated complex in which the sinusoidal grating gets decreased with the help of two-dimensional Gaussian envelope. In two-dimensional co-ordinate (a,b) system, GF comprises of a real component and imaginary one and are represented as follows [18]:

Gδ,θ,ψ,σ,γ(a,b)=exp(−a′2+γ2b′22σ2)×exp(j(2πa′δ+ψ))(1)

where

a′=acosθ+bsinθ(2)

b′=−asinθ+bcosθ(3)

In the above equation, δ indicates the wavelength of sinusoidal factor and θ denotes the orientated separation angle of Gabor kernel. Assume θ in the range of [0o,180o] as symmetry which makes other directions redundant. ψ determines phase offset, σ defines the Standard Deviation (SD) of Gaussian envelope and γ represents spatial feature (default value is 0.5) to identify the ellipticity of Gabor function support. The variable 0 is determined to be 6 and spatial frequency bandwidth bw is as follows.

σ=δpiln222bw+12bw−1(4)

3.2 U-Net Segmentation

In this study, U-Net segmentation technique is utilized for the segmentation of lesion region in dermoscopic image. U-net network is classified as follows: initially, the contracting path uses a standard CNN structure [19]. All the blocks in contracting path comprise of two consecutive 3 × 3 convolutions along with max-pooling layer and ReLU activation unit. This procedure is repeated iteratively. U-net is innovated in the upcoming section and is named as expansive path in which all the phases upsample the feature maps using 2 × 2 up-convolutions. Next, the feature map from respective layer in the contracting path, can be cropped and concatenated onto up-sampled feature maps. It is followed by ReLU activation and two consecutive 3 × 3 convolutions. Finally, a 1 × 1 convolution is employed in the reduction of feature maps to necessary amount of channels and generation of the segmented images. Cropping is essential, because the pixel features in the edges have minimum amount of context data due to which it should be dismissed. This results in a u-shaped network and propagates the context dataset alongside the network. This scenario enables the user to segment objects in a region with the help of context from a large overlapping region.

3.3 Ensemble Learning Process

For feature extraction, an ensemble of DL models including ResNet50 and ElasticNet models is applied. Ensemble learning is a fusion method in which two or three models are fused together to gain benefits in terms of performance and computation. The outcomes of an ensemble of DNNs are always better compared to the outcomes achieved from a single model. In this study, average ensemble learning is utilized by a similar weight assigned to all the models.

P=∑MiN(5)

In this expression, Mi refers to the probability of model i, and N stands for overall number of models. DL model is complex in nature and possesses different architectures; they do not offer a similar outcome. Thus, it is advantageous to assign additional weights to the model for better implementation. Hereby, the maximal output is extracted from other models. The problem is to identify the accurate combination of model weights. Then, the grid search method is utilized to resolve the problem. In this study, the authors employed a total of 1000 weight combinations. The search process continues until each variety is checked. Finally, the algorithm is implemented by means of correct weight combination for the maximum of the presented assessment metrics.

3.3.1 ResNet-50 Model

ResNet is commonly used for improving the problem of vanishing or exploding gradients [20]. ResNet is a collection of residual blocks while each residual block is in turn a collection of different layers such as Batch Normalization (BN), convolution, and ReLU layers. In addition to all the residual blocks, the input is directly processed as the output using identity whereas a short connection permits the user to perform residual learning; this is important to resolve gradient problems in training deep network. The residual block is expressed as follows.

Hl=Hl−1+F(Hl−1)(6)

Here, Hl and Hl−1 signify the output and input of lth residual block correspondingly. F(x) denotes the residual mapping function of stack layers. It is apparent that the dimensions such as Hl−1 and F(Hl−1) are equivalent. But, the convolutional function generally has dimensional variations. Thus, the linear projection Ws is utilized to match the dimension. So, Eq. (6) is transformed as follows.

Hl=WsHl−1+F(Hl−1)(7)

Thus, ResNet-50 is attained by stacking the residual block so as to count the last network layer to 50.

3.3.2 ElasticNet Model

ElasticNet is basically a linear regression mechanism developed on the basis of Lasso and ridge regressions [21]. Assume that the sample size is N and the number of prediction parameters is p.

yi=β0+β1xi1+⋯+βpxip+Ei,Ei−N(0,σ2),i=1,2,⋯,N(8)

In Eq. (8), the regression coefficient is represented as β whereas the constant term is characterized by β0 and σ2 denotes a disturbance of target value around real value. The definition of Ridge regression is as follows.

β^ridge=argminβ{∑i=1N(yi−∑j=1pxijβj−β0)2+λ1∑j=1pβj2}(9)

Lasso regression is defined as follows.

β^ridge=argminβ{∑i=1N(yi−∑j=1pxiJ−β0)2+λ2∑j=1p|βj|}(10)

To break the limitation of two techniques, elastic net method is developed. Regression coefficient is represented as follows.

β^ridge=argminβ{∑i=1N(yi−∑j=1pxijβJ−β0)2+λ1∑j=1pβj2+λ2∑j=1p|βj|}(11)

If λ=λ1+λ2,α=λ2λ1+λ2:

β^ridge=argminβ{∑i=1N(yi−∑j=1pxijβj−β0)2+λ∑j=1p(α|βj|+(1−α)βj2)}(12)

The penalty function of the elastic net is ∑j=1p(α|βj|+(1−α)β2) i.e., a convex linear integration of the penalty function λ2∑j=1p|βj| of Lasso regression and λ1∑j=1pβj2 of ridge regression. If α=1, the elastic net is Lasso regression. If α=0, the elastic net is ridge regression. Thus, the elastic net has the advantages of both ridge and Lasso regressions. It can determine which parameters to be selected so that a better group effect can be achieved.

3.4 Melanoma Classification

Finally, AO algorithm with GRU model is employed for identification and classification of melanoma. RNN is a well-known neural network that can process sequential datasets. This characteristic of RNN makes it relevant to learn the algorithmic tasks [22]. Though RNN is utilized for different NLP applications, it has a primary constraint i.e., if suffers from gradient vanishing in deep network. For sequential datasets (x1,x2,x3,…,xt), the hidden state ht of the RNN is evaluated by the formula given below.

ht=f(ht−1,xt)(13)

In Eq. (13), f refers to the activation function. GRU is a kind of RNN model with less number of gates than LSTM. In GRU cell units, both forget gate and input gate can be controlled by a single gate. Therefore, both input and forget gates are integrated into a single gate which makes the GRU a simple technique than LSTM. For instance, when zt=1, the entry of the novel dataset for forget gate is opened, and the input gate is closed, if zt=0. The reset gate defines that the novel input should be integrated with the preceding memory in order to determine the novel state as given below.

rt=σ(wxrTx(t)+whrO(t−1)T+br)(14)

zt=σ(wxzTx(t)+woTzo(t−1)+bz)(15)

ot=zt⊗0t−1+(1−zt)⊗O~t(16)

In these expression, rt refers to reset gate; zt indicates the update gate; and ot represents the output gate. ⊗ represents component-wise multiplication; t denotes the time step; T refers to the length of window; w indicates the layer weight demonstrating input x, and b signifies the threshold of output gate.

AOA comprises of exploitation, and exploration stages, inspired by arithmetical operations, like −,+, *, and /. Initially, AOA produces a set of N solutions [23]. Thus, the agent or solution signifies X population in the following equation.

X=[xN−1,1xN,1x2,1x1,1xN−1,jxN,jx2,jx1,jxN,n−1x1,n−1xN−1,nxN,nx2,nx1,n]

X=[x1,1⋯x1,jx1,n−1x1,nx2,1⋯x2,j⋯x2,n⋯⋯⋯⋯⋯⋮⋮⋮⋮⋮xN−1,1⋯xN−1,j⋯xN−1,nxN,1⋯xN,jxN,n−1xN,n](17)

Followed by, the fitness function of the solution is evaluated to identify the finest one ,Xb. Then, AOA executes exploration or exploitation method according to Math Optimizer Accelerated (MOA) values. Subsequently, MOA is upgraded as given below.

MOA(t)=Min+t×(MaxMOA−MinMOAMt)(18)

Here, Mt indicates the total quantity of iterations. MaxOA and MinMOA indicate the maximal and minimal values of the accelerated operation while multiplication (M) and division (D) are exploited in the exploration process of AOA:

Xi,j(t+1)={Xbj÷(MOP+∈)×((UBj−LBj)×μ+LBj),r2<0.5Xbj×MOP×((UBj−LBj)×μ+LBj),otherwise(19)

Then, e denotes the small integer value,UBjandLBj indicates upper and lower bounds of the search space at j-th variable. μ=0.5 represents the control function. Moreover, Math Optimizer (MOP) is defined as follows:

MOP(t)=1−t1/αMt1/α(20)

α=5 indicates the dynamic parameter that determines the performance of exploitation phase. Fig. 2 depicts the flowchart of AOA.

Figure 2: Flowchart of AOA

Furthermore, subtraction (D) and addition (A) operators are also applied in the implementation of AOA exploitation process as given below.

xi,j(t+1)={Xbj−MOP×((UBj−LBj)×μ+LBj),r3<0.5Xbj+MOP×((UBj−LBj)×μ+LBj),otherwise(21)

Here, r3 indicates an arbitrary value that lies in the interval of [0, 1]. Later, the agent update process is also implemented using AOA operator. Basically, Algorithm 1 illustrates the steps involved in AOA.

:

4  Performance Validation

The proposed EAOEDTT-MC model was experimentally validated using two benchmark datasets namely, ISIC 2017 [24] and ISIC 2020 [25]. The proposed EAOEDTT-MC approach was simulated with the help of Python 3.6.5 tool. The datasets has two class labels namely, melanoma (MEL) and benign (BEN) as shown in Table 1. A few sample images are portrayed in Fig. 3.

Figure 3: Sample images

Fig. 4 highlights the confusion matrices generated by the proposed EAOEDTT-MC model on test ISIC 2017 dataset. On entire dataset, the proposed EAOEDTT-MC model classified 1,452 samples under MEL class and 2,389 samples under BEN class. Eventually, on 70% of TR dataset, EAOEDTT-MC approach categorized 1,010 samples under MEL class and 1,679 samples under BEN class. Concurrently, on 30% of TS dataset, the proposed EAOEDTT-MC system recognized 442 samples under MEL class and 710 samples under BEN class.

Figure 4: Confusion matrices of EAOEDTT-MC approach under ISIC 2017 dataset (a) entire dataset, (b) 70% of TR data, and (c) 30% of TS data

Table 2 and Fig. 5 shows a brief melanoma classification results achieved by the proposed EAOEDTT-MC model on test ISIC 2017 dataset. The experimental results imply that the proposed EAOEDTT-MC model achieved effectual outcomes in every aspect. For instance, with entire dataset, EAOEDTT-MC model attained an average accuy of 92.07%, precn of 93.06%, recal of 90.87%, Fscore of 91.64%, AUCscore of 90.87%, and an MCC of 83.90%. In addition, with 70% of TR data, the proposed EAOEDTT-MC technique obtained an average accuy of 95.12%, precn of 95.23%, recal of 95.16%, Fscore of 95.12%, AUCscore of 95.16%, and an MCC of 90.39%. Also, with 30% of TS data, EAOEDTT-MC system accomplished an average accuy of 92.09%, precn of 93.08%, recal of 90.86%, Fscore of 91.65%, AUCscore of 90.86%, and an MCC of 83.91%.

Figure 5: Average analysis of EAOEDTT-MC method under ISIC 2017 dataset

Training Accuracy (TA) and Validation Accuracy (VA) values, attained by the proposed EAOEDTT-MC approach on ISIC 2017 dataset, are demonstrated in Fig. 6. The experimental outcomes reveal that the proposed EAOEDTT-MC algorithm gained maximum TA and VA values. To be specific, VA seemed to be higher than TA. Next, Fig. 7 shows the training loss (TL) and validation loss (VL) analysis of the proposed model. The results indicated that the proposed model has offered minimal loss with an increase in epochs.

Figure 6: TA and VA analysis of EAOEDTT-MC method under ISIC 2017 dataset

Figure 7: TL and VL analysis results of EAOEDTT-MC method under ISIC 2017 dataset

Table 3 and Fig. 8 shows the comparative study results accomplished by the proposed EAOEDTT-MC method and other existing techniques on ISIC 2017 dataset [26]. The experimental values indicate that ResNet18, Inception v3, and AlexNet models obtained the least classification performance over other models. At the same time, LCNet model reached a moderately improved classification performance while it accuy, precn, and recal values were 88.20%, 78.50%, and 87.80% respectively.

Figure 8: Comparative analysis results of EAOEDTT-MC method under ISIC 2017 dataset

5  Conclusion

In this study, a novel IAOEDTT-MC approach has been developed for identification and classification of melanoma on dermoscopic images. Initially, the proposed IAOEDTT-MC model applies image pre-processing using GF technique. Followed by, U-Net segmentation approach is followed to segment the lesion regions in dermoscopic image. Furthermore, an ensemble of DL models including ResNet50 and ElasticNet models is applied. At last, AO algorithm with GRU algorithm is utilized for identification and classification of melanoma. The design of AO algorithm involves the integration of LAHC concept with traditional AOA in order to enhance the quality of the solution. The proposed IAOEDTT-MC system was experimentally validated utilizing benchmark datasets and the outcomes were inspected under distinct measures. The extensive comparative analysis outcomes highlighted the enhanced performance of IAOEDTT-MC model over recent algorithms. In future, the performance of IAOEDTT-MC model can be enhanced with the help of hybrid metaheuristic algorithms with deep instance segmentation models.

Funding Statement: This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ICAN (ICT Challenge and Advanced Network of HRD) program (IITP-2022-2020-0-01832) supervised by the IITP (Institute of Information & Communications Technology Planning & Evaluation) and the Soonchunhyang University Research Fund.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.