Electromyogram Based Personal Recognition Using Attention Mechanism for IoT Security

1  Introduction

Recently, with the development of IoT technology, cloud services have been widely used, and various devices are equipped with network functions to connect communication between machines and people, as shown in Fig. 1 [1]. Accordingly, information security has become vital as user information is used to control various devices [2]. Confidentiality and integrity, among the critical elements of information security, mean giving authorized user’s permission to disclose and modify information [3]. A personal recognition process is required to process such confidentiality and integrity. Recognizing an existing user uses unique features such as facial recognition, fingerprints, etc. However, this information can also be collected without the user’s consent and can be forged, causing security problems [4]. In order to solve this problem, a study is being conducted that incorporates biosignals as a method for personal recognition in IoT devices [5].

Figure 1: Example of connection communication between machine and people

Biosignals are attracting attention as a next-generation personal recognition method due to the advantage of being impossible to forge, and include EMG, electrocardiogram (ECG), and electroencephalogram (EEG). The EMG, among other biosignals, is a signal collected by measuring the microcurrent occurring when muscle moves, and shows composite information reflecting nerve control information and physiological information of the muscle tissues [6]. Additionally, because the signal changes along with performed gestures, EMG can solve the problem of non-cancelable schemes, an existing personal recognition biometrics issue. Methods for measuring EMG include an invasive method and a non-invasive method. Because a needle electrode is inserted into a muscle for measurement in the invasive method, less noise occurs. However, because it entails pain when the needle electrode is inserted, this measuring instrument is not preferred, and it is hard to apply this method for personal recognition. For the non-invasive method, the electrode is affixed to the user’s skin for measurement; thus it is easier to measure EMG than the invasive method. The EMG measured in the non-invasive method is called surface EMG (sEMG) [7]. sEMG has non-periodic characteristics, and it is impossible to obtain sEMG (gesture and rest sections) of the same waveform from the same subject as shown in Fig. 2. However, the existing personal recognition method using sEMG extracts features with the same weight in both the gesture and rest sections of the signal without considering non-periodic characteristics. When using such sEMG signals, data irregularities decrease personal recognition performance [8]. Therefore, a feature extraction study considering the non-periodic characteristics of sEMG is required.

Figure 2: Exemplary irregularity of measured sEMG

This paper proposes a personal recognition method that combines the siamese fusion network with an auxiliary classifier to solve the data irregularity of sEMG signals [9]. Existing personal recognition studies using sEMG did not consider data irregularities. The proposed method extracts feature with EMD to analyze sEMG signals in the time-frequency domain. The model designed to solve the issue with the existing personal recognition method comprises two sub-networks and one decision network. The two sub-networks are designed in the siamese structure and output the compatibility function to calculate the similarity of input data pairs. The decision network calculates features in the compatibility function entered using a convolutional neural network (CNN). In addition, the compatibility function is computed using euclidean distance (ED) as an auxiliary classifier. The information gathered using the auxiliary classifier is combined with the information of the CNN to apply the attention mechanism. In order to solve the irregularity problem of sEMG, the proposed method applies an attention mechanism by calculating a one-dimensional vector with ED with a small amount of computation.

The proposed personal recognition method based on a siamese fusion network combined with an auxiliary classifier achieved a classification accuracy of 94.35% for 100 subjects of CU_sEMG DB. The performance was 2.34% better than a method not using the auxiliary classifier. Moreover, this paper confirmed the versatility of the proposed method, as combining auxiliary classifiers with existing siamese network-based personal recognition methods improved accuracy by more than 3%. A comparative experiment was conducted using the Ninapro DB2 to examine the superiority of the proposed method. The proposed personal recognition method showed better recognition performance than the existing method (max: 5.28%, min: 0.85%), thus proving its superiority. Therefore, as a method that is robust to data irregularities in sEMG, it was confirmed that the proposed method can be used as a method to compensate for data irregularities in the personal recognition method using sEMG.

2  Conventional Personal Recognition Methods Using sEMG

The personal recognition method based on sEMG uses personal differences that occur due to muscle development levels, activity, and habits when the same gesture is conducted. sEMG for personal recognition measures signals occurring in different muscles of a body, e.g., arms, legs, and lips. Example feature extraction methods for sEMG data include using handcrafted features in the time and frequency domain and using features created by a neural network.

2.1 Handcrafted Feature Extraction Based on Domain Information

Handcrafted features do not need to be analyzed with a complex equation, resulting in a small amount of operation, thus fast processing speed. Therefore, the method is widely used in personal recognition studies based on sEMG. Yamaba et al. [7] used handcrafted features for personal recognition. He used DL-3100 and DL-141 to measure single channel sEMG of 11 subjects when they conducted 12 hand gestures, and used the nine gestures in which more obvious signals, than for the other gestures, were created in the measured sEMG in his experiment. He divided sEMG into ten segments to extract 11 handcrafted features including the sum, mean, skewness, and standard deviation (SD) in each piece. He classified the subjects with the support vector machine (SVM) and dynamic time warping (DTW).

Li et al. [10] studied two-factor personal recognition with sEMG measured when subjects drew their unlock pattern on their smartphone screen. OpenBCI was used to measure sEMG in flexor digitorum superficialis. The ten subjects drew their two open patterns while seated and relaxed in a chair. The 5 Hz high-pass filter (HPF) and the 60 Hz notch filter (NF) were used to remove baseline wander and powerline interference from the measured sEMG. Eleven handcrafted features of mean absolute value (MAV), variance (VAR), waveform length (WL), and zero crossing (ZC) were extracted from the time domain, and personal recognition achieved 98.2% of accuracy by using the one-class SVM (OCSVM) and the local outlier factor (LOF).

Lee et al. [11] studied personal recognition using sEMG measured from muscles other than hands. Lee studied personal recognition based on personal differences using personal walking habits, e.g., pace, speed, and muscle size. He used 11-channel Biopac MP150 sEMG from nine muscles in the left leg and two muscles in the right leg, including the sartorius, biceps femoris, and tibialis anterior muscles. The sEMG of 12 subjects was measured while walking, and interference was removed using a 20–450 Hz band-pass filter (BPF) and 60 Hz NF. Five-time and frequency domain features including root mean square (RMS), MAV, and dominant frequency (DF) were extracted, and the linear discriminant analysis (LDA) was used to classify the subjects at 93% of accuracy.

Table 1 summarizes these three personal recognition studies using handcrafted features of sEMG. The time domain uses the morphological features of sEMG, and the frequency domain uses the domain transformation information of sEMG. However, it can be impossible for handcrafted feature extraction to extract optimized data features [12].

2.2 Feature Extraction Using Neural Networks

The neural network feature extraction approach is designed to solve the disadvantages of handcrafted feature extraction, and extract optimized features from data adaptively unlike the conventional method for conducting features engineering manually [13]. Wong et al. [14] used a sEMG-based neural network to study personal recognition. He measured eight-channel sEMG using the MYO armband, and set the sampling rate at 200 Hz. The measured sEMG was then divided using a three-second window, and interference in the signal was removed using a 100 Hz low pass filter (LPF) and 50~60 Hz NF. Personal recognition was performed using an artificial neural network (ANN) composed of three hidden layers (30, 20, and 10) to achieve 98.09% accuracy.

Lu et al. [15] used an armband to measure eight-channel sEMG when 21 subjects opened their hand, and the sensor was arranged/positioned on the right forearm. The sampling rate was set as 200 Hz and the gesture was repeated 30 times. The measured sEMG was transformed into a continuous wavelet transform (CWT) image of 32 * 300 to use the time-frequency features. The CNN was designed to have four convolution layers and a pooling layer. The first three convolution layers were a filter of 3 * 3 and the fourth convolution layer was a filter of 2 * 6. The rectified linear unit (ReLU) was used for the activation function, and 21 subjects were recognized to achieve 99.2% accuracy.

Kim et al. [16] used both handcrafted features and a neural network. He used the sEMG measured from the thigh muscle when the subjects were walking to study personal recognition. In that study, when 20 subjects were walked along a corridor while a sensor was attached to their rectus femoris, vastus medialis, vastus lateralis, and semitendinosus muscles, the signal was measured using Biopac Mp150. Twelve handcrafted features including RMS, MAV, and VAR were extracted from the measured sEMG. An ANN composed of 30 hidden layers was used for personal recognition to achieve 99.7% accuracy. Table 2 summarizes these three personal recognition studies using neural networks.

2.3 Similarity Learning of Data Pairs Using Siamese Structures

Although an ordinary neural network needs a lot of data, the amount of usable data is limited. Moreover, because new users always join and leave the user group with users registered for personal recognition, implying its dynamic property, there is an issue of learning again because it is necessary to learn the model again when the user group changes. The siamese network for solving this disadvantage is a representative few-shot learning method that only requires a little data when learning a model. The siamese network compares similarity of input data sets to recognize users and does not need to learn the model again even if a new user is added [13].

Lu et al. [17] measured and used sEMG using an MYO armband when 21 subjects opened their hand. An eight-channel sensor was arranged at the height of the radiohumeral joint of the right arm of the subjects. The siamese network for personal recognition comprised four convolution layers and a pooling layer. The activation function was ReLU, and 100 data pairs randomly combined in each subject were used for siamese network learning. The positive/negative data ratio was 1:1, and the output of two sub-networks was calculated using ED. The sEMG of four new subjects was measured after learning the siamese network model, and the personal recognition experiment was performed to show an accuracy of 97.76%.

Fan et al. [13] studied siamese network-based personal recognition with sEMG measured when 40 subjects held and used their smartphone. He measured eight-channel sEMG using the MYO armband, and used the data augmentation technique. The siamese network designed with three convolution layers with different filters was used, and the first convolution layer used the 8 * 1 filter to learn features between channels. The activation function was ReLU, and dropout was used to prevent overfitting. The output difference between two sub-networks was calculated using ED, and a personal recognition experiment was performed to show an accuracy of 92.06%.

Zhong et al. [18] used the siamese network to experiment with a palm pattern-based personal recognition. He designed VGG-16 in siamese structure, composed of five convolution layers and five pooling layers to use the learned network. Fully connected layer (FC) 2 and FC3 were transfer-learned. When learning them, data pairs were randomly selected from the model, and the ratio of positive/negative was decided to be 1:2. The XJTU palm pattern data created in a familiar environment was used to conduct the experiment and 4.56% equal error rate (EER) was shown.

Table 3 summarizes these three similarity learning studies using the existing siamese network and personal recognition to recognize new users not used in a model learning with a small amount of data. Two sub-networks created the compatibility function of the same size and compared similarity in the data pairs entered into the model. ED or FC was used to compare similarity. However, personal recognition studies using the sEMG did not consider data irregularity, and extracted features with the same weight. Such a method not considering data irregularities lowers personal recognition performance [8].

3  The Proposed Personal Recognition Method Using sEMG Based on the Attention Mechanism

This chapter describes a method for solving the sEMG irregularities and the personal recognition method to which the proposed method is applied [9]. Fig. 3 shows proposed personal recognition method flowchart based on a siamese fusion network combined with an auxiliary classifier.

Figure 3: Flowchart of the proposed personal recognition method

For using sEMG to recognize users when they conduct a gesture, interference included in the signal is first preprocessed using a filtering technique. To extract time-frequency domain features in the data, the sEMG signal is processed using EMD, and extracted in the unit of intrinsic mode function (IMF). The IMF is used as input data in the siamese fusion network combined with an auxiliary classifier designed to have data pairs of positive and negative to learn the siamese model. This model comprises sub-networks designed in the siamese structure and does not require learning again even if a new user is added to the system. A decision network is combined with the proposed auxiliary classifier to solve the signal irregularity. The two sub-networks sharing a weight calculate the compatibility function in the input data pairs, and the computed information is transferred to the decision network. The auxiliary classifier of the decision network calculates the compatibility function using ED, and is combined with a neural network composed of the CNN to apply the attention mechanism. Finally, it learns the model through similarity calculation to recognize users.

3.1 Feature Extraction Using EMD

sEMG can be measured using a non-invasive method without pain, implying an easy measurement of signals. However, electrodes are attached at muscle locations on the skin to measure signals, causing skin impedance. Moreover, the measured sEMG signal can be corrupted due to various factors such as power line interference, white gaussian noise, and baseline wander [19]. Therefore, NF is used at 60 Hz to remove power line interference before extracting features in this paper. In addition, the BPF is used in 5–500 Hz having important information in sEMG [20].

EMD is a method for analyzing the signal in the time-frequency domain, and decomposing it to IMF and residuals through the sifting process as follow.

Step 1: Calculate local minima and local maxima in signal x

Step 2: Calculate the lower and upper envelopes using local extrema

Step 3: Calculate the average envelope m of the lower and upper envelopes

Step 4: Calculate the difference c between the signal x and the average envelope m

Step 5: Check if the calculated c satisfies the IMF conditions

IMF is calculated based on the unique vibrational structure of a signal. Residuals use a monotonic function, and reflect data trends [21]. The decomposed IMF has one vibration mode, the lower-order IMF shows high-frequency components of an input signal, and the high-order IMF shows low-frequency components [22]. The decomposed IMF satisfies the following two conditions, and IMF 1–4 contains information from the sEMG [23]. Fig. 4 shows the calculation process of sEMG when performing the sifting process. The extracted IMF is normalized to 1 * 8000 according to the input size of the network.

Figure 4: IMF decomposition process through a sifting process

Condition 1: The number of the extreme values and the number of zero crossings are the same or the difference is not greater than 1.

Condition 2: The bottom and top envelope mean must be 0 in the time series.

3.2 Siamese Fusion Network with Auxiliary Classifiers

The siamese network ensures generalized performance for data encountered for the first time and a data set composed of a small number. This is a method for learning the similarity of input data to compare a data pair composed of two data points through deep learning operations. The siamese fusion network combined with the auxiliary classifier proposed in this paper is shown in Fig. 5, and it comprises two sub-networks and one decision network. The sub-network comprises convolutional layers and designed as a siamese model. The decision network comprises an auxiliary classifier and a neural network to solve the sEMG irregularities. For the input data of the designed model, IMF feature pairs extracted using EMD are used. The two sub-networks in the designed model have the same layer structure, and learn the model. The output of the sub-networks is used as input data in the decision network. Additionally, the decision network uses the auxiliary classifier and the neural network to calculate the compatibility function. It then combines and compares them for similarity with the input data pair based on the attention mechanism.

Figure 5: Siamese fusion network structure combining an auxiliary classifier proposed for personal recognition

The designed network extracts features of IMF using a deep encoder and learns the network using the similarity. The designed network operates as follows:

Step 1: The data pairs (IMFs1, IMFs2) consisting of a set of IMF are used as input data of each stream for the designed network.

IMFs1=[IMF11,IMF21,IMF31,IMF41],IMFs2=[IMF12,IMF22,IMF32,IMF42](1)

Step 2: Deep encoder (α) converts input data into embedding space (β) using convolutional layers.

α: IMF→β(2)

Step 3: A linear layer (L) is applied to project the embedding information onto a one-dimensional (1D) vector, which is output as a compatibility function (F).

L: [β1,β2,β3,β4]→F(3)

Step 4: The compatibility functions (F1,F2) generated from the two sub-networks extract features (dF) with the neural network (DN) of the designed decision network.

DN: [F1,F2]→dF(4)

Step 5: The compatibility functions (F1,F2) are computed as auxiliary classifiers for the attention mechanism.

SN=|F1−F2|2(5)

Step 6: The network combines dF and SN to compute the loss function and train the model.

Fig. 6 shows the structure of the designed two sub-networks. The designed sub-networks use four streams and FC for projection onto a 1D vector. Each stream comprises ten convolution layers, four pooling layers, and five dropout layers, and is transformed into an embedding space. They are then combined through a linear layer.

Figure 6: Designed sub-network structure

The decision network structure designed to solve the sEMG irregularities is shown in Fig. 7. The designed network comprises a neural network composed of two convolution layers, pooling layers, dropout layers, and an auxiliary classifier for solving any data irregularities. The neural network is combined with sub-networks, and uses the convolution layers to extract features in the input compatibility function. The auxiliary classifier computes the compatibility function as ED for the attention mechanism. The attention mechanism using ED has fewer parameters than the existing method as it operates the 1D vector. It combines the neural network output with the auxiliary classifier output to calculate the similarity of a data pair.

Figure 7: Designed decision network structure

The attention mechanism is a method for modeling the importance of the learning data using weights [24]. Accordingly, the data irregularity problem may be solved by applying a high weight to the signal of the gesture area. However, a lot of parameter operation is required to design an additional neural network and the relationship with all indexes must be defined for the existing attention mechanism [25]. In this paper, to solve such a disadvantage, a method is proposed that applies the attention mechanism to the data entered using the auxiliary classifier. The attention mechanism using the proposed auxiliary classifier is a method for 1D vector operation by using ED, and has the advantage of needing less operation than the existing method. The attention mechanism method using the proposed auxiliary classifier is based on the non-periodic characteristics of sEMG. The rest section of the input sEMG is generally the same (close to the baseline), but the gesture section varies depending on the strength of the force transmitted to the muscles and the holding time. The non-periodic characteristic interval of the data can be calculated by the auxiliary classifier from the compatibility function calculated with the IMF set1 and IMF set2. If the interval with non-periodic characteristics (gesture section) is calculated as ED, a high value is output, and if the interval with periodic characteristics (rest section) is calculated as ED, a low value is output. The output of the auxiliary classifier is combined with the features computed from DN to give higher weight to the gesture section in sEMG. The proposed method solves the data irregularity problem by assigning a lower weight to the rest section based on non-periodic characteristics.

4  Experiment Results and Discussion

The experiment in this paper used the DB of sEMG measured from subjects in multi-sessions to consider signal variability that can occur in daily living. The used DB was CU_sEMG DB [26], an open sEMG DB created by the IT Research Institute of Chosun University. Biopac MP160 was used to obtain sEMG from the right arms of 200 subjects (98 male and 102 female subjects), 24.69 (19–70) years of age on average. The Ag/AgCl sensor was attached to the subject’s palmaris longus and extensor digitorum muscles. When measuring sEMG, the subjects performed 12 hand gestures. The CU_sEMG DB was obtained in three sessions to consider sEMG variability, and there were one or more days of time intervals between sessions. The DB was reviewed to avoid effects due to the way of creating a wrong sEMG DB by the testing person and subjects, and the signal from a subject measured incorrectly was excluded from the experiment.

The experiment to examine the performance of the proposed personal recognition method proves the superiority of the proposed personal recognition method by analyzing the performance of IMF and auxiliary classifier, and comparing it with the performance of the existing personal recognition method. The experiment was conducted using sEMG measured from 100 subjects in the CU_sEMG DB. The designed model conducted learning using ReLU, and set 0.001 of learning rate, 100 of epochs. This paper divided the sEMG signals into 70% for training and 30% for testing.

First, the personal recognition performance using the IMF features formulated in this paper was analyzed. In the experiment of personal recognition using IMF features, features were extracted using data pairs as inputs to a model composed of ED and two sub-networks, as shown in Fig. 6. The experimental result is shown in Fig. 8 where: S1 represents the sEMG DB measured in session 1, S2 in session 2, and S3 in session 3. When the siamese network was designed as a multi-stream structure, and IMF 1–4 was used, it showed an average recognition accuracy of 92.01% for 100 subjects. This improved performance by 1.83% when compared to the method using IMF 1–3 and residual and by 2.21% when compared to the method using IMF 1–3 and handcraft features [16]. In addition, it was confirmed that the accuracy was 3.21% higher than the method using the spectrogram and 5.16% higher than the method using repetition data of raw EMG. It showed a high personal recognition performance by processing EMG using EMD and IMF 1–4, which included important components in EMG [11].

Figure 8: Experiment result according to IMF

The the model performance using the decision network combined with the auxiliary classifier proposed in this paper to solve irregularities in data was also analyzed. The experiment used IMF 1–4 and the network shown in Fig. 5. Additionally, performance was proven by combining the auxiliary classifier with the method for using the existing siamese network [13,17] to examine the versatility of the proposed method. The experimental results are shown in Fig. 9 where S1 represents the sEMG DB measured in session1, S2 in session2, and S3 in session3. Personal recognition by the proposed method classified 100 subjects to achieve 94.35% of accuracy (blue line in Fig. 9), and performance was 2.34% better than the siamese network not using the auxiliary classifier. Better performance results from using the auxiliary classifier in the decision network to apply higher weights to the gesture area signal of sEMG. Moreover, the proposed personal recognition method showed better performance than the method for using the existing siamese network (red and green lines in Fig. 9). For proving the versatility of the proposed method, performance was examined after combining the auxiliary classifier with the existing siamese network-based personal recognition method. The experiment revealed better accuracy by 3.81% and 3.15% as shown in Fig. 9 when the proposed auxiliary classifier is combined with siamese networks. This results from improved performance by applying the proposed auxiliary classifier to the attention mechanism.

Figure 9: Experiment result of siamese fusion network combining auxiliary classifier

An experiment was conducted to compare the superiority of the proposed method to the existing personal recognition method for using sEMG. The comparative experiment included the method using handcrafted features [16], the method using the CNN [15,23], the method using the CNN-long short term memory (LSTM) [27], and the method using the siamese network [13,17]. A single session of CU_sEMG DB was used for the experiment. Fig. 10 shows the experimental results. The existing method using handcrafted features, CNN, and LSTM showed a significant decrease in accuracy when the number of subjects increased. The experiment using handcrafted features showed the lowest performance, with an accuracy of 73.4% for 100 subjects. The existing method using a CNN showed low personal recognition performance with an accuracy of 85.55% and 88.67% for 100 subjects. The existing method using a CNN-LSTM showed low personal recognition performance with an accuracy of 86.74% for 100 subjects. The existing method using the siamese network showed an accuracy of 90.36% and 91.94% for 100 subjects, confirming once again that the siamese structure is suitable even when there is insufficient data for learning the model, such as EMG. The personal recognition method using a multi-stream siamese fusion network combined with an auxiliary classifier proposed in this paper showed 94.35% accuracy in multi-session and 95.33% in single-session with 100 subjects. Therefore, performance was improved by applying the attention mechanism using the auxiliary classifier, and the superiority of the proposed method was confirmed through comparative experiments.

Figure 10: Experiment result of personal recognition method using CU_sEMG DB

In addition, to confirm the versatility of the proposed method, personal recognition accuracy was verified using Ninapro DB2 [28]. Fig. 11 shows these experimental results. As a result of the experiment, the personal recognition method using the multi-stream siamese fusion network combined with an auxiliary classifier proposed in this paper showed a 98.41% accuracy for 40 subjects, which indicates the highest performance level, just as in the case of using the CU_sEMG DB.

Figure 11: Experiment result of personal recognition method using Ninapro DB2

Fig. 12 shows a Grad-CAM result of the attention mechanism using the auxiliary classifier proposed in this paper to solve the data irregularities. Grad-CAM [29] shows the weight of input data by using a color bar ranging from blue to yellow, with the color closer to yellow, a top value, means a higher weight. The data (Fig. 12a) output from the neural network of the decision network, shows that the weight is not concentrated on the signal in the gesture area. However, the data (Fig. 12b) combining the auxiliary classifier for the attention mechanism shows that a high weight is given on the signal in the gesture area (Fig. 12c). Therefore, it was confirmed that data irregularities were solved by adaptively assigning higher weights to signals in the gesture area of sEMG.

Figure 12: Attention mechanism using the proposed auxiliary classifier

5  Conclusion

This paper proposed a personal recognition method using sEMG based on a siamese fusion network combined with an auxiliary classifier. It is required for confidentiality and integrity of IoT devices. The model proposed for personal recognition comprised two sub-networks and one decision network combined with an auxiliary classifier. The auxiliary classifier applied the attention mechanism to solve data irregularities. The experiment was conducted with the CU_sEMG DB and Ninapro DB2. The siamese fusion network combined with the auxiliary classifier proposed in this paper recognized 100 subjects of CU_sEMG DB with 94.35% accuracy to show better performance by 2.34% than the method not using the auxiliary classifier. As a result of checking with Grad-CAM, it was confirmed that the sEMG gesture section was given a higher weight and robust against data irregularities. CU_sEMG DB and Ninapro DB2 was used to examine the superiority of the proposed method when comparing the performance with the existing personal recognition method.

However, this paper does not consider the data fusion method, optimization, and noise removal. The proposed method can operate only in the siamese structure. In addition, the proposed method was tested with two EMG DB. Therefore, it is necessary to confirm whether it applies to other biosignals. In the future, it will be necessary to change the method of combining auxiliary classifiers so that they can operate in various models such as CNN and spiking neural networks (SNN) [30] and study other fusion methods such as score level fusion. It will be necessary to study noise removal technology according to the environmental change of sEMG and study data optimization technology of spectral regression discriminant analysis (SRDA) [31]. Lastly, we plan to apply the attention mechanism to ECG and EEG, and conduct a personal recognition study using multiple biosignals.

Acknowledgement: This paper presents the results of a study on the “HPC Support Project”, supported by the “Ministry of Science and ICT” and NIPA.

Funding Statement: This study was supported by the Basic Science Research Program through the National Research Foundation of Korea, funded by the Ministry of Education (Nos. NRF-2017R1A6A1A03015496, RS-2023-00249555).

Author Contributions: The authors confirm contribution to the paper as follows: study conception and design: J. S. Kim; data collection: J. S. Kim, S. Pan; analysis and interpretation of results: J. S. Kim; draft manuscript preparation: J. S. Kim, S. Pan. All authors reviewed the results and approved the final version of the manuscript.

Availability of Data and Materials: The CU_sEMG DB used to support the findings of the study have been deposited in IT Research Institute of Chosun University (http://www.chosun.ac.kr/riit). The Ninapro DB2 used to support the findings of the study have been deposited in Ninaweb (http://ninaweb.hevs.ch/).

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

References

1. T. V. Nguyen, T. Huynh-The, K. Shim and B. An, “On the performance of multihop D2D communications in cognitive IoT networks with energy harvesting: A deep learning approach,” in Proc. of ICCE-Asia, Seoul, Korea, pp. 1–4, 2020. [Google Scholar]

2. S. Fonyi, “Overview of 5G security and vulnerabilities,” The Cyber Defense Review, vol. 5, no. 1, pp. 117–134, 2020. [Google Scholar]

3. D. A. Hahn, A. Munir and S. P. Mohanty, “Security and privacy issues in contemporary consumer electronics [energy and security],” IEEE Consumer Electronics Magazine, vol. 8, no. 1, pp. 95–99, 2018. [Google Scholar]

4. D. K. Tak, “Mobile fingerprint information based image and pattern-lock cross convergence authentication method,” Journal of Information Technology and Applied Engineering, vol. 12, no. 2, pp. 17–26, 2022. [Google Scholar]

5. Z. Sun, X. Xi, C. Yuan, Y. Yang and X. Hua, “Surface electromyography signal denoising via EEMD and improved wavelet thresholds,” Mathematical Biosciences and Engineering, vol. 17, no. 6, pp. 6945–6962, 2020. [Google Scholar] [PubMed]

6. P. Shenoy, K. J. Miller, B. Crawford and R. P. N. Rao, “Online electromyographic control of a robotic prosthesis,” IEEE Transactions on Biomedical Engineering, vol. 55, no. 3, pp. 1128–1135, 2008. [Google Scholar] [PubMed]

7. H. Yamaba, S. Usuzaki, K. Takatsuka, K. Aburada and T. Katayama, “On a user authentication method to realise an authentication system using s-EMG,” International Journal of Grid and Utility Computing, vol. 11, no. 5, pp. 725–734, 2020. [Google Scholar]

8. C. Savur and F. Sahin, “Real-time American sign language recognition system using surface EMG signal,” in Proc. of ICMLA, Miami, FL, USA, pp. 497–502, 2015. [Google Scholar]

9. J. S. Kim, “A study on personal recognition using electromyogram based on multi-stream siamese fusion network combining auxiliary classifier,” Ph.D. dissertation, Department of Control and Instrumentation Engineering, Chosun University, Gwangju, Korea, 2022. [Google Scholar]

10. Q. Li, P. Dong and J. Zheng, “Enhancing the security of pattern unlock with surface EMG-based biometrics,” Applied Sciences, vol. 10, no. 2, pp. 541, 2020. [Google Scholar]

11. M. R. Lee, J. H. Ryu and I. C. Youn, “Biometric personal identification based on gait analysis using surface EMG signals,” in Proc. of ICCIA, Beijing, China, pp. 318–321, 2017. [Google Scholar]

12. J. S. Kim, M. G. Kim and S. B. Pan, “Two-step biometrics using electromyogram signal based on CNN-LSTM networks,” Applied Sciences, vol. 11, no. 15, pp. 6824, 2021. [Google Scholar]

13. B. Fan, X. Liu, X. Su, P. Hui and J. Niu, “EmgAuth: An EMG-based smartphone unlocking system using siamese network,” in Proc. of PerCom, Austin, TX, USA, pp. 1–10, 2020. [Google Scholar]

14. A. M. H. Wong, M. Furukawa, H. Ando and T. Maeda, “Dynamic hand gesture authentication using electromyography (EMG),” in Proc. of SII, Honolulu, HI, USA, pp. 300–304, 2020. [Google Scholar]

15. L. Lu, J. Mao, W. Wang, G. Ding and Z. Zhang, “An EMG-based personal identification method using continuous wavelet transform and convolutional neural networks,” in Proc. of BioCAS, Nara, Japan, pp. 1–4, 2019. [Google Scholar]

16. S. H. Kim, J. H. Ryu, B. H. Lee and D. H. Kim, “Human identification using EMG signal based artificial neural network,” Journal of the Institute of Electronics and Information Engineers, vol. 53, no. 4, pp. 142–148, 2016. [Google Scholar]

17. L. Lu, J. Mao, W. Wang, G. Ding and Z. Zhnag, “A study of personal recognition method based on EMG signal,” IEEE Transactions on Biomedical Circuits and Systems, vol. 14, no. 4, pp. 681–691, 2020. [Google Scholar] [PubMed]

18. D. Zhong, Y. Yang and X. Du, “Palmprint recognition using siamese network,” Biometric Recognition, vol. 10996, pp. 48–55, 2018. [Google Scholar]

19. X. Song, S. Guo, B. Gao and W. Wang, “Motion recognition of the bilateral upper-limb rehabilitation using sEMG based on ensemble EMD,” in Proc. of ICMA, Tianjin, China, pp. 1637–1642, 2014. [Google Scholar]

20. Y. Zhang, P. Xu, P. Li, K. Duan, Y. Wen et al., “Noise-assisted multivariate empirical mode decomposition for multichannel EMG signals,” Biomedical Engineering Online, vol. 16, no. 1, pp. 107, 2017. [Google Scholar] [PubMed]

21. B. Lee and S. K. Yoo, “Amplitude and phase analysis of the brain evoked potential about performing a task related to visual stimulus using empirical mode decomposition,” Korean Journal of the Science of Emotion & Sensibility, vol. 18, no. 1, pp. 15–26, 2015. [Google Scholar]

22. N. E. Huang, X. Chen, M. T. Lo and Z. Wu, “On Hilbert spectral representation: A true time-frequency representation for non-linear and nonstationary data,” Advances in Adaptive Data Analysis, vol. 3, pp. 63–93, 2021. [Google Scholar]

23. J. S. Kim, M. G. Kim, J. M. Kim and S. B. Pan, “Multi-stream CNN-based personal recognition method using surface electromyogram for 5G security,” Computers, Materials & Continua, vol. 72, no. 2, pp. 2997–3007, 2022. [Google Scholar]

24. Y. Hu, Y. Wong, W. Wei, Y. Du, M. Kankanhalli et al., “A novel attention-based hybrid CNN-RNN architecture for sEMG-based gesture recognition,” PLoS One, vol. 13, no. 10, pp. 0206049, 2018. [Google Scholar]

25. J. Ku, L. Shen and G. Sun, “Squeeze-and-excitation networks,” in Proc. of CVPR, SLC, Utah, USA, pp. 7132–7141, 2018. [Google Scholar]

26. J. S. Kim, C. H. Song, E. Bak and S. B. Pan, “Multi-session surface electromyogram signal database for personal identification,” Sustainability, vol. 14, no. 9, pp. 5739, 2022. [Google Scholar]

27. E. Velasquez, J. Cornelis, L. Omelina and B. Jansen, “Muscle classification via hybird CNN-LSTM architecture from surface EMG signals,” in Proc. of DSP, Rhodes, Greece, pp. 1–5, 2023. [Google Scholar]

28. M. Atzori, A. Gijsberts, C. Castellini, B. Caputo, A. G. M. Hager et al., “Electromyography data for non-invasive naturally-controlled robotic hand prostheses,” Scientific Data, vol. 1, no. 1, pp. 1–13, 2014. [Google Scholar]

29. R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh et al., “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” in Proc. of ICCV, Venice, Italy, pp. 618–626, 2017. [Google Scholar]

30. A. Zhang, Y. Niu, Y. Gao, J. Wu and Z. Gao, “Second-order information bottleneck based spiking neural networks for sEMG recognition,” Information Sciences, vol. 585, pp. 543–558, 2022. [Google Scholar]

31. M. J. Islam, S. Ahmad, F. Haque, M. B. I. Reaz, M. A. S. Bhuiyan et al., “Application of min-max normalization on subject-invariant EMG pattern recognition,” IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 2521612, 2022. [Google Scholar]