Modified Wild Horse Optimization with Deep Learning Enabled Symmetric Human Activity Recognition Model

1  Introduction

Human activity recognition (HAR) is a process of finding human activity correctly (standing, working, walking, and eating) by examining sensor information accumulated by the Internet of Things (IoT) gadgets. It helps to understand the human behavioural paradigms in an IoT platform [1]. This study had the main focus on HAR in indoor atmospheres. The indoor HAR systems gain more significance in numerous fields, like body motion analysis in sports, assisted living, and healthcare, monitoring safety (injuries, collisions, and falls) in the IoT environments, biometric user identification for security, assessing employee performances in smart factories for Industry 4.0, and wellbeing in smart homes [2]. Activity recognition was an important indicator of lifestyle, participation, and quality of life.

Different symmetric and asymmetric activities are shown in Fig. 1. Symmetric activities are activities which utilizes both sides of the body in a mirror-like way. For example, standing, sitting, walking, jogging, cycling, etc. Asymmetric activities, on the other hand, involve the use of one side of the body more than the other. For example, punching, kicking, pushing, reading, etc. Both symmetrical and asymmetrical activities are important for human development and can offer a range of health benefits. Symmetrical activities can enhance balance and coordination, while asymmetrical activities can increase strength and endurance in specific muscle groups.

Figure 1: Different symmetric and asymmetric activities

Human actions carry more data relating to the context (a person’s mental state, identity, and personality) and assist mechanisms in reaching context awareness [3]. In the same way, therapists and rehabilitation specialists can benefit remotely from data on patient activities outside of a medical centre. To reply to the queries of when and where users perform which kinds of actions, a wide range of analyses (activities by day of the week, age group, gender, etc.) can be executed. It could assist in finding the abnormality in surveillance systems, thereby thwarting undesirable consequences [4]. By utilizing wearable sensors, HAR applications find the user’s action to offer intelligent personal recommendations and assistance. In the border security force, it was significant to detect the armed forces’ activities to offer feedback to the managers that helps them practically. Thus, HAR serves a significant role [5] in numerous effective computational mechanisms.

There were several difficulties in HAR. For instance, biometric user recognition uses HAR techniques to capture the individual behaviour of persons [6], like motion capture signs, as biometrics was a science where the potential for identifying a person depends on their characteristics for preventing device accessibility without authorization, was learned [7]. Currently, the basis of biometric detection mostly includes the person’s physiological properties. But, strong concerns about HAR and privacy were posed by such physiological features, which can be regarded as a possible substitute, working only as a system for behavioural biometrics [8]. The time sequence classifier tasks were the main difficulties in utilizing HAR, which is if individual movements were estimated using sensory information. This normally includes precisely engineering features from the basic information through signal processing methods and deep domain expertise for fitting one of the methods of machine learning (ML) [9]. Recently, deep learning (DL) approaches, which include LSTM and CNN, automatically derive useful features from the raw sensor information and get an advanced outcome [10].

This study develops a Modified Wild Horse Optimization with Deep Learning Enabled Symmetric Human Activity Recognition (MWHODL-SHAR) model. The major intention of the MWHODL-SHAR model lies in identifying symmetric activities, namely jogging, walking, standing, sitting, etc. In the presented MWHODL-SHAR technique, the human activities data is pre-processed in various stages to make it compatible for further processing. A convolutional neural network with an attention-based long short-term memory (CNN-ALSTM) method is applied for activity recognition. The MWHO model is utilized as a hyperparameter tuning strategy to improve the detection rate of the CNN-ALSTM algorithm. The experimental validation of the MWHODL-SHAR technique is simulated utilizing benchmark datasets.

The rest of the paper is organized as follows. Section 2 offers the literature review, and Section 3 presents the proposed model. Next, Section 4 provides performance validation and Section 5 concludes the work.

2  Literature Review

Basset et al. [11] introduced a supervised dual-channel method with LSTM, followed by an attention system for temporal fusion of inertial sensor data synchronized with residual convolution networks. The author even presents an adaptive channel-squeezing function for fine-tuning CNN feature-extracting ability by exploiting multi-channel dependency. The authors in [12] devise a Lightweight DL method for HAR demanding minimum computational power, making it appropriate for deployment on edge devices. The efficiency of the presented method was tested on the 6 day-to-day activities data of the participants.

Khan et al. [13] proposed a hybrid technique integrating LSTM and CNN for activity recognition. CNN was employed for extracting spatial features, and LSTM was used to learn temporal data. Nafea et al. [14] present an innovative technique using CNN with changeable kernel dimensions and bi-directional LSTM (BiLSTM) for capturing features at several resolutions. This study efficiently extracts spatial and temporal features from sensor data using conventional BiLSTM and CNN and the optimal selection of video representations.

In [15], a new HAR method that uses the potential of wearable gadgets with the skills of DL approaches was offered for identifying an individual’s day-to-day activities at home. The sensor will be integrated with a CNN designed to make inferences with the minimal possible resources to keep open the way of its application on embedded devices or low-cost. Gumaei et al. [16] devise an effective multi-sensors-oriented structure for HAR utilizing a hybrid DL technique, which integrates the simple recurrent unit (SRU) with the GRU of NNs. In [17], an intellectual auto-labeling method related to deep Q-network (DQN) was formulated with a new distance-related reward rule which could enhance learning performance in IoT platforms. A multi-sensor-related data fusion system was formulated to seamlessly compile the on-body, personal profile, and context sensor datasets. An LSTM-oriented classifier technique was modelled to find a finely-grained paradigm per the higher-level feature derived from the sequential motion information.

3  The Proposed Model

In this study, we have introduced an automated symmetric activity recognition model named MWHODL-SHAR technique. The MWHODL-SHAR model aims to detect and classify symmetric activities such as jogging, walking, standing, and sitting. In the presented MWHODL-SHAR model, three stages of operations were involved, namely pre-processing, activity recognition, and parameter tuning. Fig. 2 shows the workflow of the MWHODL-SHAR model.

Figure 2: Working process of the MWHODL-SHAR model

3.1 Pre-processing

Initially, the data recorded by the wearable sensor is cleaned and normalized to obtain appropriate and consistent data to train a detection module.

•   Missing values of the sensor dataset are fixed by the imputation method with the linear interpolation model;

•   Noises are eliminated with the median filter and a 3rd order low-pass Butterworth filter with a 20 Hz cut-off frequency.

•   A normalization technique transforms every sensor information with standard derivation and means [18]. The input for model training and feature extraction are normalized and cleaned.

3.2 Symmetric Activity Recognition Model

This study employs the CNN-ALSTM model for accurate symmetric activity recognition [19]. The CNN method is a highly useful NN technique from the human neural system and exhibits remarkable efficacy in numerous applications. The feature of CNN comprises shared weight and sparse connectivity. The CNN is a hierarchical module that successively implements 2 computational layers (convolution and pooling or sub-sampling layers) and the last classification through the FC layer. The convolution layer extracts feature from the input via the sliding window that realizes the feature map that expresses the temporal arrangement features of the time sequence dataset. The last FC layer produces the CNN output. Fig. 3 demonstrates the architecture of the LSTM method.

Figure 3: Flowchart of WHO algorithm

LSTM, a distinct type of RNN that learns long-term dependency, is intended to resolve problems via short-term memory. LSTM can process long sequence datasets without gradient disappearing; currently, it is extensively applied to resolve series dataset problems, namely speech recognition, NLP, and automated annotation of images. LSTM has a complicated recurrent module in an individual cell that is successively interconnected to time. The LSTM has 2 most important characteristics, the cell state C(z), which enables the preservation of memory in the long term and the hidden state H(z), which adapts with time. The LSTM might remove or add information from the cell state [19]. The O(z) output gate determines the exit according to the C(z) cell state.

F(z)=σ(Wf[H(z−1),X(z)]+Bf),(1)

I(z)=σ(Wi[H(z−1),X(z)]+Bi),(2)

O(z)=σ(Wo[H(z−1),X(z)]+Bo),(3)

I(z)=tanh(Wi[H(z−1),X(z)]+Bi),(4)

C(z)=F(z)⋅C(z−1)+I(z)⋅I(z),(5)

H(z)=O(z)⋅tanh(C(z)),(6)

From the expression, B and W correspondingly denote the vector of bias and weight matrices; tanh⁡(⋅) denotes the hyperbolic tangent function, and σ(⋅) indicates the sigmoid function. Note that the accurate control of the input dataset and internal state, viz., reflected in the cell state from the LSTM feature, is proceeded with: the fixed length data and variable are treated at the exit and entrance. This benefit is predominant in the case of utilizing LSTM incorporated into distinct types of DL models than applying LSTM. The attention module is a brain signal-processing method peculiar to human vision. Human vision rapidly scans the global images to attain the target region, which requires attention and ignores other regions of irrelevant data. The attention module was implemented and employed to model training and other related fields. The presented method employs the LSTM hidden layer output vector H={h1, h2,…, ht} as the input of the attention module, and they find the attention weight αi of hi, which is evaluated by

ei=tanh(Whhi+bh)(7)

αi=exp(ei)∑i=1texp(ei)(8)

In Eq. (8), Wh denotes the weight matrices of hj, and bh represents the bias.

3.3 MWHO-Based Hyperparameter Optimization Model

The MWHO algorithm is utilized as a hyperparameter tuning strategy to optimize the detection rate of the CNN-ALSTM algorithm [20,21]. The WHO approach was based on the characteristics of the social living of wild horses. They live mainly in herds with stallions and numerous mares and foals [20]. They have demonstrated different properties, such as commanding, mating, grazing, pursuing, and dominating. The key procedure included in the WHO is described as follows. Initially, the first population is subdivided into various groups. All the groups hold a leader (stallion), and the remaining population (mares and Foals) are equally dispersed. The grazing nature is determined by:

Xi,Gj=2Zcos(2πRZ)×(Stallionj−Xi,Gj)+Stallionj(9)

In the expression, Xi, Gj denotes the existing foal’s location, R specifies a uniform stochastic number, and Z shows the adaptive process defined below:

P=R1→<TDR; IDX=(P==0);

Z=R2ΘIDX+R3→Θ(∼IDX)(10)

Now P signifies a vector encompassing 0 to 1, R1→ and R3→ specify arbitrary values within [0, 1], and R2 shows the arbitrary integer ranges from zero and one:

TDR=1−it×(1maxit)(11)

Now maxit shows the maximum iteration count. For developing the mating features of the horse, the foal drives from i swarm to the impermanent group, while a foal comes from the j swarm to a momentary group:

XG,KP=Crossover (XG,iq, XG,jZ)i≠j≠k,p=q=end,

Crossover=Mean(12)

In this work, the Stallion leads the swarm to the water hole, and they compete with each other for the water hole. The dominant swarm uses the water hole mainly, and the remaining group utilizes the water hole:

StallionGi¯={2Zcos(2πRZ)×(WH−StallionGi)+WH if R3>0.52Zcos(2πRZ)×(WVH−StallionGi)−WH if R3≤0.5(13)

StallionGi¯ denotes the leader’s next location, and WH denotes the water hole’s location. Next, the leader is chosen according to the fitness values in the subsequent phases as follows:

StallionGi¯={XG,i if cos t(XG,i)<cos t(StallionGi)StallionGi if cos t(XG,i)>cos t(StallionGi)(14)

Fig. 3 shows the flowchart of the WHO algorithm. The MWHO algorithm is derived using the oppositional-based learning (OBL) concept. The OBL method constitutes a unique opposition solution to the current solution [22] and even attempts to define the superior solution that leads to increasing convergence speed. The opposite (X0) of the real number (X∈[U, L]) was assessed by:

X0=U+L−X(15)

Opposite point: Assume that X=[X1, X2,…, XDim] indicates the point in a Dim-dimension search space, and X1,X2,…,XDim∈R and Xj[Uj,Lj]. Thus, the opposite point (X0) of X is as follows:

Xj0=UBj+Lj−Xj,wherej=1….D(16)

As per the values of the fitness function, the most useful two points (X0 and X) were selected, and the other was ignored. For minimizing the issue, if (X)≤f(X0), X was sustained; oppositely, X0 was sustained.

4  Results and Discussion

The proposed model is simulated using Python 3.6.5 tool on PC i5-8600k, GeForce 1050Ti 4 GB, 16 GB RAM, 250 GB SSD, and 1 TB HDD. In this section, the symmetric activity recognition of the MWHODL-SHAR model is tested using two datasets (https://www.kaggle.com/competitions/uci-har/data?select=UCI+HAR+Dataset+for+Kaggle; https://sipi.usc.edu/had/): the UCI HAR dataset and USC HAD dataset. The details relevant to these datasets are given in Table 1.

The confusion matrices of the MWHODL-SHAR model on the UCI HAR dataset are reported in Fig. 4. The outcomes demonstrated that the MWHODL-SHAR method had identified all the different types of symmetric human activities.

Figure 4: Confusion matrices of the MWHODL-SHAR model on the UCI HAR dataset (a) 60% of TR data, (b) 40% of TS data, (c) 70% of TR data, and (d) 30% of TS data

Table 2 offers an overall activity recognition performance of the MWHODL-SHAR method on the UCI HAR dataset. The MWHODL-SHAR model has proficiently recognized all the activities. For instance, on 60% of TR data, the MWHODL-SHAR model has attained an average accuy of 99.29%, precn of 97.80%, recal of 97.81%, Fscore of 97.80%, and AUCscore of 98.69%. Concurrently, on 40% of TS data, the MWHODL-SHAR technique has achieved an average accuy of 99.35%, precn of 98.05%, recal of 97.98%, and Fscore of 98.01%, and AUCscore of 98.79%. Simultaneously, on 70% of TR data, the MWHODL-SHAR approach has achieved an average accuy of 99.38%, precn  of 98.16%, recal of 98.07%, and Fscore of 98.11%, and AUCscore of 98.85%. Finally, on 30% of TS data, the MWHODL-SHAR method has reached an average accuy of 99.39%, precn of 98.08%, recal of 98.14%, Fscore of 98.11%, and AUCscore of 98.89%.

The TACC and VACC of the MWHODL-SHAR method are investigated on the UCI HAR dataset in Fig. 5. The figure exhibits the MWHODL-SHAR approach has displayed enhanced performance with increased values of TACC and VACC. It is visible that the MWHODL-SHAR algorithm has attained maximum TACC outcomes.

Figure 5: TACC and VACC of MWHODL-SHAR model on the UCI HAR dataset

The TLS and VLS of the MWHODL-SHAR method were tested on the UCI HAR dataset in Fig. 6. The figure exhibited that the MWHODL-SHAR approach has revealed superior performance with minimal values of TLS and VLS. It is visible that the MWHODL-SHAR technique has resulted in reduced VLS outcomes.

Figure 6: TLS and VLS of MWHODL-SHAR model on the UCI HAR dataset

The confusion matrices of the MWHODL-SHAR model on the USC HAD dataset are reported in Fig. 7. The outcomes demonstrated that the MWHODL-SHAR method had identified all the different types of symmetric human activities.

Figure 7: Confusion matrices of the MWHODL-SHAR model on the USC HAD dataset (a) 60% of TR data, (b) 40% of TS data, (c) 70% of TR data, and (d) 30% of TS data

Table 3 presents the overall activity recognition performance of the MWHODL-SHAR method on the USC HAD dataset. The MWHODL-SHAR approach has proficiently recognized all the activities. For example, on 60% of TR data, the MWHODL-SHAR technique has achieved an average accuy of 99.51%, precn of 98.44%, recal of 98.47%, Fscore of 98.45%, and AUCscore of 99.08%. Concurrently, on 40% of TS data, the MWHODL-SHAR method has achieved an average accuy of 99.35%, precn of 98.05%, recal of 97.98%, and Fscore of 98.01%, and AUCscore of 98.79%. In parallel, on 70% of TR data, the MWHODL-SHAR technique has reached an average accuy of 99.53%, precn of 98.56%, recal of 98.52%, Fscore of 98.54%, and AUCscore of 99.12%. Eventually, on 30% of TS data, the MWHODL-SHAR method has achieved an average accuy of 99.52%, precn of 98.46%, recal of 98.47%, Fscore of 98.46%, and AUCscore of 99.09%.

The TACC and VACC of the MWHODL-SHAR method are inspected on the USC HAD dataset in Fig. 8. The figure implied that the MWHODL-SHAR method had shown improved performance with increased values of TACC and VACC. It is visible that the MWHODL-SHAR model has reached maximum TACC outcomes.

Figure 8: TACC and VACC of MWHODL-SHAR model on the USC HAD dataset

A comparative symmetric activity recognition result of the MWHODL-SHAR model on the UCI HAR dataset is in Table 4. The experimental values demonstrated that the Residual network, Human Activity Recognition on Signal Images (HARSI), and deep CNN models had shown poor recognition performance. Next, the CNN-RF model has depicted certainly improved performance, while the LSTM and convolutional autoencoder (CAE) models have obtained reasonable outcomes. But the MWHODL-SHAR model has attained maximum performance with accuy of 99.39%, precn of 98.08%, recal of 98.14%, and Fscore of 98.11%.

Finally, a comparative symmetric activity recognition result of the MWHODL-SHAR model is made on USC HAD Dataset in Table 5. The simulation values established that the Residual network, HARSI, and deep CNN approaches had exhibited poor recognition performance. Next, the CNN-RF model has improved performance, while the LSTM and CAE approaches have attained reasonable outcomes. But the MWHODL-SHAR technique has achieved maximum performance with accuy of 99.53%, precn of 98.56%, recal of 98.52%, and Fscore of 98.54%. These results show the better performance of the MWHODL-SHAR model over other models.

5  Conclusion

In this study, we have introduced an automated symmetric activity recognition model named MWHODL-SHAR technique. The MWHODL-SHAR model’s goal is to detect and classify symmetric activities such as jogging, walking, standing, and sitting. In the presented MWHODL-SHAR technique, the human activities data is pre-processed in various stages to make it compatible for further processing. Next, the CNN-ALSTM method is employed for accurate symmetric activity recognition. The MWHO algorithm is utilized as a hyperparameter tuning strategy to optimize the detection rate of the CNN-ALSTM algorithm. The experimental validation of the MWHODL-SHAR technique is simulated using a benchmark dataset. An extensive comparison study revealed the betterment of the MWHODL-SHAR technique over other recent approaches.

Funding Statement: The authors received no specific funding for this study.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

References

1. L. Pei, S. Xia, L. Chu, F. Xiao, Q. Wu et al., “MARS: Mixed virtual and real wearable sensors for human activity recognition with multidomain deep learning model,” IEEE Internet Things Journal, vol. 8, no. 11, pp. 9383–9396, 2021. [Google Scholar]

2. B. Yousefi and C. K. Loo, “Biologically-inspired computational neural mechanism for human action/activity recognition: A review,” Electronics, vol. 8, no. 10, pp. 1169, 2019. [Google Scholar]

3. S. O. Slim, A. Atia, M. M. Elfattah and M. S. M. Mostafa, “Survey on human activity recognition based on acceleration data,” International Journal of Advanced Computer Science and Applications, vol. 10, no. 3, pp. 84–98, 2019. [Google Scholar]

4. J. Maitre, K. Bouchard and S. Gaboury, “Alternative deep learning architectures for feature-level fusion in human activity recognition,” Mobile Networks and Applications, vol. 26, no. 5, pp. 2076–2086, 2021. [Google Scholar]

5. J. Wang, Y. Chen, S. Hao, X. Peng and L. Hu, “Deep learning for sensor-based activity recognition: A survey,” Pattern Recognition Letters, vol. 119, pp. 3–11, 2019. [Google Scholar]

6. D. Han, C. Lee and H. Kang, “Gravity control-based data augmentation technique for improving VR user activity recognition,” Symmetry, vol. 13, no. 5, pp. 845, 2021. [Google Scholar]

7. S. Mekruksavanich, A. Jitpattanakul, P. Youplao and P. Yupapin, “Enhanced hand-oriented activity recognition based on smartwatch sensor data using LSTMs,” Symmetry, vol. 12, no. 9, pp. 1570, 2020. [Google Scholar]

8. N. Tasnim, M. K. Islam and J. -H. Baek, “Deep learning based human activity recognition using spatio-temporal image formation of skeleton joints,” Applied Sciences, vol. 11, no. 6, pp. 2675, 2021. [Google Scholar]

9. Y. M. Hwang, S. Park, H. O. Lee, S. -K. Ko and B. -T. Lee, “Deep learning for human activity recognition based on causality feature extraction,” IEEE Acces, vol. 9, pp. 112257–112275, 2021. [Google Scholar]

10. K. Xia, J. Huang and H. Wang, “LSTM-CNN architecture for human activity recognition,” IEEE Access, vol. 8, pp. 56855–56866, 2020. [Google Scholar]

11. M. A. Basset, H. Hawash, R. K. Chakrabortty, M. Ryan, M. Elhoseny et al., “ST-DeepHAR: Deep learning model for human activity recognition in IoHT applications,” IEEE Internet of Things Journal, vol. 8, no. 6, pp. 4969–4979, 2020. [Google Scholar]

12. P. Agarwal and M. Alam, “A lightweight deep learning model for human activity recognition on edge devices,” Procedia Computer Science, vol. 167, pp. 2364–2373, 2020. [Google Scholar]

13. I. U. Khan, S. Afzal and J. W. Lee, “Human activity recognition via hybrid deep learning based model,” Sensors, vol. 22, no. 1, pp. 323, 2022. [Google Scholar] [PubMed]

14. O. Nafea, W. Abdul, G. Muhammad and M. Alsulaiman, “Sensor-based human activity recognition with spatio-temporal deep learning,” Sensors, vol. 21, no. 6, pp. 2141, 2021. [Google Scholar] [PubMed]

15. V. Bianchi, M. Bassoli, G. Lombardo, P. Fornacciari, M. Mordonini et al., “IoT wearable sensor and deep learning: An integrated approach for personalized human activity recognition in a smart home environment,” IEEE Internet of Things Journal, vol. 6, no. 5, pp. 8553–8562, 2019. [Google Scholar]

16. A. Gumaei, M. M. Hassan, A. Alelaiwi and H. Alsalman, “A hybrid deep learning model for human activity recognition using multimodal body sensing data,” IEEE Access, vol. 7, pp. 99152–99160, 2019. [Google Scholar]

17. X. Zhou, W. Liang, I. Kevin, K. Wang, H. Wang et al., “Deep-learning-enhanced human activity recognition for internet of healthcare things,” IEEE Internet of Things Journal, vol. 7, no. 7, pp. 6429–6438, 2020. [Google Scholar]

18. S. Mekruksavanich and A. Jitpattanakul, “Biometric user identification based on human activity recognition using wearable sensors: An experiment using deep learning models,” Electronics, vol. 10, no. 3, pp. 308, 2021. [Google Scholar]

19. J. R. Jiang, J. E. Lee and Y. M. Zeng, “Time series multiple channel convolutional neural network with attention-based long short-term memory for predicting bearing remaining useful life,” Sensors, vol. 20, no. 1, pp. 166, 2019. [Google Scholar] [PubMed]

20. I. Naruei and F. Keynia, “Wild horse optimizer: A new meta-heuristic algorithm for solving engineering optimization problems,” Engineering with Computers, vol. 38, no. Suppl 4, pp. 3025–3056, 2022. [Google Scholar]

21. A. Ramadan, S. Kamel, I. B. Taha and M. Tostado-Véliz, “Parameter estimation of modified double-diode and triple-diode photovoltaic models based on wild horse optimizer,” Electronics, vol. 10, no. 18, pp. 2308, 2021. [Google Scholar]

22. M. J. Goldanloo and F. S. Gharehchopogh, “A hybrid OBL-based firefly algorithm with symbiotic organisms search algorithm for solving continuous optimization problems,” The Journal of Supercomputing, vol. 78, no. 3, pp. 3998–4031, 2022. [Google Scholar]