Recent Advances in Fatigue Detection Algorithm Based on EEG

1  Introduction

Fatigue is defined as a state of declined physical or mental activity, usually due to continuous study or work [1]. It can commonly be divided into physical fatigue and mental fatigue. Generally speaking, mental and physical fatigue are intertwined and appear simultaneously [2]. Compared with physical fatigue, mental fatigue is more complicated and more harmful. Mental fatigue is often the result of long-term accumulation of cognitive activities in the brain, such as excessive thinking and exhaustion. After brain fatigue, cognitive function is limited, and alertness decreases [3]. In a state of mental fatigue, people will feel dizzy, unresponsive, distracted, and difficult to perform their thinking activities normally [4]. This affects learning and work efficiency and leads to insomnia, anxiety, worry, and forgetting. Driver fatigue is common mental fatigue in daily life, which is one of the most important reasons for many traffic accidents. An earlier study by the American Automobile Association Road Safety Foundation estimated that 7% of traffic accidents, 13% of hospital traffic accidents and 21% of fatal traffic accidents in relation to driver fatigue [5]. According to the American Road Safety Foundation, about 30.8% of people have been driving fatigued [6]. Therefore, how to detect fatigue correctly has become an urgent issue.

A variety of methods of fatigue detection have emerged in the past decades. We can divide them into 3 categories: questionnaires (e.g., Psycho-motor Vigilance Test (PVT) [7], Checklist Individual Strength questionnaire (CIS) [8]), behavioral detection methods (e.g., head position [9], blink frequency [10]), and methods based on physiological signals (e.g., EEG [11,12], electrooculogram (EOG) [13,14], electromyography (EMG) [15,16], electrocardiogram (ECG) [17,18], pulse beat [19]). Among these, the questionnaires are psychological tests, the advantage of which are cheap and easy to promote. However, these can’t detect the state of the subjects in real-time and are also highly subjective [20]. Behavioral detection methods use image recognition methods to identify the state of the subject’s head and eyes, the head position and the blinking frequency of eyes change with fatigue. These methods require using a camera to collect head features in real-time. The illumination greatly affects its accuracy and cannot adapt to the ever-changing environment [21,22]. Physiological signals can accurately reflect the human state in real-time. Bioelectricity is generated by cell activity. For instance, EOG is bioelectricity generated by eye movement. EEG is produced by the human brain at the scalp and can directly reflect different activity states of the brain. In a study on fatigue detection by Laurent et al. [23], the accuracy of EEG is significantly better than ECG and EOG. EEG is recognized as one of the most objective and accurate indicators of brain fatigue detection in all common physiological signals [24]. This study focuses on investigating the development of EEG-based fatigue detection methods.

The entire EEG-based fatigue detection methods or systems commonly requires the three blocks: signal acquisition, signal processing and analysis, feedback and application, as shown in Fig. 1. Specifically, signal acquisition uses the EEG cap or other equipment to collect the fatigue and awake state EEG signals of the subjects. The signal processing and analysis are the most critical steps, including the three processes of preprocessing, feature extraction, and classification. The preprocessing is to delete the original EEG signal’s artefacts such as EOG and EMG, and the feature extraction is to extract key features to help the classifier to decode effectively such as Wavelet Transform, Autoregressive (AR), Fast Fourier Transform (FFT), etc. Then the effective features such as entropy, time/frequency domain characterization are used to train a model in classification to distinguish fatigue from the alert. We usually divide these classification algorithms into three categories: machine learning, deep learning and transfer learning. Machine learning methods are easy to implement and can achieve good results without a large training dataset, such as Support Vector Machines (SVM), K-Nearest Neighborhood (KNN), etc. The deep learning method needs to adjust the network structure and parameters, but it can get effective results with enough training data. The commonly deep learning methods are Convolutional Neural Network (CNN), Long-Short Term Memory (LSTM), etc. For transfer learning, it’s suitable for application with small or even zero-shot training. At present, some transfer learning algorithms such as Transfer Component Analysis (TCA), Domain Adversarial Neural Network (DANN) have been widely used in fatigue detection based on EEG. Through the block of application and feedback the detection results can be presented to remind the user that they are in a state of fatigue.

Figure 1: The framework of EEG-based fatigue detection system

The preprocessing includes filtering, epoch extraction, down-sampling, artifact removal, ICA decomposition and other processes. The function of the preprocessing process is to filter and clean to obtain better EEG data. Borowicz [25] designed time-domain linear filtering called Multichannel Wiener Filter (MWF) to wipe out the artifacts in EEG data. After the preprocessing, there are a lot of methods to extract the features such as Fast Fourier transform (FFT), wavelet transform. Wali et al. [26] applied the Discrete Wavelet Transform (DWT) and FFT to get time-frequency features and obtained a 79.21% accuracy classifier.

In order to avoid overlap with previous reviews, this study is the first to review the classification algorithms and some feature extraction algorithms in EEG-based fatigue detection in recent years. We divided these algorithms into two categories, intra-subject and cross-subject. Intra-subject refers to using the same subject’s data to train and test, while cross-subject refers to the case that the training and testing data come from different subjects. We summarized and analyzed these algorithms to lay the foundation for future research on algorithms based on EEG fatigue detection technology. Moreover, we also discussed the existing problems of EEG-based fatigue detection and analyzed the future development direction of the field based on the existing foundation.

The remainder of this study will be organized as follows: Section 2 reviews the EEG-based fatigue detection algorithms in intra-subject; we review the EEG-based fatigue detection algorithms in cross-subject in Section 3; Section 4 provides a discussion about current problems and future directions, and finally, Section 5 summarizes of the full text.

2  EEG-based Fatigue Detection in Intra-subject

This section will review in detail the research progress of EEG-based fatigue detection in intra-subject in recent years. As shown in Fig. 2, the classification algorithms in intra-subject are divided into two categories: traditional machine learning and deep learning. SVM [27,28], Random Forest (RF) [29], KNN [30], and Artificial Neural Networks (ANN) [31] are commonly used as traditional machine learning classifiers in the field of EEG-based fatigue detection. For deep learning algorithms, researchers exploit the characteristics of each network to extract effective features and improve classification performance. CNN [32–34] is commonly utilized to extract frequency domain characterization, and LSTM [35,36] is applied to analyze time series. Besides, Deep Belief Network (DBN) [37] and some other deep learning networks have been used in fatigue detection based on EEG.

Figure 2: The EEG-based fatigue detection classification algorithms in intra-subject

2.1 Traditional Machine Learning Algorithms Applicated in Classification

Traditional machine learning classification algorithms are usually employed with feature extraction methods. In the early days, researchers used traditional feature extraction algorithms with machine learning algorithms to improve classification accuracy. Besides, researchers proposed some networks which were composed of multiple feature extraction methods to automatically extract deep effective features. According to the characteristics of the feature extraction algorithm, we could introduce the development of traditional machine learning for EEG-based fatigue detection from two aspects: traditional feature extraction methods and feature extraction network methods. Tab. 1 is the summary of the machine learning articles about the author, feature extraction method, classification algorithm and accuracy. In the table, the first four items use traditional feature extraction methods, and the remaining two items apply feature extraction network to extract features.

2.1.1 Traditional Machine Learning Methods Combined with Traditional Feature Extraction Methods

SVM is one of the most widely used method for EEG classification. It can accurately predict numerous variables, while the performance is very competitive compared to other methods [40]. Zhao et al. [27] used MVAR model to extract multi-channel EEG signal features, and then they used KPCA-SVM to get 81.64% accuracy. KPCA was applied to reduce the dimension of feature vectors and accelerating the convergence speed of SVM training. The major shortcoming of SVM is the key parameters are difficult to adjust for best results [41]. ANN also needs to adjust several parameters, and it is prone to overfitting, but it has an associative memory function and can adequately approach intricate nonlinear relationships [42]. Zhang et al. [31] applied wavelet transform to extract three kinds of entropy from 20 healthy male’s (20–35 years) EEG, EOG and EMG data. Then a four-layer (two hidden layers) MLPNN was used as the classifiers. The results showed that their method could achieve about 96.5%–99.5% accuracy. What can be known is the structure of MLPNN has a great influence on the model. Introducing the concept of Bayesian into the neural network can produce a measure of uncertainty and treat the weights as random variables, which is not easy to overfit [43]. Chai et al. [39] employed BNN to analyze EEG fatigue state data and alert state data from 43 healthy participants. The accuracy for detecting fatigue status is 84.3%, and for detecting alertness, the accuracy is 83.0%. It shows that BNN improves generalization ability without being affected by data quality. However, BNN usually has more parameters, and the performance in classification/regression problems on large-scale datasets does not have much advantage over ordinary neural networks [44].

Besides, the above methods all require calibration work when applied to new subjects, which need the collection of data from subject-specific to adjust the model parameters. To deal with this issue, Cui et al. [38] applied 16 healthy subjects’ EEG data FWET to train and test FWET model. The results showed FWET could get a good performance without calibration work. The author expanded domain generalization from computer vision to brain-computer interface, which is applied to eliminate the calibration effort caused by individual differences completely.

2.1.2 Traditional Machine Learning Methods Combined with Feature Extraction Network Methods

Traditional feature extraction is only a superficial level extraction, and some discriminative information cannot be effectively extracted. Some shallow learning methods also require artificial design or application of certain features as the machine learning input. Researchers combine multiple feature extraction methods to form a feature extraction network to obtain more features in recent research. Compared with traditional extraction methods, it has better performance and more useful features.

There are also many classification methods used in conjunction with feature extraction networks. Ma et al. [28] combined PCA and PCANet to build a feature extraction network, and applied SVM for classification. It is obvious that PCA could reduce the dimensionality of EEG data to prevent PCANet from causing a dimensional explosion. At the same time, with the advantages of SVM processing small sample data, the accuracy of 95.14 ± 4.87 had been achieved. Different from Ma et al., Tuncer et al. [30] utilized Dynamic Center based Binary Pattern (DCBP) and Multi Threshold based Ternary Pattern (MTTP) to generate features, and then employed NCA, ReliefF and PCA to build a feature selection network. KNN was applied as a classifier because of its simplicity and excellent performance. This feature generation and selection can shorten training time and improve classification accuracy without any optimization algorithm.

2.2 Deep Learning Algorithms Applicated in Classification

Compared with machine learning, deep learning can easily utilize large amounts of data and has better classification performance [45]. Many deep networks have been applied in EEG-based fatigue detection in recent years. Tab. 2 summarizes all the deep learning articles about author, classification algorithm and accuracy. The first three items in the table use CNN, the next two items employ LSTM, and the last two apply other deep networks.

CNNs are commonly utilized in image recognition and classification [47]. It can automatically and adaptively learn the feature space hierarchy at each level and reduce the number of learned parameters [48], thereby improving training efficiency to avoid overfitting and reduce network complexity. CNNs have been proverbially applied in EEG-based fatigue detection and has achieved good results in recent years.

Zeng et al. [32] designed two novel models named EEG-Conv and EEG-Conv-R to detect fatigue through EEG. EEG-Conv is a traditional CNN, and EEG-Conv-R integrates CNN with deep residual learning. EEG-Conv-R converges more rapidly and gets a higher accuracy in intra-subject compared to EEG-Conv. The idea of residual learning in EEG-Conv-R has an excellent learning effect on nonlinear EEG signals and can reduce the interferences of the signals. CNN is more appropriate for extracting spatial features from EEG signals, while temporal features are easily ignored. Gao et al. [34] built a novel ESTCNN whose core block is introduced to extract temporal features. And dense layers were employed to merge spatial features and finish classification. ESTCNN has excellent performance whose classification accuracy reached 97.37%. Meanwhile, ESTCNN reduces the processing of multi-channel data, which is more conducive to realizing the BCI online fatigue detection system. Many researchers also combined RNN with CNN to extract spatiotemporal features. Gao et al. [33] employed a Recurrent Network (RN) to analyses time series. The information matrix of RN was used as the input of CNN. This model applied multiple RNs for dimensionality reduction and feature extraction and then CNN for classification. The accuracy of RN-CNN reached 92.95%, which has good classification performance. RNN can easily to lead the problems of gradient vanishing and gradient explosion [49].

LSTM is a particular recurrent neural network (RNN) [50]. It is primarily to deal with gradient vanishing and explosion issue during long sequence training [51]. Budak et al. [35] designed three building blocks to detect drowsiness. These three building blocks used different methods to extract different features, respectively. Then all the features of the three building blocks were adopted for the LSTM network to classification. And the result of LSTM was combined with a majority voting layer. The final experiment got an average accuracy of 94.31%. This method employed TQWT to extract different features and combine them on the multi-voting mechanism of the LSTM network, which is reliable, robust and efficient. LSTM may overfit due to less EEG data. Generative Adversarial Network (GAN) is widely used in image generation and processing [52,53]. Now it is applied to expand the EEG data. Jiao et al. [36] applied Conditional Wasserstein GAN to expand the sample, then utilized LSTM to process the time information of EEG and EOG signals.

Although LSTM solves the gradient problem of traditional RNN, it also adds a lot of content and parameters, making training more difficult [51]. Accordingly, researchers have also applied other deep learning algorithms to EEG classification in recent years. The Broad Learning System (BLS) offers an alternative to deep learning networks, which solves the problem of continuously increasing the network layers or adjusting the parameters to improve the accuracy of deep models [54]. Yang et al. [46] constructed a CNBLS which starts from the Multivariate Weighted Recurrence Networks (MWRNs). They got an excellent result which the area under the curve (AUC) over 98% in all 11 subjects. What’s more, CNBLS achieved superior average accuracy (99.58%), sensitivity (99.67%) and specificity (99.48%). BLS has two major advantages: horizontal expansion and incremental learning because of its single hidden layer structure. Different from deep Learning, BLS can use horizontal scaling to add new enhancement nodes, feature nodes and input data to improve model performance.

Generally speaking, we use supervised learning to complete fatigue detection. But in real life, unlabeled data is easily available, while labeled data is often hard to get, and labeling is time consuming and laborious. Semi-Supervised learning is more suitable for real-world applications and has recently become a hot new direction in deep learning. This method requires only a bit labeled samples and numerous unlabeled samples. Chai et al. [37] improved the DBN and used 43 peoples’ EEG data to train and test the network. They used a regularization term to add sparsity in sparse-DBN. It can prevent model overfitting and learn low-level and high-level structures. The results showed that sparse-DBN has an excellent performance whose accuracy achieved 93.1%. Forming the sparse-DBN using sparse-restricted Boltzmann machine (RBM) enables learning of helpful low-level and high-level features for unlabeled data.

3  EEG-based Fatigue Detection in Cross-subject

Due to the large difference between EEG among individuals, a reliable model should be constructed after training the collected data from the same person. Furthermore, this model is not suitable for new users, and the original model is less robust. Therefore, it is critical to establish a model in real application which can detect fatigue in cross-subject. In recent studies, a small number of researchers used deep learning on EEG-based fatigue detection in cross-subject, e.g., enhancing CNN to improve cross-subject accuracy. Most researchers tried to use transfer learning to reduce the variability between subjects, which can transfer the information from the source domain to the target domain. A few algorithms about domain adaptions were employed in this field, such as Maximum Independence Domain Adaptation (MIDA), DANN, Easy Transfer Learning (EasyTL). Tab. 3 is the summary of all the cross-subject studies about the author, classification algorithm and accuracy. The first two items in the table used CNN for classification, the next three items applied domain adaptation to classification, and the last one improved easy transfer learning (EasyTL) as the classifier.

Paulo et al. [55] applied a Butterworth filter to get the power density of the theta (4–8 Hz), alpha (8–13 Hz), and beta (13–30 Hz). Then they calculated attention metrics associated with fatigue. The recurrence plot and the gramian angular fields were applied to convert the above features to images which was the input of CNN. As a result, they achieved an average precision of 75.87% on 27 subjects for leave-one-out cross-validation. Cui et al. [56] tried to improved an interpretation technique called class activation map (CAM) to discover the factors that favor and disfavor classification. Separable convolution was utilized to reduce parameters and speed up network convergence. And a seven layers CNN was used to recognize fatigue. The results showed that the average accuracy of the network reached 78.35% in the leave-one-out cross-validation of 11 subjects.

The deep learning algorithms currently seem to have poor theoretical interpretability in cross-subject classification, and the experimental results need to be improved. Therefore, many researchers have focused on the study of transfer learning algorithms to improve the performance of EEG-based fatigue detection in cross-subject. The most commonly transfer learning method is domain adaptation, in which the source task is the same as the target task, but the data distribution in the two domains are different. And the source domain has numerous labeled samples, but the target domain has no (or only very few) labeled samples [62]. Wu et al. [57] developed an algorithm called OwARR that combined fuzzy sets with domain adaptation to cut back the number of alignment data for specific objects. But when using OwARR for each source domain, it consumed a lot of time. Therefore, a source domain selection (SDS) method was employed for reducing the calculated cost of OwARR. The results showed that OwARR has high applicability, and SDS could reduce the computing time of OwARR by about half. TCA can mitigate the distribution mismatch between source and target data. Liu et al. [58] used RF to select channels and TCA to classify the selected one channel. At the same time, a transfer learning algorithm called MIDA was applied for classifiers in all channels. The results showed that the performance of MIDA is slightly better than that of TCA, which indicates that although TCA could transfer knowledge, it will affect the classification performance. In a recent study, GAN was utilized to effectively reduce the difference between training and test distribution and improve generalization performance. Zeng et al. [59] combined GAN and DANN to propose a Generative-DANN model (GDANN). In the cross-subject EEG-based fatigue detection, GDANN could achieve 91.63% accuracy. It could be concluded that for subjects with better DANN classification, GDANN can slightly improve accuracy, while for subjects with poor DANN performance, GDANN can significantly improve. As shown in Fig. 3, GDANN applied a generate network from GAN to generate the fake data and discriminant network to select the most suitable source domain subjects. Then using the selected data and fake data to train the DANN.

Figure 3: The structure of GDANN [59]. (The process of minimizing Nf, Ny, and maximizing Nd to train a DANN, which involves sifting the data for the previous Ns source-domain subjects most resemblance to the target domain and using random noise to generate data to train a domain-aligned model together)

In addition to domain adaptation, other transfer learning methods were also used in brain-computer interface. For EEG-based fatigue detection, there are not many methods currently applied. EasyTL was designed by Wang et al. [63]. It does not need to select model and adjust hyperparameter. It can learn non-parametric transfer features and classifiers at the same time. Zeng et al. [60] improved EasyTL and proposed a classifier called InstanceEasyTL for cross-subject EEG-based fatigue detection. It shows that the InstanceEasyTL could use fewer data to get a higher accuracy and robustness model. In the same cross-subject experiment, InstanceEasyTL could reach an average accuracy of 88.33%, much higher than DANN (70.75%) and EasyTL (70.91%).

4  Limitations and Development Directions of EEG-based Fatigue Detection

Over the past few decades, EEG-based fatigue detection has made great progress. However, it still has great limitations. Since the EEG signal is usually feeble and the signal-to-noise ratio is very low, for most algorithms in intra-subject, accurate feature extraction and efficient classification algorithms are usually required to obtain high accuracy. Meanwhile, for the traditional machine learning algorithms, it is difficult to take advantage of deep features, so high accuracy is hard to achieve when utilizing traditional feature extraction methods [64]; but when using feature extraction network, a large amount of data is required to train the network to ensure accuracy. And when we employ deep learning as a classifier, it is hard to explain the training process of the model [65]. What’s worse, it takes a long time and an amount of EEG data to build the model while the data can’t be collected in large quantities [66]. Besides, these trained models are not available online because of the complicated process. Although there are so many problems to be solved, it can be seen from the literature that in recent years, most researches tend to use deep learning. With numerous data, establishing a deep learning algorithm with strong interpretability, high accuracy, and real-time is an important future direction.

Because the EEG signal is non-stationary, the algorithms in intra-subject cannot achieve good results when using cross-subject training data [67]. However, it is often arduous to require each subject to do extensive calibration exercises in practical applications. The algorithms in cross-subject can build a universal classifier to a certain extent [60]. However, in terms of the current transfer learning results, the accuracy of cross-subjects is too low, which needs to be improved urgently. In the case of small or zero training samples, improving the accuracy and scope of transfer learning methods is another direction. At present, researchers need to improve the feature extraction method and classification method. Some traditional algorithm ideas, such as BP neural network [68], may have certain reference value.

EEG-related studies have shown that multimodal-based systems can extract or learn the features of multiple signals. Their overall performance is higher than single-modality systems [69–72]. In the study of EEG emotion recognition, some authors fused facial expressions with EEG to analyze [71,72]. Therefore, it is also a future research direction to study multi-modal algorithms to combine other signals, such as behavioral signals, EOG and EEG, to improve fatigue detection. According to the research of Laurent et al. [23], combining EEG features with ECG and EOG features can obviously improve classification performance for short time windows.

Both deep learning and traditional machine learning methods require numerous labeled samples. Furthermore, the transfer learning method is more effective if there are more samples. Therefore, it is very important to collect many training samples in the future. How to design new effective paradigms, improve the wearable and portable sensor and save huge useful EEG data in the case of information security are important issues. As an ever-evolving field of study, fatigue detection research increasingly requires multidisciplinary collaborative efforts.

5  Conclusions

The fast-paced life in today’s society can easily lead to fatigue which can directly or indirectly cause harm to the human body. There is no doubt that high-precision, real-time, and universal fatigue detection is of great significance. This study reviewed the progress of EEG-based fatigue detection in recent years from two types of algorithms: intra-subject and cross-subject. The summary of these methods provided a solid theoretical foundation for future algorithmic research in this field. Moreover, we also analyzed the existing problems of EEG-based fatigue detection and provided some directions for solutions. As an evolving research area, fatigue detection needs to combine multi-disciplinary knowledge to improve the performance. While improving the detection algorithm, designing new effective paradigms and portable equipment, and collecting valid standard data are also urgent.

Funding Statement: This project was funded by the National Natural Science Foundation of China (Grant Nos. 61906019, 62006082 and 62076103), the Guangdong Basic and Applied Basic Research Foundation (Grant Nos. 2021A1515011853, 2021A1515011600 and 2020A1515110294), Guangzhou Science and Technology Plan Project (Grant No. 202102020877) and the Guangzhou Science and Technology Plan Project Key Field R&D Project (202007030005).

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

References

 1.  L. S. Aaronson, C. S. Teel, V. Cassmeyer, G. B. Neuberger, L. Pallikkathayil et al., “Defining and measuring fatigue,” Image: The Journal of Nursing Scholarship, vol. 31, no. 1, pp. 45–50, 1999. [Google Scholar]

 2.  S. Nozaki, M. Tanaka, K. Mizuno, S. Ataka, H. Mizuma et al., “Mental and physical fatigue-related biochemical alterations,” Nutrition, vol. 25, no. 1, pp. 51–57, 2009. [Google Scholar]

 3.  R. D. Moore, M. W. Romine, P. J. O’connor and P. D. Tomporowski, “The influence of exercise-induced fatigue on cognitive function,” Journal of Sports Sciences, vol. 30, no. 9, pp. 841–850, 2012. [Google Scholar]

 4.  S. Kar, M. Bhagat and A. Routray, “EEG signal analysis for the assessment and quantification of driver’s fatigue,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 13, no. 5, pp. 297–306, 2010. [Google Scholar]

 5.  B. C. Tefft, Acute Sleep Deprivation and Risk of Motor Vehicle Crash Involvement, Washington, DC, USA: AAA Foundation for Traffic Safety, 2016. [Online]. Available: https://aaafoundation.org/acute-sleep-deprivation-risk-motor-vehicle-crash-involvement/. [Google Scholar]

 6.  T. Johnson, 2017 Traffic Safety Culture Index, Washington, DC, USA: AAA Foundation for Traffic Safety, 2018. [Online]. Available: https://aaafoundation.org/2017-traffic-safety-culture-index/. [Google Scholar]

 7.  M. Basner and J. Rubinstein, “Fitness for duty: A 3-minute version of the psychomotor vigilance test predicts fatigue-related declines in luggage-screening performance,” Journal of Occupational and Environmental Medicine, vol. 53, no. 10, pp. 1146–1154, 2011. [Google Scholar]

 8.  A. J. Beurskens, U. Bültmann, U. Kant, J. H. Vercoulen, G. Bleijenberg et al., “Fatigue among working people: Validity of a questionnaire measure,” Occupational and Environmental Medicine, vol. 57, no. 5, pp. 353–357, 2000. [Google Scholar]

 9.  M. R. Smith, A. J. Coutts, M. Merlini, D. Deprez, M. Lenoir et al., “Mental fatigue impairs soccer-specific physical and technical performance,” Med. Sci. Sports Exerc., vol. 48, no. 2, pp. 267–276, 2016. [Google Scholar]

10. M. H. Sigari, M. Fathy and M. Soryani, “A driver face monitoring system for fatigue and distraction detection,” International Journal of Vehicular Technology, vol. 2013, no. 263983, pp. 1–11, 2013. [Google Scholar]

11. H. Wang, A. Dragomir, N. I. Abbasi, J. Li, N. V. Thakor et al., “A novel real-time driving fatigue detection system based on wireless dry EEG,” Cognitive Neurodynamics, vol. 12, no. 4, pp. 365–376, 2018. [Google Scholar]

12. J. Hu and J. Min, “Automated detection of driver fatigue based on EEG signals using gradient boosting decision tree model,” Cognitive Neurodynamics, vol. 12, no. 4, pp. 431–440, 2018. [Google Scholar]

13. M. Kołodziej, P. Tarnowski, D. J. Sawicki, A. Majkowski, R. J. Rak et al., “Fatigue detection caused by office work with the use of EOG signal,” IEEE Sensors Journal, vol. 20, no. 24, pp. 15213–15223, 2020. [Google Scholar]

14. M. Song, L. Li, J. Guo, T. Liu, S. Li et al., “A new method for muscular visual fatigue detection using electrooculogram,” Biomedical Signal Processing and Control, vol. 58, pp. 101865, 2020. [Google Scholar]

15. Q. Abbas, M. E. Ibrahim, S. Khan and A. R. Baig, “Hypo-driver: A multiview driver fatigue and distraction level detection system,” Computers, Materials & Continua, vol. 71, no. 1, pp. 1999–2007, 2022. [Google Scholar]

16. P. A. Karthick, D. M. Ghosh and S. Ramakrishnan, “Surface electromyography based muscle fatigue detection using high-resolution time-frequency methods and machine learning algorithms,” Computer Methods and Programs in Biomedicine, vol. 154, pp. 45–56, 2018. [Google Scholar]

17. L. Wang, H. Wang and X. Jiang, “A new method to detect driver fatigue based on EMG and ECG collected by portable non-contact sensors,” Promet-Traffic&Transportation, vol. 29, no. 5, pp. 479–488, 2017. [Google Scholar]

18. F. Wang, H. Wang and R. Fu, “Real-time ECG-based detection of fatigue driving using sample entropy,” Entropy, vol. 20, no. 3, pp. 196, 2018. [Google Scholar]

19. S. J. Jung, H. S. Shin and W. Y. Chung, “Driver fatigue and drowsiness monitoring system with embedded electrocardiogram sensor on steering wheel,” IET Intelligent Transport Systems, vol. 8, no. 1, pp. 43–50, 2014. [Google Scholar]

20. B. L. Egleston, S. M. Miller and N. J. Meropol, “The impact of misclassification due to survey response fatigue on estimation and identifiability of treatment effects,” Statistics in Medicine, vol. 30, no. 30, pp. 3560–3572, 2011. [Google Scholar]

21. W. J. Chang, L. B. Chen and Y. Z. Chiou, “Design and implementation of a drowsiness-fatigue-detection system based on wearable smart glasses to increase road safety,” IEEE Transactions on Consumer Electronics, vol. 64, no. 4, pp. 461–469, 2018. [Google Scholar]

22. F. Bi, X. Ma, W. Chen, W. Fang, H. Chen et al., “Review on video object tracking based on deep learning,” Journal of New Media, vol. 1, no. 2, pp. 63–74, 2019. [Google Scholar]

23. F. Laurent, M. Valderrama, M. Besserve, M. Guillard, J. P. Lachaux et al., “Multimodal information improves the rapid detection of mental fatigue,” Biomedical Signal Processing and Control, vol. 8, no. 4, pp. 400–408, 2013. [Google Scholar]

24. F. Wang, S. Wang, X. Wang, Y. Peng and Y. Yang, “Design of driving fatigue detection system based on hybrid measures using wavelet-packets transform,” in Proc. of the 2014 IEEE Int. Conf. on Robotics and Automation, Hong Kong, HK, China, pp. 4037–4042, 2014. [Google Scholar]

25. A. Borowicz, “Using a multichannel Wiener filter to remove eye-blink artifacts from EEG data,” Biomedical Signal Processing and Control, vol. 45, pp. 246–255, 2018. [Google Scholar]

26. M. K. Wali, M. Murugappan and B. Ahmmad, “Wavelet packet transform based driver distraction level classification using EEG,” Mathematical Problems in Engineering, vol. 2013, pp. 297587, 2013. [Google Scholar]

27. C. Zhao, C. Zheng, M. Zhao, Y. Tu and J. Liu, “Multivariate autoregressive models and kernel learning algorithms for classifying driving mental fatigue based on electroencephalographic,” Expert Systems with Applications, vol. 38, no. 3, pp. 1859–1865, 2011. [Google Scholar]

28. Y. Ma, B. Chen, R. Li, C. Wang, J. Wang et al., “Driving fatigue detection from EEG using a modified PCANet method,” Computational Intelligence and Neuroscience, vol. 2019, pp. 4721863, 2019. [Google Scholar]

29. J. Hu, “Comparison of different features and classifiers for driver fatigue detection based on a single EEG channel,” Computational and Mathematical Methods in Medicine, vol. 2017, pp. 5109530, 2017. [Google Scholar]

30. T. Tuncer, S. Dogan, F. Ertam and A. Subasi, “A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing EEG signals,” Cognitive Neurodynamics, vol. 15, pp. 223–237, 2021. [Google Scholar]

31. C. Zhang, H. Wang and R. Fu, “Automated detection of driver fatigue based on entropy and complexity measures,” IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 1, pp. 168–177, 2013. [Google Scholar]

32. H. Zeng, C. Yang, G. Dai, F. Qin, J. Zhang et al., “EEG classification of driver mental states by deep learning,” Cognitive Neurodynamics, vol. 12, no. 6, pp. 597–606, 2018. [Google Scholar]

33. Z. K. Gao, Y. L. Li, Y. X. Yang and C. Ma, “A recurrence network-based convolutional neural network for fatigue driving detection from EEG,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 29, no. 11, pp. 113126, 2019. [Google Scholar]

34. Z. Gao, X. Wang, Y. Yang, C. Mu, Q. Cai et al., “EEG-based spatio–temporal convolutional neural network for driver fatigue evaluation,” IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 9, pp. 2755–2763, 2019. [Google Scholar]

35. U. Budak, V. Bajaj, Y. Akbulut, Q. Atila and A. Sengur, “An effective hybrid model for EEG-based drowsiness detection,” IEEE Sensors Journal, vol. 19, no. 17, pp. 7624–7631, 2020. [Google Scholar]

36. Y. Jiao, Y. Deng, Y. Luo and B. L. Lu, “Driver sleepiness detection from EEG and EOG signals using GAN and LSTM networks,” Neurocomputing, vol. 408, pp. 100–111, 2020. [Google Scholar]

37. R. Chai, S. H. Ling, P. P. San, G. R. Naik, T. N. Nguyen et al., “Improving EEG-based driver fatigue classification using sparse-deep belief networks,” Frontiers in Neuroscience, vol. 11, pp. 103, 2017. [Google Scholar]

38. Y. Cui, Y. Xu and D. Wu, “EEG-based driver drowsiness estimation using feature weighted episodic training,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 27, no. 11, pp. 2263–2273, 2019. [Google Scholar]

39. R. Chai, G. R. Naik, T. N. Nguyen, S. H. Ling, Y. Tran et al., “Driver fatigue classification with independent component by entropy rate bound minimization analysis in an EEG-based system,” IEEE Journal of Biomedical and Health Informatics, vol. 21, no. 3, pp. 715–724, 2016. [Google Scholar]

40. N. Kumar, K. Alam and A. H. Siddiqi, “Wavelet transform for classification of EEG signal using SVM and ANN,” Biomedical and Pharmacology Journal, vol. 10, no. 4, pp. 2061–2069, 2017. [Google Scholar]

41. S. Thakur, R. Dharavath and D. R. Edla, “Spark and rule-KNN based scalable machine learning framework for EEG deceit identification,” Biomedical Signal Processing and Control, vol. 58, pp. 101886, 2020. [Google Scholar]

42. J. V. Tu, “Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes,” Journal of Clinical Epidemiology, vol. 49, no. 11, pp. 1225–1231, 1996. [Google Scholar]

43. F. Burden and D. Winkler, “Bayesian regularization of neural networks,” Artificial Neural Networks, vol. 458, pp. 23–42, 2008. [Google Scholar]

44. M. S. Khan and P. Coulibaly, “Bayesian neural network for rainfall-runoff modeling,” Water Resources Research, vol. 42, no. 7, pp. W07409, 2006. [Google Scholar]

45. Y. LeCun, Y. Bengio and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015. [Google Scholar]

46. Y. Yang, Z. Gao, Y. Li, Q. Cai, N. Marwan et al., “A complex network-based broad learning system for detecting driver fatigue from EEG signals,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, pp. 5800–5808, 2019. [Google Scholar]

47. J. Qin, W. Pan, X. Xiang, Y. Tan and G. Hou, “A biological image classification method based on improved CNN,” Ecological Informatics, vol. 58, pp. 101093, 2020. [Google Scholar]

48. Y. Luo, J. Qin, X. Xiang, Y. Tan, Q. Liu et al., “Coverless real-time image information hiding based on image block matching and dense convolutional network,” Journal of Real-Time Image Processing, vol. 17, no. 1, pp. 125–135, 2020. [Google Scholar]

49. Y. Huang, L. Shen and H. Liu, “Grey relational analysis, principal component analysis and forecasting of carbon emissions based on long short-term memory in China,” Journal of Cleaner Production, vol. 209, pp. 415–423, 2019. [Google Scholar]

50. A. Sherstinsky, “Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network,” Physica D: Nonlinear Phenomena, vol. 404, pp. 132306, 2020. [Google Scholar]

51. Ö. Yildirim, “A novel wavelet sequence based on deep bidirectional LSTM network model for ECG signal classification,” Computers in Biology and Medicine, vol. 96, pp. 189–202, 2018. [Google Scholar]

52. W. Fang, E. Gu, W. Yi, W. Wang and V. S. Sheng, “A new method of image restoration technology based on WGAN,” Computer Systems Science and Engineering, vol. 41, no. 2, pp. 689–698, 2022. [Google Scholar]

53. P. Gong, J. Liu and S. Lv, “Image denoising with GAN based model,” Journal of Information Hiding and Privacy Protection, vol. 2, no. 4, pp. 155–163, 2020. [Google Scholar]

54. C. L. P. Chen and Z. Liu, “Broad learning system: An effective and efficient incremental learning system without the need for deep architecture,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 1, pp. 10–24, 2017. [Google Scholar]

55. J. R. Paulo, G. Pires and U. J. Nunes, “Cross-subject zero calibration driver’s drowsiness detection: Exploring spatiotemporal image encoding of EEG signals for convolutional neural network classification,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 29, pp. 905–915, 2021. [Google Scholar]

56. J. Cui, Z. Lan, O. Sourina and W. Müller-Wittig, “EEG-based cross-subject driver drowsiness recognition with an interpretable convolutional neural network,” IEEE Transactions on Neural Networks and Learning Systems, vol. 2022, pp. 1–13, 2022. [Google Scholar]

57. D. Wu, V. J. Lawhern, S. Gordon, B. J. Lance and C. T. Lin, “Driver drowsiness estimation from EEG signals using online weighted adaptation regularization for regression (OwARR),” IEEE Transactions on Fuzzy Systems, vol. 25, no. 6, pp. 1522–1535, 2016. [Google Scholar]

58. Y. Liu, Z. Lan, J. Cui, O. Sourina and W. Müller-Wittig, “Inter-subject transfer learning for EEG-based mental fatigue recognition,” Advanced Engineering Informatics, vol. 46, pp. 101157, 2020. [Google Scholar]

59. H. Zeng, X. Li, G. Borghini, Y. Zhao, P. Aricò et al., “An EEG-based transfer learning method for cross-subject fatigue mental state prediction,” Sensors, vol. 21, no. 7, pp. 2369, 2021. [Google Scholar]

60. H. Zeng, J. Zhang, W. Zakaria, F. Babiloni, B. Gianluca et al., “InstanceEasyTL: An improved transfer-learning method for EEG-based cross-subject fatigue detection,” Sensors, vol. 20, no. 24, pp. 7251, 2020. [Google Scholar]

61. Z. Cao, C. H. Chuang, J. K. King and C. T. Lin, “Multi-channel EEG recordings during a sustained-attention driving task,” Scientific Data, vol. 6, no. 1, pp. 1–8, 2019. [Google Scholar]

62. S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 22, no. 10, pp. 1345–1359, 2009. [Google Scholar]

63. J. Wang, Y. Chen, H. Yu, M. Huang and Q. Yang, “Easy transfer learning by exploiting intra-domain structures,” in Proc. of the 2019 IEEE Int. Conf. on Multimedia and Expo, Shanghai, SH, China, pp. 1210–1215, 2019. [Google Scholar]

64. Y. Yang, D. Li and Z. Duan, “Chinese vehicle license plate recognition using kernel-based extreme learning machine with deep convolutional features,” IET Intelligent Transport Systems, vol. 12, no. 3, pp. 213–219, 2018. [Google Scholar]

65. C. Wu, M. J. F. Gales, A. Ragni, P. Karanasou and K. C. Sim, “Improving interpretability and regularization in deep learning,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no. 2, pp. 256–265, 2017. [Google Scholar]

66. L. Sun, H. Gao, S. Pan and J. X. Wang, “Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data,” Computer Methods in Applied Mechanics and Engineering, vol. 361, pp. 112732, 2020. [Google Scholar]

67. M. A. Duhayyim, H. M. Alshahrani, F. N. Al-Wesabi, M. A. Al-Hagery, A. M. Hilal et al., “Intelligent machine learning based EEG signal classification model,” Computers, Materials & Continua, vol. 71, no. 1, pp. 1821–1835, 2022. [Google Scholar]

68. X. R. Zhang, X. Sun, W. Sun, T. Xu and P. P. Wang, “Deformation expression of soft tissue based on BP neural network,” Intelligent Automation & Soft Computing, vol. 32, no. 2, pp. 1041–1053, 2022. [Google Scholar]

69. Y. Li, J. Pan, J. Long, T. Yu, F. Wang et al., “Multimodal BCIs: Target detection, multidimensional control, and awareness evaluation in patients with disorder of consciousness,” Proceedings of the IEEE, vol. 104, no. 2, pp. 332–352, 2015. [Google Scholar]

70. H. V. Chand and J. Karthikeyan, “CNN based driver drowsiness detection system using emotion analysis,” Intelligent Automation & Soft Computing, vol. 31, no.2, pp. 717–728, 2022. [Google Scholar]

71. Y. Huang, J. Yang, P. Liao and J. Pan, “Fusion of facial expressions and EEG for multimodal emotion recognition,” Computational Intelligence and Neuroscience, vol. 2017, pp. 1–8, 2017. [Google Scholar]

72. Y. Huang, J. Yang, S. Liu and J. Pan, “Combining facial expressions and electroencephalography to enhance emotion recognition,” Future Internet, vol. 11, no. 5, pp. 105, 2019. [Google Scholar]