DOI:10.32604/cmc.2021.014550
| Computers, Materials & Continua DOI:10.32604/cmc.2021.014550 | |
| Article |
Cognitive Skill Enhancement System Using Neuro-Feedback for ADHD Patients
1Al-khawarizmi Institute of Computer Science, University of Engineering and Technology, Lahore, 54890, Pakistan
2Department of Computer Science, University of Engineering and Technology, Lahore, 54890, Pakistan
3Artificial Intelligence and Data Analytics Lab, CCIS Prince Sultan University, Riyadh, Saudi Arabia
4College of Computer Engineering and Science, Prince Sattam bin Abdulaziz University, Alkharj, Saudi Arabia
*Corresponding Author: Usman Tariq. Email: u.tariq@psau.edu.sa
Received: 28 September 2020; Accepted: 27 December 2020
Abstract: The National Health Interview Survey (NHIS) shows that there are 13.2% of children at the age of 11 to 17 who are suffering from Attention Deficit Hyperactivity Disorder (ADHD), globally. The treatment methods for ADHD are either psycho-stimulant medications or cognitive therapy. These traditional methods, namely therapy, need a large number of visits to hospitals and include medication. Neurogames could be used for the effective treatment of ADHD. It could be a helpful tool in improving children and ADHD patients’ cognitive skills by using Brain–Computer Interfaces (BCI). BCI enables the user to interact with the computer through brain activity using Electroencephalography (EEG), which can be used to control different computer applications by processing acquired brain signals. This paper proposes a system based on neurofeedback that can improve cognitive skills such as attention level, mediation level, and spatial memory. The proposed system consists of a puzzle game where its complexity increases with each level. EEG signals were acquired using the Neurosky headset; then sent the signals to the designed gaming environment. This neurofeedback system was tested on 10 different subjects, and their performance was calculated using different evaluation measures. The results show that this game improves player overall performance from 74% to 98% by playing each game level.
Keywords: Neurogaming; brain–computer interface; attention deficit hyperactivity disorder; electroencephalography; spatial memory; cognitive skills; attention level
Attention Deficit Hyperactivity Disorder (ADHD) is a mental disorder that is common in children. People who have ADHD have symptoms such as inattention (unable to give attention to a specific task, loose focus easily, face difficulty to concentrate for a longer time and complete tasks, impulsivity (interrupting people, perform actions without knowing the outcomes, prematurely express a feeling, want everything quickly), hyperactivity (restlessness feelings, can’t sit still, excessively talking on something). According to a secondary data investigation and cross-sectional of the National Health Interview Survey (NHIS), 2017, 2,965 children had ADHD and data analyses for this study involved logistic regression analyses and Chi-square bivariate analyses [1]. Children who have ADHD leads to hindering academic achievement and social interactions.
The available methods for treating children with ADHD are either psycho-stimulant or cognitive therapies [2]. This cognitive training help ADHD patients to advance their cognitive skills such as attention [3]. The assessment method that is accepted for ADHD is through clinical judgment, which contains a range of questions based on psychometric and progress of children and parents, which help monitor patients’ cognitive behavior [4]. The clinical valuation method of ADHD is complex by a high co-existence with other disorders, making a differential analysis complicated and often resulting in numerous clinical visits to get an accurate and precise diagnosis and extensive cost to the health services [5].
The other method is QbTest desktop-based system, which tracks head motion during the continuous perforce test (CPT) that is designed to calculate the attention and impulse control with a high-resolution moment-tracking system to measure hyperactivity and shows the result in the form of graphs where patients check their performance during the test [6]. The QbTest (QbTest Ltd., www.qbtech.com) is a commercially accessible measure of ADHD symptoms permitted by the FDA (Ref: K133382) associated with a cognitive test. Although it must not use CTP as a stand-alone device for valuation, it can be combined with other information, such as teachers’ and parents’ reports and clinical assessments, to reach a diagnostic conclusion [7]. According to a study on management and diagnosis of ADHD in adults, young people, and children in the National Institute for Health and Care Excellence (NICE) [8], treatment for ADHD include cognitive-behavioral therapy (CBT), social skill training, and parenting-training and drug treatment which is more effective. Therapies and monitoring of cognitive behavior using games are not in common practice but require extensive research to develop such treatment methods that could provide a simpler, natural, real environment to improve attention skills in children with ADHD [9,10]. They develop games having motivational actions, which can ensure improvement in attention level [11].
Research shows that Neurogaming can be used for treating ADHD in children to some extent. It has the potential to highlight some of the shortcomings of current behavior management-based therapies. One of the most difficult tasks with children suffering from ADHD is to sustain their attention [12]. Neurofeedback is a rehabilitation method that aims to modulate and retrain brain activity to resolve physiological symptoms [13]. One of the original proofs of cognitive-behavioral therapy potency included the Sensory-Motor Rhythm (SMR), a low-beta EEG rhythm originating from the scalp’s EEG area situated above the sensory-motor line [14]. Neurofeedback games and applications designed to improve focus, attention, and neurotechnology can improve skills such as visual focus and memory training modules, which also help improve the skills in which ADHD children struggle [15].
Brain–computer interface (BCI) is an emerging technology involving software and hardware communication systems that can control external devices through brain signals [16]. BCI system has an input, output, and an algorithm that maps input from the brain EEG signals to output on computer and external devices in the form of commands [17]. EEG is a technique used to measure the brain’s electrical activity produced by the neuron’s activity. Brain signals reflect ongoing brain dynamics and how the brain function for a time, which present a series of fluctuation in brain signals that have different waveforms and amplitude of signals depending upon the cognitive state of humans. These signals can be captured by a noninvasive technique in which electrodes are placed on the scalp to measure the electrical activity [18].
BCI is being used to aid individuals with disabilities by providing communication mechanisms to control various applications in multiple domains, i.e., health, rehabilitation, and entertainment [19]. Brain signals classification involves four main steps: brain data acquisition, preprocessing, feature selection, feature extraction, and classification. i) Acquisition of brain signal is the measurement of brain signal using sensors on the scalp, which can be captured using different EEG headsets. ii) preprocessing the brain signal in which the raw signal is processed to remove noise using various filters and hence enhance the quality of signal iii) feature selection and extracting of the desired features iv) classification in which the signal is classified into different classes based on the extracted features.
The early BCI devices were designed mainly for clinical and research purposes due to their large and complex signal acquisition methods [20]. Nowadays, BCI applications are being developed for healthy people, such as Neurogames, enhancing the user’s learning capabilities. Therefore, many researchers are now working on developing BCI applications in the field of entertainment [21]. These applications are developed by using different BCI headsets available in the market and are easy to use. The brain data from these headsets are captured from sensors such as Emotive EPOC and Neurosky Mindwave mobile [22].
Research shows that the Neurofeedback training (NFT) system for enhancing patients’ cognitive abilities has been developed [23]. This game-based NFT system contains multiple games specially designed to improve patients’ cognitive performance and attention/focus. These systems calculate the attention level by checking alpha and beta wave values [24]. The researchers performed different sessions and calculated cognitive functions using the Cambridge Neuropsychological test before and after the treatment. NFT improved special working memory (SWM) and visual processing [25]. It found that treatment with NFT improves attention, executive functions, and SWM [26]. The headset used for capturing brainwaves in these systems is complex and has a large number of electrodes.
In this paper, a system based on neurofeedback has been proposed. This system has multiple levels specially designed for children and ADHD patients, which will enhance their cognitive skills. In this system, we used the Neurosky headset to capture the brain signal with only one electrode to be placed on the forehead and easy to use for the user compared to other multi-electrodes headsets. Neurofeedback-based games will provide interactive, integrated games with the brain, which will improve mental states. These games will provide psychiatrists opportunities by combining clinical applications with neurofeedback, such as feedback therapy, neurorehabilitation, and neuro-therapies. This system will enhance the cognitive skills and overall learning performance of the user.
In the proposed system, EEG data were acquired using the Neurosky mind wave, and data was then analyzed using the Fast Fourier transform (FFT). Different features then extracted such as Alpha, beta, Gemma, and theta, brain waves. Separated FFT bands were used to measure attention and mediation values. These values then integrated into the designed game. The game consists of different puzzle cards that can only be flipped using user attention and meditation levels. The meditation phase can achieve when the user maintains a meditation level reach above 50 for 5 seconds. Following that user enters in attention phase where the attention level should be above 50. Users can get direct feedback of attention from the Neurofeedback game in the form of card flipping. The player has to match all the cards in a given time and for matching, users have to memorize them. The complete system diagram is shown in Fig. 1.
Figure 1: Complete system workflow diagram
For Brain EEG signal acquisition, Neurosky Mindwave mobile headset was used, which provides a non-wired Bluetooth connection that is a commercially available device for translating brainwaves. It provides real-time neurofeedback and extracts the user’s emotional state using an eSense meter for monitoring attention, mediation, raw EEG, and other EEG frequency band values. Attention indicates the user level of mental focus, and meditation level shows the user’s mental relaxation level [27]. Brain waves that are extracted from the EEG headset are difficult to visualize and process. The signal contains artifacts that are hard to eliminate. Therefore, different preprocessing and filtering techniques are applied. Fast Fourier transform (FFT) is a most important algorithm in data analysis and signal processing as it divides the signal into constituent sinusoidal frequency components. FFT is a translation of signal from the time domain to the frequency domain. FFT based on a specific number of samples is usually performed using Discrete Fourier transform (DFT). DFT is not a separate transform. It is used in signal processing. In DFT, when we have x0 to xn-1 number of samples, the formula to calculate transform is given below in Eq. (1)
where,
Fast Fourier transform (FFT) analysis allows quick and accurate identification of signal components in the brain signal. While performing FFT analysis, one should remember that biological signals are never a sinusoidal signal but rather a component of many. After applying FFT, we can visualize the brain data and user emotional state by dividing the signal into different frequency bands [28].
These frequency bands help to detect the attention and focus of the user. As Gemma waves reflect the user’s consciousness mechanism and range from 31 HZ and greater, beta waves are associated with the user’s attention and cognition. Beta waves range from 12 HZ to 30 HZ, often divided into small and fast waves as
The game has been designed using the Unity 3D platform, which is a cross-platform engine to develop immersive two-dimensional, three-dimensional, and different virtual and augmented reality experiences. Unity 3D is compatible with different graphics engines such as Maya, Blender, Direct3D, and OpenGL, and efficient workflows and built-in UI systems help create high-performance gameplays. Therefore, the functionality for this game has been developed using the Unity platform. The game is based on a puzzle that consists of multiple difficulty levels. The desired cognitive abilities extracted in this game are attention level, meditation level, memory, and time management. The game is designed so that for playing this game, the attention and mediation level of players should be 50. This puzzle has pairs of cards hidden at the start of the game, and users can flip these cards using their attention and meditation.
The game works on the following parameters:
• To start the game (when you can see hidden images one by one), the player needs to enter in meditation phase.
• For entering in meditation phase, the meditation level should once reach 50, and then it should remain over 30 for 5 seconds. Otherwise, the meditation phase is canceled.
• After the meditation phase, the game works on the attention level.
• The hidden image is revealed once the attention level reaches 50.
• The card displayed to the player for a very shorter period, and then the next card displayed.
• The player has to match the cards containing identical images.
• Now the player can memorize them and match them.
• On every match, 1 score is added.
• Once two identical cards have been matched, they disappear from the gameplay, leading the player closer to the finish goal, i.e., matching all the pairs to each other. So, attention and meditation levels are used to uncover the cards.
The Game has multiple levels increasing its complexity level at each level. The number of cards in each level increase to find the matching pairs that are placed randomly. So, the user has to memorize and then match them. The game starts with four cards and the last level has sixteen cards as the number of cards keeps increasing with each level to increase the game’s complexity. Moreover, each level has a timer associated with it to complete the game within the allocated time. Otherwise, the game will be over. The user has to memorize the cards and match them, increasing or decreasing the game score based on the count of all successful attempts. Each level and attention and meditation values graphs of level 1, level 2, level 3, and level 4 are shown in Figs. 2–5.
Figure 2: (a) Attention values in puzzle game of level 1 (b) Graphical representation of Attention values w.r.t time in level 1
Figure 3: (a) Attention values in puzzle game of level 2 (b) Graphical representation of Attention values w.r.t time in level 2
Figure 4: (a) Attention values in puzzle game of level 3 (b) Graphical representation of Attention values w.r.t time in level 3
Figure 5: (a) Attention values in puzzle game of level 4 (b) Graphical representation of Attention values w.r.t time in level 4
The Neurosky is integrated with the game using the ThinkGear library as a plugin in Unity to get brain control signals from the Neurosky Mindset. The control flow diagram of the ThinkGear connection of Neurosky is shown in Fig. 6.
Figure 6: Control flow diagram of ThinkGear
The designed game has been tested on ten players of age ranging from 20 to 30 years. The users were instructed to wear the Neurosky headset to start the game. The user had played different levels, and each player had completed the game within different time slots. Statistics of each participant were recorded at that time for all levels and the completion time of different participants was analyzed to measure their performance. The analysis of results showed that the average attention level within each level was 50, which is shown in Figs. 7a–7d. The completion time analysis of level 1 shows that subjects 4 and 9 took the maximum time to complete that level. Similarly, subject 1 and subject 9 took the maximum time to complete level 2. It has been observed that subject 7 took the maximum time to complete level 3 and subject 9 took the second maximum time. The subject 9 competition time was greater than all the levels as compared to other subjects.
Figure 7: (a) Comparison of attention level of different subjects in level 1 (b) Comparison of the attention level of different subjects in level 2 (c) Comparison of the attention level of different subjects in level 3 (d) Comparison of the attention level of different subjects in level 4
3.1 Performance Measure Criteria
In the proposed study, the performance matrix was measured to evaluate all subjects’ performance based on completion time, attempts, and game scores, which can be calculated using Eq. (2).
where,
ts is a total score given to each level which is 5.
Completion and the ideal time for each level are given below in Tabs. 1–4 and the performance of each player can be measured by using the parameters given below:
• Level 1 has 120 seconds (2 Minutes) completion time and 40 second ideal completion time.
• Level 2 has 240 seconds (4 minutes) completion time and 80 seconds is ideal completion time.
• Level 3 has 360 seconds (6 Minutes) completion time and 120 second ideal completion time.
• Level 4 has 480 seconds (8 Minutes) completion and 160 second ideal completion time.
Table 1: Level 1 performance measure for different subjects
Table 2: Level 2 performance measure for different subjects
Table 3: Level 3 performance measure for different subjects
Table 4: Level 4 performance measure for different subjects
When the player focuses enough where the mediation and attention level reach the required level, the cards are shown and the user can memorize them for matching purposes. Here, card 4 is displayed to the user when the value in the attention bar is more than the required game attention value, as you can see in the in-game view as well as in Fig. 8.
Figure 8: (a) Card 4 is displayed with required attention values in game (b) Graphical representation of attention values w.r.t time
At the start of the game, every participant took more time to bring their attention level to the required levels. However, with time and practice in different sessions, they could easily focus on images and their attention level improved. For testing purposes, only one participant played level 4 several times. When he played the first time, he took 281 sec to complete level 4. The second time, he took 248 sec to complete the same level and so on. When he played the sixth time, he completed the same level 4 within just 165 sec. This means in every next session user took a short time to refocus his attention. Performance measures on the same level and same subject with multiple trials are shown in Tab. 5 and Fig. 9. They observed that the player’s performance increases with each trial. By playing these games, players’ performance has been improved in level 4 from 74% to 98%.
Table 5: Performance measures of the subject on different trials
Figure 9: Improvement in attention level on the subject in different trials
In this paper, a neuro game has been designed for children with attention deficit hyperactivity disorder (ADHD) to improve their cognitive abilities such as memory span, attention, and time management based on the user’s attention and meditation values. Neurosky headset has been used for the EEG signal that was integrated with the designed game. This brain-controlled game has multiple levels based on cognitive features. Every level has a puzzle, pairs of cards that players have to match, and users have to memorize them for matching. Game scores have been measured for every correct and wrong matching, and each level has a completion time where a player should have to complete the game within the given time. Ten subjects participated in this game and performance has been measured based on the player’s completion time, memorization of cards, and game score, which enhance the player’s attention, memory, and time management skills. Every subject’s performance at different levels has been studied and observed that playing these game subjects has improved their performance in level 4 from 74% to 98%. Performance comparison of different subjects shows that the performance of a subject increases with every new trial.
Therefore, the results show that this BCI Integrated system can improve user attention and focus. They can improve their focus, which can help parents and psychiatrists monitor children’s cognitive abilities. In the future, we will develop more immersive and interactive games using advanced techniques such as Virtual Reality (VR) and Augmented Reality (AR) that will provide a more attractive gaming environment for ADHD patients.
Funding Statement: The author(s) received funding for this study under Technology Development Fund (TDF-02-228). The research is also supported by AIDA Lab CCIS Prince Sultan University Riyadh Saudi Arabia and authors would also like to acknowledge the support of Prince Sultan University for paying the Article Processing Charges (APC) of this publication.
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
1. R. C. Wiener, C. Waters, R. Bhandari, R. Constance, A. K. T. Shockey et al., “Healthcare utilization and morbidity among adolescents with ADHD in children aged 11–17 years, NHIS, 2017,” Child Development Research, vol. 2019, Article ID 4047395, 10 pages, 2019. [Google Scholar]
2. C. Advokat and M. Scheithauer, “Attention-deficit hyperactivity disorder (ADHD) stimulant medications as cognitive enhancers,” Frontiers in Neuroscience, vol. 7, pp. 82, 2013. [Google Scholar]
3. A. Achunair and H. M. Tairan, “Recent advances in treating patients with attention deficit hyperactivity disorder: A review of nonpharmacological interventions,” in Critical Reviews in Physical and Rehabilitation Medicine, vol. 31. USA: Begell House Inc., pp. 233–251, 2019. [Google Scholar]
4. M. A. Arruda, R. Arruda and L. Anunciação, “Psychometric properties and clinical utility of the executive function inventory for children and adolescents: A large multistage populational study including children with ADHD,” Applied Neuropsychology: Child, vol. 10, pp. 1–17, 2020. [Google Scholar]
5. M. A. Pettie and Michaela, “When genetic and environmental factors meet in modelling autism,” 2020. [Google Scholar]
6. C. L. Hall, A. Z. Valentine, G. M. Walker, H. M. Ball, H. Cogger et al., “Study of user experience of an objective test (QbTest) to aid ADHD assessment and medication management: A multi-methods approach,” BMC Psychiatry, vol. 17, pp. 1–12, 2017. [Google Scholar]
7. S. Pellegrini, M. Murphy and E. Lovett, “The QbTest for ADHD assessment: Impact and implementation in child and adolescent mental health services,” Children and Youth Services Review, vol. 114, pp. 105032, 2020. [Google Scholar]
8. National Collaborating Centre for Mental Health (UK“Attention deficit hyperactivity disorder: Diagnosis and management of ADHD in children, young people and adults,” PMID: 22420012, 2009. [Google Scholar]
9. R. Alderson, L. J. Matt, K. L. Hudec and C. H. Patros, “Attention-deficit/hyperactivity disorder (ADHD) and working memory in adults: A meta-analytic review,” Neuropsychology, vol. 27, no. 3, pp. 287–302, 2013. [Google Scholar]
10. C. A. Denton, L. Tamm, C. Schatschneider and J. N. Epstein, “The effects of ADHD treatment and reading intervention on the fluency and comprehension of children with ADHD and word reading difficulties: A randomized clinical trial,” Scientific Studies of Reading, vol. 24, no. 1, pp. 72–89, 2020. [Google Scholar]
11. J. Parong, A. Wells and R. E. Mayer, “Replicated evidence towards a cognitive theory of game-based training,” Journal of Educational Psychology, vol. 112, no. 5, pp. 922–937, 2020. [Google Scholar]
12. H. Lamothe, E. Acquaviva, J. Baleyte and R. Delorme, “Is sustained attention deficit related to subclinical obsessive thoughts in children and adolescents with ADHD?,” International Journal of Psychiatry in Clinical Practice, 2020. [Google Scholar]
13. J. Luigjes, R. Segrave, N. de Joode, M. Figee and D. Denys, “Efficacy of Invasive and noninvasive brain modulation interventions for addiction,” Neuropsychology Review, vol. 29, no. 1, pp. 116–138, 2019. [Google Scholar]
14. M. Arns, C. R. Clark, M. Trullinger, R. deBeus, M. Mack et al., “Neurofeedback and attention-deficit/hyperactivity-disorder (ADHD) in children: Rating the evidence and proposed guidelines,” Applied Psychophysiology Biofeedback, vol. 45, pp. 39–48, 2020. [Google Scholar]
15. B. Kerous, F. Skola and F. Liarokapis, “EEG-based BCI and video games: A progress report,” Virtual Reality, vol. 22, no. 2, pp. 119–135, 2018. [Google Scholar]
16. A. N. Belkacem, N. Jamil, J. A. Palmer, S. Ouhbi and C. Chen, “Brain computer interfaces for improving the quality of life of older adults and elderly patients,” Frontiers in Neuroscience, vol. 14, Article 692, Frontiers Media S.A., 2020. [Google Scholar]
17. U. Chaudhary, N. Mrachacz-Kersting and N. Birbaumer, “Neuropsychological and neurophysiological aspects of brain–computer-interface (BCI)-control in paralysis,” Journel of Physiology, vol. 2020, pp. 1–9, 2020. [Google Scholar]
18. A. Salami, J. Andreu-Perez and H. Gillmeister, “Symptoms of depersonalisation/derealisation disorder as measured by brain electrical activity: A systematic review,” Neuroscience & Biobehavioral Reviews, vol. 118, pp. 524–537, 2020. [Google Scholar]
19. N. Pellas and S. Mystakidis, “A systematic review of research about game-based learning in virtual worlds,” Journal of Universal Computer Science, vol. 26, no. 8, pp. 1017–1042, 2020. [Google Scholar]
20. H. Yoon, S. W. Park, Y. K. Lee and J. H. Jang, “Emotion recognition of serious game players using a simple brain computer interface,” in Int. Conf. on ICT Convergence, Jeju, South Korea, pp. 783–786, 2013. [Google Scholar]
21. M. A. M. Joadder, J. J. Myszewski, M. H. Rahman and I. Wang, “A performance based feature selection technique for subject independent MI based BCI,” Health Information Science and Systems, vol. 7, no. 1, pp. 15, 2019. [Google Scholar]
22. P. Sawangjai, S. Hompoonsup, P. Leelaarporn, S. Kongwudhikunakorn and T. Wilaiprasitporn, “Consumer grade eeg measuring sensors as research tools: A review,” IEEE Sensors Journal, vol. 20, no. 8, pp. 3996–4024, 2020. [Google Scholar]
23. K. C. J. Eschmann, R. Bader and A. Mecklinger, “Improving episodic memory: Frontal-midline theta neurofeedback training increases source memory performance,” Neuroimage, vol. 222, pp. 117219, 2020. [Google Scholar]
24. N. S. Karuppusamy and B. Y. Kang, “Multimodal system to detect driver fatigue using eeg, gyroscope, and image processing,” IEEE Access, vol. 8, pp. 129645–129667, 2020. [Google Scholar]
25. A. Nesayan, R. Asadi Gandomani and N. Moin, “Effect of neurofeedback on perceptual organization, visual and auditory memory in children with attention deficit/hyperactivity disorder,” Iranian Journal of Child Neurology, vol. 13, no. 3, pp. 75–82, 2019. [Google Scholar]
26. S. Jirayucharoensak, P. Israsena, S. Pan-ngum, S. Hemrungrojn and M. Maeset, “A game-based neurofeedback training system to enhance cognitive performance in healthy elderly subjects and in patients with amnestic mild cognitive,” 2019. [Online]. Available: ncbi.nlm.nih.gov. [Google Scholar]
27. N. S. Bonfiglio, D. Parodi, D. Rollo, R. Renati, E. Pessa et al., “Use of training with BCI (brain computer interface) in the management of impulsivity,” in IEEE Medical Measurements and Applications, MeMeA 2020–-Conf. Proc., Italy, pp. 1–5, 2020. [Google Scholar]
28. S. Paszkiel, “Control based on brain–computer interface technology for video-gaming with virtual reality techniques,” Journal of Automation Mobile Robotics and Intelligent Systems, vol. 10, no. 4, pp. 3–7, 2016. [Google Scholar]
29. R. B. Pachori, R. Sharma and S. Patidar, “Classification of normal and epileptic seizure eeg signals based on empirical mode decomposition,” in Complex System Modelling and Control Through Intelligent Soft Computations. USA: Springer, pp. 367–388, 2015. [Google Scholar]
 | This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |