DOI:10.32604/cmc.2021.018080
| Computers, Materials & Continua DOI:10.32604/cmc.2021.018080 | |
| Article |
Virtual Reality-Based Random Dot Kinematogram
1Department of Software Convergence, Soonchunhyang University, Asan, 31538, Korea
2Research Administration Team, Seoul National University Bundang Hospital, Seongnam, 13620, Korea
3Dizziness Center, Clinical Neuroscience Center, Department of Neurology, Seoul National University Bundang Hospital, Seongnam, 13620, Korea
4Department of Neurology, Seoul National University College of Medicine, Seoul, 03080, Korea
5Department of Neurology, College of Medicine, Soonchunhyang University, Bucheon, 14584, Korea
6Department of Computer Software Engineering, Soonchunhyang University, Asan, 31538, Korea
*Corresponding Author: Min Hong. Email: mhong@sch.ac.kr
Received: 24 February 2021; Accepted: 30 March 2021
Abstract: This research implements a random dot kinematogram (RDK) using virtual reality (VR) and analyzes the results based on normal subjects. Visual motion perception is one of visual functions localized to a specific cortical area, the human motion perception area (human analogue for the middle temporal/middle superior temporal area) located in the parieto–occipito–temporal junction of the human brain. The RDK measures visual motion perception capabilities. The stimuli in conventional RDK methods are presented using a monitor screen, so these devices require a spacious dark room for installation and use. Recently, VR technology has been implemented in different medical domains. The test method proposed in this study include a VR-based RDK that can independently measure human motion perception abilities without any spatial constraints via a VR head-mounted display. Subsequently, the VR-based RDK was implemented, and the visual perception abilities of the normal subjects were measured based on varying coherences. In both screen- and VR-based RDK tests, the easier the stimulus is, the higher the correct answer rate and the shorter the reaction time. No significant differences in coherence thresholds were observed between the two test methods. The VR-based RDK proposed in this study can be used as a diagnosis tool for visual motion perception and neurodegenerative disorders affecting the posterior region of the brain.
Keywords: Random dot kinematogram; virtual reality; visual psychophysics; diagnostic device
Random dot kinematogram (RDK) test sets a coherence value among random dots displayed on a screen, where the designated dots move in a specific direction defined by the ratio and the remaining dots move randomly as noise [1]. The RDK test is effective for measuring visual motion perception in spatiotemporal dimensions. However, as the conventional RDK is measured by displaying the moving dots on a screen, certain spatial and environmental constraints require control over the surrounding environment, such as the size of the screen, the viewing distance between the test subject and screen, and the overall brightness of the laboratory environment.
Thus, we propose a virtual reality (VR)-based RDK in this study to overcome these limitations. As the VR-based RDK can control various measurement conditions such as screen size, field of view, and dark-room environment composition using VR technology, the visual perception ability can be measured in specific to the individual body shape and unique viewing angle of the subject. In addition, the effectiveness of the VR-based RDK was validated by conducting a comparative experiment with screen-based RDK. Therefore, this study demonstrated that the proposed VR-based RDK could conveniently test the visual motion perception without space constraints, and the coherence threshold remained unvaried from the conventional screen-based RDK in normal subjects.
In RDK tests, a certain percentage of dots are designated as signal dots that display unified motion in a single coherent direction, and the remaining percentage of dots are regarded as noise dots moving in random directions, as depicted in Fig. 1. The ratio of the number of signal-to-noise dots is defined as the coherence. The direction of the signal dots and coherence were randomly presented to the subjects, who were required to enter the direction of the perceived global motion. A correct response was recorded when the subject identified the same direction as that in the stimulus. Overall, the main objective of this test was to measure the subjects’ perception of visual motion that is primarily conducted at the human motion perception area (human analogue for the middle temporal/middle superior temporal area, hMT+) located in the parieto–occipito–temporal junction of the human brain [2–6]. In particular, extensive damage to the bilateral posterior cortical area causes complete loss of visual motion perception, which is a clinical syndrome called akinetopsia [7–11]. In less severe instances, impaired visual motion perception is evident in diseases with posterior cortical dysfunction, such as the Alzheimer’s disease [12,13], Parkinson’s disease [14], and autism spectrum disorders [15]. Therefore, the visual motion perception measured using the RDK can be applied to various clinical situations that may assist in the early diagnosis of neurodegenerative diseases.
Figure 1: Signal coherence of random dot kinematogram
2.2 Clinical Uses of VR Technology
VR is a simulation technology that can realize and provide the experience of a real world via a computer-generated virtual world. Recently, various hardware technologies have been developed for introducing VR environments in several diverse applications. As the display is the most important element of a VR system, various types of display hardware have been developed for practical use. The latest development of display hardware for VR is the head-mounted display (HMD), which blocks the outside view to induce a sense of virtual immersion. Based on the user’s movement, the HMD device seamlessly tracks the current position and orientation to continually produce the virtual world and immerse the user’s entire field of view [16].
Conventional computer-based applications allow the users to interact with the environment using a keyboard and mouse, but the HMD-based VR applications employ user-friendly controllers such as remote controls, sensor gloves, and treadmills [17]. Therefore, numerous currently implemented VR applications use HMDs for highly enhancing the user experience in the virtual environment with striking similarities to the real world [18]. Recent research has confirmed the possibility of utilizing VR as an alternative route for medical intervention and delivery. In particular, VR related technology has is used as a technology for assisting the diagnosis and rehabilitation of stroke, cerebral palsy, severe burns, and Parkinson’s disease. The VR is advantageous as a medical tool as it saves space and provides real-time feedback in a convenient interface to ensure safe practice and training environment. In addition, VR extends the ability to customize treatment requirements including the improved ability to coordinate assessment and training procedures [19].
Furthermore, VR is classified as immersive, semi-invasive, and non-invasive. Research on neuroplasticity has revealed that VR rehabilitation is positively linked to active participation and motivation during intervention. In addition to motivation, VR rehabilitation enhances nerve reconstruction, which apparently optimizes rehabilitation outcomes for children with cerebral palsy. As a consequence of reviewing the rising number of related studies, the accruing evidence indicates that VR rehabilitation improves sensory and functional motor skills. Moreover, VR rehabilitation scenarios suggestively facilitate practice learning, retention of learned skills, and skill transfer in real situations. In particular, several studies demonstrated that VR rehabilitation improves posture and balance, upper limb function, joint control, and gait. Thus, physicians can systematically design and control intervention strategies by establishing augmented biofeedback in neuromotor rehabilitation [20].
3 Design and Implementation of VR-Based RDK
The screen-based RDK was simultaneously implemented in the Unity 2019.2.9.1f environment to confirm the applicability and suitability of the VR-based RDK; the hardware specifications of each RDK are listed in Tab. 1.
Table 1: Hardware specifications for screen and VR-based RDK
3.1 Basic Structure of VR-Based RDK
The basic structure of the VR-based RDK is illustrated in Fig. 2. The basic information regarding the subject was entered. Subsequently, the VR-based RDK drives the noise section that causes the random movement of the dots as well as the signal section from which the subject should determine the movement of the dots. Thereafter, the VR-based RDK is terminated on the instant the subject enters the movement direction of the dots. Simultaneously, the CSV writer function stores the subject’s individual information including subject data, time data, coherence data, input data, and reaction time data.
Figure 2: Structure of the VR-based random dot kinematogram
Figure 3: Flow chart of the VR-based random dot kinematogram
A detailed flow chart of the VR-based RDK is presented in Fig. 3. Subjects first perform the tutorial task, and afterwards, easy and difficult stimuli are presented in order. Difficulty levels are determined by the coherence ratio of the signal dots: 100% or 90% for tutorial level, 70%, 60%, 50%, or 40% for easy level, and 30%, 25%, 20%, or 15% for the hard level move in the same direction. Each VR-based RDK was repeated 10 instances for a particular subject, and the entire data were automatically saved in a .csv file as the subject’s entry was completed.
3.2 Test Scene of VR-Based RDK
The VR-based RDK test of the subject (left) including the screen being displayed by the HMD (right) is portrayed in Fig. 4. Both the screen-and VR-based RDKs were implemented to display identical scenes of moving dots onto the screen and the HMD, respectively. Furthermore, the screen-based RDK was displayed on the monitor screen during the VR-based RDK test as well to facilitate the examiner’s review of the test progress.
Figure 4: Test scene (left) and screenshot (right) of VR-based random dot kinematogram
4 Experimental Results for VR-Based RDK
All the experiments followed the tenets of the Declaration of Helsinki, and the study was approved by the Institutional Review Board of Seoul National University Bundang Hospital (B-2008/633-303). We recruited 20 normal subjects over 18 years of age; each subject provided a written informed consent to participate in the current study under the following inclusion criteria: 1) subjects without a history of dizziness, neurological disorders, eye movement disorders, or ophthalmic diseases; 2) subjects without a history of neuromuscular or musculoskeletal disorders that may significantly interfere with button operation using fingers; 3) subjects with normal cognitive function (score on the Korean version of Mini-Mental State Examination
Tab. 2 presents detailed information on the 20 subjects (mean age: 33.6), including 7 men and 13 women, who were randomly assigned for testing with the VR or the screen first.
At the tutorial level, which is relatively easy, a coherence of 100% and 90% was conducted four times for each direction (left: “ −”, right: “+”) of the moving dot to induce adaptation toward the operation of the VR-based RDK for the test subjects. Subsequently, the test was performed 10 times by randomly adjusting the coherence ratio to 70%, 60%, 50%, and 40% for the easy level, and thereafter, 30%, 25%, 20%, and 15% for the hard level. The average value of the correct score and reaction for each of the 10 tests were calculated and recorded in the VR-based RDK system. Regardless of the global motion acting in the left or right direction, the subject’s correct response rate increased and the reaction time decreased for a larger coherence (easier stimulation). This trend was consistently observed for both the screen and VR stimulation, as depicted in Figs. 5 and 6.
Table 2: Demographics of subjects
Figure 5: Mean correct response according to coherence
The smallest coherence with a correct answer rate > 75% was defined as the coherence threshold for each subject. The mean values of coherence thresholds for the screen-and VR-based RDKs were 17.8
Figure 6: Mean reaction time according to coherence
Figure 7: Mean values for coherence threshold
In this study, we designed and implemented a VR-based RDK that measured visual motion perception ability using a VR HMD with Unity engine. Unlike the screen-based RDK that operates on a monitor screen, the VR-based RDK is advantageous of being relatively independent of the space constraints. In addition, it provides an excellent environment with an enhanced sense of immersion for the test subjects. Moreover, the VR-based RDK is suitable for testing under various assigned conditions (inducing changes in the subject’s posture and varying environmental factors), and the examiner can conveniently review the subject’s response and test results in real time. However, the application of VR on patients under certain conditions (children, disabled personnel, novel users) could be challenging owing to the nature of VR. This VR-based RDK was designed and implemented under testing conditions for adults and the elderly, and its upgradation based on suitability for various ages and conditions is planned for future research. An additional advantage of VR-based RDKs is that the Unity engine is highly compatible with VR HMDs and can be readily applied to various VR hardware devices as well as operating systems. Moreover, future improvement and testing will be conducted to include various environments and locations for the subjects. Furthermore, the present research team will develop a new version of the RDK based on the current VR-based RDK by incorporating new motion variants for the dot objects in the existing RDK tests.
Funding Statement: This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (Nos. 2019R1F1A1062752 and 2019R1C1C1006539) and was supported by the Soonchunhyang University Research Fund.
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
1. K. R. Lee, T. D. Dague, K. V. Sobel, N. J. Paternoster and A. M. Puri, “Coherent global motion percepts from stochastic local motions,” ACM SIGGRAPH Computer Graphics, vol. 18, no. 1, pp. 24, 1984. [Google Scholar]
2. S. Zeki, J. D. G. Watson, C. J. Lueck, K. J. Friston, C. Kennard et al., “A direct demonstration of functional specialization in the human visual cortex,” Journal of Neuroscience, vol. 11, no. 3, pp. 641–649, 1991. [Google Scholar]
3. J. D. Watson, R. Myers, R. S. Frackowiak, J. V. Hajnal, R. P. Woods et al., “Area V5 of the human brain: Evidence from a combined study using positron emission tomography and magnetic resonance imaging,” Cerebral Cortex, vol. 3, no. 2, pp. 79–94, 1993. [Google Scholar]
4. R. B. H. Tootell, J. B. Reppas, K. K. Kwong, R. Malach, R. T. Born et al., “Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging,” Journal of Neuroscience, vol. 15, no. 4, pp. 3215–3230, 1995. [Google Scholar]
5. D. J. McKeefry, J. D. G. Watson, R. S. J. Frackowiak, K. Fong and S. Zeki, “The activity in human areas V1/V2, V3, and V5 during the perception of coherent and incoherent motion,” NeuroImage, vol. 5, no. 1, pp. 1–12, 1997. [Google Scholar]
6. J. Culham, S. He, S. Dukelow and F. A. J. Verstraten, “Visual motion and the human brain: What has neuroimaging told us?,” Acta Psychologica, vol. 107, no. 1, pp. 69–94, 2001. [Google Scholar]
7. J. Zihl, D. von Cramon and N. Mai, “Selective disturbance of movement vision after bilateral brain damage,” Brain, vol. 106, no. 2, pp. 313–340, 1983. [Google Scholar]
8. L. M. Vaina and A. Cowey, “Impairment of the perception of second order motion but not first order motion in a patient with unilateral focal brain damage,” Proceedings of the Royal Society B: Biological Sciences, vol. 263, no. 1374, pp. 1225–1232, 1996. [Google Scholar]
9. G. T. Plant and K. Nakayama, “The characteristics of residual motion perception in the hemifield contralateral to lateral occipital lesions in humans,” Brain, vol. 116, no. 6, pp. 1337–1353, 1993. [Google Scholar]
10. G. T. Plant, K. D. Laxer, N. M. Barbaro, J. S. Schiffman and K. Nakayama, “Impaired visual motion perception in the contralateral hemifield following unilateral posterior cerebral lesions in humans,” Brain, vol. 116, no. 6, pp. 1303–1335, 1993. [Google Scholar]
11. L. M. Vaina, “Selective impairment of visual motion interpretation following lesions of the right occipito-parietal area in humans,” Biological Cybernetics, vol. 61, no. 5, pp. 347–359, 1989. [Google Scholar]
12. G. C. Gilmore, H. E. Wenk, L. A. Naylor and E. Koss, “Motion perception and Alzheimer’s disease,” Journal of Gerontology, vol. 49, no. 2, pp. 52–57, 1994. [Google Scholar]
13. M. Rizzo and M. Nawrot, “Perception of movement and shape in Alzheimer’s disease,” Brain: A journal of Neurology, vol. 121, no. 12, pp. 2259–2270, 1998. [Google Scholar]
14. A. Jaywant, M. Shiffrar, S. Roy and A. Cronin-Golomb, “Impaired perception of biological motion in Parkinson’s disease,” Neuropsychology, vol. 30, no. 6, pp. 720, 2016. [Google Scholar]
15. C. E. Robertson, C. Thomas, D. J. Kravitz, G. L. Wallace, S. Baron-Cohen et al., “Global motion perception deficits in autism are reflected as early as primary visual cortex,” Brain, vol. 137, no. 9, pp. 2588–2599, 2014. [Google Scholar]
16. S. Lee, M. Hong, S. Kim and S. J. Choi, “Effect analysis of virtual-reality vestibular rehabilitation based on eye-tracking,” KSII Transactions on Internet & Information Systems, vol. 14, no. 2, pp. 826–840, 2020. [Google Scholar]
17. J. Choi, S. Kim, S. Kim and S. Kang, “Generating a ball sport scene in a virtual environment,” KSII Transactions on Internet and Information Systems, vol. 13, no. 11, pp. 5512–5526, 2019. [Google Scholar]
18. M. C. Howard, “A meta-analysis and systematic literature review of virtual reality rehabilitation programs,” Computers in Human Behavior, vol. 70, no. 17, pp. 317–327, 2017. [Google Scholar]
19. T. Rose, C. S. Nam and K. B. Chen, “Immersion of virtual reality for rehabilitation-Review,” Applied Ergonomics, vol. 69, no. 1, pp. 153–161, 2018. [Google Scholar]
20. D. K. Ravi, N. Kumar and P. Singhi, “Effectiveness of virtual reality rehabilitation for children and adolescents with cerebral palsy: An updated evidence-based systematic review,” Physiotherapy, vol. 103, no. 3, pp. 245–258, 2017. [Google Scholar]
 | This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |