Neurological Assessment in Virtual Reality Through Behavioral Response Analysis
Hawkar Oagaz (Ph.D. student),  Breawn Schoun (Ph.D. student), Manpreet Pooji (B.F.A. student) & Dr. Min Choi (CMTC Co-Director)

Virtual baseball environment for assessment of visual cognition and behavioral responses. The system analyzes the user’s attention to the flying ball, responses to visual and spatial stimuli, and movement.

Virtual baseball environment for assessment of visual cognition and behavioral responses. The system analyzes the user’s attention to the flying ball, responses to visual and spatial stimuli, and movement.

Cognitive impairment in an individual presents itself in various ways but share similar symptoms like motor disorders and problems in visual cognition. There is no general-purpose tool that analyzes motor behavior to assess cognitive function. So we propose a novel method of cognitive function assessment that tests the patient’s cognition using virtual reality (VR) and motion analysis. By placing the patient in a controlled virtual environment and analyzing their movements, we can evoke certain physical responses from subjects for assessment. We have developed a prototype system that places the subject in a virtual baseball field and captures their full body motion as they try to catch baseballs. This scenario tests the subject’s ability to determine the landing time and position of the ball, as well as the user’s balance, motor skills, and attention.

Problem that Inspired Research:
Lacking a general-purpose cognitive assessment tool that could offer an objective comparison between different neurological disorders was the inspiration for this research. Impairments such as traumatic brain injury (TBI), Parkinson diseases (PD), Alzheimer diseases (AD) and multiple sclerosis (MS) all could be assessed using the developed system. 

Objective / Proposed Solution:
This system places the user in a virtual baseball field to play an outfielder role in the game. The system records the user’s data during their attention, reaction to the flying ball, and tracks the user’s body motion for balance and motor purposes. All recorded data will be used for analyzing later to assess the cognitive impairment in the user. Having such a general-purpose tool might offer an objective comparison between different cognitive impairments, and thus, it’s designed to be an easy-use tool for neurologists and psychiatrists.

Greatest Challenge to Overcome:
Building general-purpose tools are challenging due to the broad problems we are facing. In addition, in our system we have to deal with many sensors and collect different type of data to have overall understating for a patient that is being tested within the system. We have used available devices such as HTC Vive and Microsoft Kinect to collect needed data. However, we need more sensors for both better data quality and increased data recording. That being said, the greatest challenge is collecting data that could be as pure as possible. As computer scientists, another challenge is researching on neurological cognitive impairments and understanding them correctly in order to make a system that is able to obtain needed data. 

Benefits of Research:
Neurologists and psychiatrists will be the people who benefit the most from our system to be used in clinics to assess neurological disorders. 

Real-World Application(s):
Our system has a potential to be used in clinical therapy for cognitive impairment patients. 

Innovations to Media and Technology:
Our system combines several sensors integrated into a virtual environment to collect precise data from the user. This system contributes to the technology world more than to the media world. 

Cutting-edge Technology Being Used:
Currently, our system uses the common technology available including HTC Vive and Microsoft Kinect. However, we plan to use the brain-computer interface (BCI) to record the brain signal of users while playing baseball. Eye tracking and recording breathing data using the thermal camera are two other ideas that we plan to integrate in the future. 

Transdisciplinary Collaboration:
This research is a collaboration among computer science field and neurology/psychology fields. In the computer science side, we provide a system that records all data related to the user’s attention and reaction to visual stimuli and capture body motion. We provide the recorded data for neurologists and psychologist to analyze the cognitive impairment in patients.