Rajshekar Guda Subhash, a computer science master's thesis student, is using brain signals to interact with augmented and virtual reality, potentially eliminating the need for physical actions. His work has been accepted to the flagship conference in computer human interaction, CHI 2024.
Let's learn from him how this all works, his research experience and his advice to those considering the master's thesis option in computer science.
What is your research?
When eye tracking alone is used to select buttons in augmented or virtual reality, it becomes challenging for a computer to know whether we want to select something or are just passively observing it.
Our research looked at whether eye gaze can be paired with a mental switch driven by brain signals.
When the brain anticipates a future stimulus, it quiets down current neural activity to process the incoming change. This anticipatory process is called Stimulus Preceding Negativity (SPN).
Let's consider a user navigating a music interface. As the user gazes at the play button, they anticipate a change that will happen when the button is pressed. Their brain then reduces neural activity in preparation, resulting in an SPN wave.
The system, sensing the SPN’s presence, can automatically activate the play button. The user no longer has to make a deliberate action; their SPN brainwaves become the primary mode of interaction.
This research is still in its early stages, we have only just proven that the SPN has the potential to be used as an indicator of user intent. The next step is to build a real-time prototype that the user can use to seamlessly select things.
How could it impact the world?
The real-world application is in developing more intuitive interaction methods for AR/VR. It’s essentially a hands-free mouse for the spatial computing realm.
Some potential uses include:
-
Accessibility: it would allow people with motor impairments to navigate immersive environments effortlessly.
-
Seamless, intuitive, and quicker control in VR games and applications, thereby also improving immersion.
-
Efficient hands-free interactions in medical applications where both hands may be occupied.
-
Novel user interfaces that can adapt to the user's intentions and other mental states.
What was your research experience like?
It was a unique and challenging experience that required a lot of hard work, but it was equally rewarding as I learned a lot from it.
I was a bit worried about whether we would be able to detect the SPN or not because it was not a highly replicated finding, but if we did, it would be a game-changer for Brain-Computer Interfaces, and the results were better than we expected.
Being a master's student presented its own set of challenges. The study was not funded by any grants, so I had to invest a lot of my savings and time into this project. I handled all elements of the studies and data collection myself.
We collected enough data and had just the right amount of time to analyze it and write up a paper for the 2024 Computer Human Interaction (CHI) conference.
What was your experience working with your advisors like?
I was co-advised by Leanne Hirshfield (Computer Science & Institute of Cognitive Science, CU Boulder) and Anthony Ries (Research Scientist, DEVCOM Army Research Laboratory & U.S. Air Force Academy). This research would not have been possible if it weren't for their guidance.
Early in the project, Leanne suggested reaching out to industry folks who might be interested in serving on my thesis committee. That's how I came across Michael Proulx from Meta Reality Labs, who also saw the research's potential and actively participated in it and provided useful direction.
I enjoyed Anthony's hands-on approach and I could tell that he found the work very exciting. Sometimes, I would screenshot the graphs and text them to him at 6 am, and he would reply instantly with his thoughts, and we would bounce ideas back and forth.
What would you recommend to someone potentially considering a master's thesis?
It's intimidating at first because you have to work on a specific problem on the frontier of science and produce something novel in a limited timeframe.
If not, you end up having to delay your graduation and nobody wants that. But, if it works out, it's a transformative experience.
I would suggest working with your advisor on coming up with a research question that you can tackle in a limited time. Or if you have an idea, pitch it to your advisor and ask them what they think about it. It could even be a slight spin-off of what they are currently working on.
Time management is another key factor. Break down your project into manageable milestones and set realistic deadlines for each phase. I used the conference submission deadline for CHI. Having an external immovable deadline creates a sense of urgency, helps maintain focus and pushes you to stay on track.
An opportunity to contribute novel findings to a field you are passionate about and develop valuable research skills along the way is a truly rewarding experience, particularly when no one expects this of you because you are a master's student. Don't listen to naysayers. The greater the risks, the greater the reward. Go all in
Rajshekar Guda Subhash (MCompSci'24)
Caption: An illustration of the approach to address the Midas touch issue in gaze selections for immersive environments. When the user merely observes target 7, no Stimulus-Preceding Negativity (SPN) is evident from the occipital region’s electroencephalogram activity. When intending to select target 12 and expecting a selection feedback, an SPN is evoked, serving as an implicit selection trigger.