Virtual Reality That Reacts to Your Thoughts

As it turns out, The Force isn't just a figment of George Lucas' imagination. Students in the UC Berkeley chapter of ACM SIGGRAPH have come up with a fascinating virtual reality system that reads users' brain waves and changes the world around them in response. The project, called Mindscape VR, utilizes the Oculus Rift and the Muse brain-sensing headset to create an immersive VR environment where users can move objects with their thoughts and interact with their surroundings by simply thinking about doing so.

Developed by Juan de Joya, Victor Leung and Kelly Peng of the Berkeley ACM SIGGRAPH student chapter, Mindscape VR looks at ways to use virtual reality as part of a brain-computer interface, and explores alternative methods for interaction with virtual spaces.

Juan de Joya explains the group's interest in new methods of interaction:

"Right now we're using existing input devices such as mice, keyboards, or gamepad controllers to interface with VR environments. However, there is cognitive disparity between mapping user actions to keys/buttons and how individuals physically act, let alone think, in spatial terms. Unless you're really familiar with the controls, it's harder to bridge that gap when you have a head-mounted display on your head obscuring your vision.

We used the Muse headset because it's an existing device that doesn't require a lot of training to use and, as an EEG device, it is the easiest and least invasive type of interface – you just put it on your head, and make sure that the sensors are picking up brain wave frequencies. In our first iteration of the project, we used one type of brain wave frequency to allow the user to levitate and collect pebbles in a simple fantasy world. While we disabled them at launch, we do have features where the user can call a dragon to appear, change night to day, summon fireflies and shoot fireballs depending on what type of brain frequencies the Muse is picking up.

We found that the simplicity of using one's thoughts to do things is a pretty gratifying and empowering experience. We had a kid try it out at launch, and as he started levitating the rocks he brought up his hands as if he were a Jedi. How cool is that? It's these kinds of seamless, easy-to-use experiences that underlie the potential of immersive VR as a medium."

The first iteration of Mindscape VR is available to experience as part of the Cognitive Technology exhibit at the San Francisco Exploratorium, which will be open during select dates through the month of February. The project continues to evolve as the team works to enhance the extrapolation and visualization of Muse-collected brain wave data.