Thinking Penguin: Multimodal Brain-Computer Interface Control of a VR Game

Robert Leeb, Marcel Lancelle, Vera Kaiser, Wolf-Dietrich Fellner, Gert Pfurtscheller

Research output: Contribution to journalArticlepeer-review

Abstract

In this paper, we describe a multimodal brain-computer interface (BCI) experiment, situated in a highly immersive CAVE. A subject sitting in the virtual environment controls the main character of a virtual reality game: a penguin that slides down a snowy mountain slope. While the subject can trigger a jump action via the BCI, additional steering with a game controller as a secondary task was tested. Our experiment profits from the game as an attractive task where the subject is motivated to get a higher score with a better BCI performance. A BCI based on the so-called brain switch was applied, which allows discrete asynchronous actions. Fourteen subjects participated, of which 50% achieved the required performance to test the penguin game. Comparing the BCI performance during the training and the game showed that a transfer of skills is possible, in spite of the changes in visual complexity and task demand. Finally and most importantly, our results showed that the use of a secondary motor task, in our case the joystick control, did not deteriorate the BCI performance during the game. Through these findings, we conclude that our chosen approach is a suitable multimodal or hybrid BCI implementation, in which the user can even perform other tasks in parallel.
Original languageEnglish
Pages (from-to)117-128
JournalIEEE Transactions on Computational Intelligence and AI in Games
Volume5
Issue number2
DOIs
Publication statusPublished - 2013

Fields of Expertise

  • Information, Communication & Computing

Treatment code (Nähere Zuordnung)

  • Application

Fingerprint

Dive into the research topics of 'Thinking Penguin: Multimodal Brain-Computer Interface Control of a VR Game'. Together they form a unique fingerprint.

Cite this