Affective Computing is the study and the development of systems to recognise human emotions, possibly from scalp brain signals or electroencephalogram (EEG). In such a study, while EEG is recorded, emotional responses of the subjects are induced by displaying audiovisual stimuli, usually on a computer monitor. Virtual Reality (VR) has been proposed as a more immersive medium to study emotional responses, diagnose or treat medical conditions such as social anxiety or post-traumatic stress disorder. In recent years, advances in computer graphics technology have enabled the display of immersive VR content even on consumer mobile phones, making this technology more accessible to everyday consumers. Incorporating feedback from the subject through subjective questionnaires and physiological signals such as EEG obtained during the VR experience could help improve the design of such VR content.
Hence, this paper presents an EEG-based Brain-Computer Interface (BCI) system that can record EEG and present audiovisual stimuli using a commercially available Android handphone placed in a VR headset. This system also works with commercially available dry EEG headbands and medical grade gel EEG headsets. The software architecture of EEG-based BCI VR system comprises the BCI and the VR components. The former component handles the data acquisition from various different EEG devices to interface with wireless Bluetooth dry EEG headbands such as MUSE, or to interface with wired gel-based full EEG headset such as Neuroscan NuAmps. The system can be deployed as single Unity3D application on Android mobile devices using Bluetooth-enabled EEG amplifier, or can be deployed in combination with a desktop computer connected to the wired EEG amplifier to send data via TCP/IP to the Unity3D application. The latter component presents the VR audiovisual stimuli, logs event and records subjective feedback during the experiment. It can display a variety of normal, VR180 or VR360 video clips to induce emotional responses in the subject.
This system facilitates experiments using existing EEG headsets and using immersive VR to induce emotional responses for affective computing studies. A prior study which examined the display of IAPS pictures on the computer monitor and performed classification of high/ low valence and arousal ratings yielded an inter-subject accuracy of about 60% on 16 subjects. Hence future plans for this system include planning data collection experiment to address questions such as whether it would be more immersive than a computer monitor in the prior study or what the degree of simulator sickness would be.