First Experiences with Smarting

The Cybathlon challenge is approaching faster than we expected! Luckily, back in spring, when we were looking for suitable EEG systems and talked to different manufacturers, one of them, the start-up mBrainTrain from Belgrade in Serbia, decided to support us by arranging a portable EEG device we could use.

Having worked with consumer EEG systems before, we knew that we needed a research-grade system with a good signal quality. While we did not necessarily need a mobile system, i.e. a system to which we could connect via Bluetooth instead of through a bunch of cables, we thought that it would be useful for being more flexible. A mobile EEG system, such as mBrainTrain's Smarting, would make it much easier for us to go to our pilot’s home for recording as we wouldn’t need to go through the hassle of unplugging and re-plugging a lot of cables before and after every recording session. A dry electrode system would have made the out-of-the-lab recording even more convenient, but we feared that it would come with too much of a drawback in terms of signal quality. Besides, the Smarting device from mBrainTrain had been recommended to us by one of our professors.

Setting up the system was easier than expected. After plugging in the connector from all the electrodes into the Smarting device and connecting it via Bluetooth with the computer, one can use a convenient GUI, provided by mBrainTrain, in order to visualize the impedances of the electrodes. As shown in the picture below, this real-time feedback of the impedance values indicates which electrodes are fine and which need more gel.

How to correctly apply the gel between the scalp and the electrodes wasn’t so easy at first, but after a while, we became pretty good at it. As is common practice, we used the wooden end of cotton swabs to push aside the hair underneath the electrodes and then pressed the gel out of the syringe in a circular movement right underneath the electrodes. The cotton end of the swabs was helpful to further spread the gel to the right position.

Once the impedances are sufficiently low, the streaming over LSL can start. Even though Bluetooth is notorious for not being reliable enough whenever there are obstacles between the sender and the receiver, we managed to keep the percentage of lost packages (i.e. blocks of EEG samples) pretty low for most of the time. However, from time to time we had problems with starting the Bluetooth connection. A few of them came from not being careful with the exact sequence of steps one has to do in order to set the connection up. For connecting with a Windows-based computer, there is a Bluetooth dongle and software from BlueSoleil provided. For us it was important to disable the Windows firewall to be able to connect.

As a tool for processing the incoming EEG data stream, we chose to use NeuroPype, a platform for real-time brain-computer interfacing and signal processing. One of the reasons for this was that the software suite includes an open-source graphical pipeline designer (coded in Python), which is quite intuitive to use. A great surprise for us was how easy it is to connect the Smarting signal acquisition software to NeuroPype. Essentially, we only had to change the name of the labstreaming layer (LSL) node to the one predefined by Smarting and then we already were able to receive the signals in real-time.

For showing that our whole pipeline works, we designed a quick experiment based on the steady-state visual evoked potentials (SSVEPs), where one of our team members was very patient, suffering through looking at a flickering checkerboard pattern over and over again (let’s hope he wasn’t dreaming about it afterwards!). You can see the results in the following video (read below for detailed explanations):

The experiment consisted of a subject focusing on a flickering stimulus, whose frequency was changed between 10 Hz and 15 Hz by another person, depending on the desired kart direction. It is known that when a subject is focusing on a simple visual stimulus of a certain constant frequency (e.g. a flickering checkerboard), the power spectrum of the occipital EEG at that frequency and its higher harmonics have a significantly higher value than in the no-stimulus case. In other words, if you look at a stimulus of frequency 10 Hz, the neurons in your visual cortex start synchronously spiking in the same frequency. This interesting discovery was first described in [1] and in most cases does not depend on the volitional state of the subject (the subject only needs to passively focus on the stimulus), which means that it is not and active/intentional BCI paradigm.

Based on this central idea of the SSVEP paradigm, we designed a model in NeuroPype which computes the power of the 10 Hz and the 15 Hz frequencies and compares their normalized values (we only use the signals from the O1 and O2 electrodes for this). If the normalized 10 Hz power is significantly higher than the normalized 15 Hz power, we conclude that the subject is looking at the 10 Hz visual stimulus and send the "turn left" command to the game (a slightly adapted version of the Unity Kart racing tutorial). If the 15 Hz power is significantly higher, then we send "turn right" and if there is no significant difference, then the vehicle just drives straight ahead.

As any other paradigm, SSVEP has its limitations. One of them relevant for us was that, since we needed 2 control signals, there had to be 2 distinguishable power bands, and that means that the stimulus frequencies could not have been integer multiples of one another. Secondly, higher frequency stimuli were harder to detect, because the signal's power itself was decreasing with frequency. Most importantly, however, SSVEP is not intentional, which is required for the Cybathlon BCI competition and which we believe is hugely important for giving BCI-users a sense of agency. For testing our pipeline from signal recording with Smarting, to signal processing with NeuroPype and lastly control in a Unity game environment, SSVEP was a great first prototype paradigm, but the real challenge will come next, when we try to replace it by motor imagery control.

References:

[1] D. Regan, Human Brain Electrophysiology: Evoked Potentials and Evoked Magnetic Fields in Science and Medicine, Elsevier, New York, NY, USA, 1989.

Aleksandar Levic and Nicolas Berberich

Photos: Nicolas Berberich