Approaches

Equipment

Phase 1: Tone-Based

Sound is just a combination of different frequencies. If we can figure out what hearing 40 Hz, 80 Hz, 120 Hz, and so on, looks like on an EEG, we can work backwards. From the EEG, we can figure out the power component at each frequency and sum it all together to reconstruct the sound.

To approximate human speech, we planned the following steps.

1. White noise as carrier sound, modulated at a single, constant frequency

2. White noise as carrier sound, modulated at a single, changing frequency

3. White noise as carrier sound, modulated at a combination of two frequencies

4. Narrowed white noise around human speech frequencies, modulated at a single frequency

5. Narrowed white noise around human speech frequencies, modulated at two frequencies

Phase 2: Speech-Based

All spoken language is made up of phonemes, individual units of speech. If we can determine what phonemes were heard, we can guess the words and the entire sentence using large language models! English has around 44 phonemes, making this a classification problem with 44 classes.

As a starting point, we developed an algorithm to predict whether a subject heard "Yes" or "No." Besides their simplicity, these basic words are fundamental to any conversation.

Audio samples of "Yes" and "No" were recorded. These samples were randomly played to a subject, and the EEG response was analyzed with the following methods.

• Determining average response with event-related potentials

• Denoising with independent component analysis

• Visualization with principal component analysis

• Classification with

  • Logistic regression

  • Support vector machines

Credit: Jeffrey C. Liu