Sensory Substitution Research Lead

Continuous vibrotactile pulse-train distance encoding for spatial boundary awareness and understanding in VR environments.

Aug 2025 - Aug 2026 (300+ hours)

The goal:

When starting this project I had several goals

Have demonstrable interest in sensory substitution research
- Specifically in creating an interface which becomes intuitive and subconscious to use over time
Learn how to actually do research, instead of just having profs give me a task
Use the project to get accepted into MIT Reality Hack (we ended up getting 2nd place here)

What I learned:

Starting from a blank slate in a new research domain is really hard. It's much easier if you or a mentor already have experience in the area,
Engineering and research are two very different things. I spent probably half of my time on this project developing the belt, VR environment, and integrating the two
Project management is hard. Managing and aligning the expectations of yourself, teammates, and mentors, who all have different levels of investment feels impossible. Delegating tasks requires a lot of trust and clear communication.

Project Overview:

Using a custom built vibrotactile belt, we communicated the distances of surrounding walls to the user. This research compares the differences between encoding the distance of those walls via pulse frequency and pulse duty cycle, with the goals of increasing the user's spatial awareness of their boundaries.

Applications:

Increased quality of life for blind / visually impaired individuals.
- They no longer just have a simple walking cane, telling them if something is right in front of them or not, but can now feel the layout of the room around them.
Safety at construction sites
- Someone who must be focused on a specific task (e.g. a welder) can now feel motion going on all around them, and detect incoming threats
Virtual environment safety
- While using VR, you can feel where real world boundaries are around you, preventing running into walls, furniture, etc.

The Research:

Full research paper coming soon!

The Vibrotactile Encoding:

To compare participants awareness of their surroundings, we encoded the distance to boundaries for each of the 5 vibrotactile motors using the following methods.

--------------------------------------------------------------------------

Temporal Frequency Modulation

As a sensor gets closer to a boundary, we exponentially increase the frequency at which the tactor activates, holding duty cycle fixed.

Temporal Duty Cycle Modulation

As a sensor gets closer to a boundary, we exponentially increase the duty cycle of the tactors pulse, holding the frequency constant

--------------------------------------------------------------------------

In both cases, we also exponentially increase the amplitude of the tactor.

The Haptic Belt:

ESP32 receives UDP commands from VR environment (representing distances of each motor to obstacle)
ESP32 provides high level commands to motor drivers to modulate pulse characteristics
Motor drivers control pulse shape of ERM motors, delivering haptic feedback around the front of the users waist

The VR Environment:

Research assistant sets up device with participant ID & condition ensuring all data is logged properly
User experience can be seen in video
Additional safety measures are implemented to ensure users do not collide with real walls

If user approaches real wall, headset will fade to black with a red arrow showing user which way to go to re-enter study space

Challenges:

See the main things I learned at the top of the page!
I actually learned what research was
- Basically, it's just anything you want it to be. People publish papers that have a vast variety of goals, but they are all labeled as "research".
- I also learned that a lot of people make a lot of questionable justifications for decisions in their work, leading me to more questions
  1. Is research really that scientific, or is it just people making decisions based on heuristics, and then finding work post-hoc to support those decisions? (obviously their heuristics have to come from somewhere, and maye this is what is rooted in the research?)
  2. How do you tell a good research paper from a bad one? I can totally BS something, and it looks just as professional as something that someone took 100x longer than me on.
- Good research takes a LOT of time, and I still don't have good metrics to tell good, high quality research, from low quality
Research just takes a long time, and I wasn't fully prepared to deal with this.
- All of my timelines got at least doubled from how long I originally thought things would took. I thought the research paper would only take two weeks. And my research mentor had told me that we could do the stats in one day :(

Page updated

Google Sites

Report abuse