AI-Enabled Smart Interfaces
By integrating wearable technologies with artificial intelligence (AI), we develop human-centric smart systems that support everyday life, solve challenges faced by people with disabilities, and unlock new ways for humans to sense, interact with, and experience the world.
Hand Navigation System for the Visually Impaired
For people with visual impairments, we assist hand navigation toward target objects using a wearable haptic interface that delivers directional force cues to the back of the hand.
Linked Smart System
Haptic Interface (back of the hand) - delivers directional force cues for hand navigation. (arrow in the right window is the directional cue)
Smart Glasses - An embedded camera (right window in the movie) tracks the hand and target object.
Smartphone with AI - Recognizes the target object (AI), calculate the relative position between the hand and targets, and send navigation commands to haptic interface in real time.
Movie_hand navigation.mp4Feeling Music Through the Skin
For people with hearing impairments, we provide an alternative way to experience music through skin vibrations. Because sound consists of mechanical vibrations, musical information can be translated into haptic stimuli delivered to the skin.
Linked Smart System
Smartphone - Captures and processes music sound signals, converting sound into haptic patterns.
Haptic interface - Generates distinct vibration patterns corresponding to different musical elements, such as vocals, guitar, and drums, at a single skin location.
AI - Separates complex audio mixtures into individual sound components (vocal, guitar, drum and so on), enabling independent analysis and haptic rendering of each instrument.
Movie_Feeling music.mp4Robotic Avatar (Synchronized Sensory Systems w/ Robot)
Humans can remotely control robots through synchronized sensory systems, including vision, hearing, and touch. While technologies for visual and auditory communication are already well established through cameras-displays and microphones-speakers, interfaces for touch remain underdeveloped and cannot yet deliver natural and realistic tactile sensations. Our goal is to achieve seamless tactile synchronization between humans and robots, enabling more dexterous manipulation and immersive teleoperation.
Synced sensory systems in the video (Robot part) ⟷ (human part)
Vision: Head-mounted camera ⟷ VR headset display
Auditory: Microphone ⟷ Speaker
Tactile: Fingertip force sensors (green part) ⟷ Actuators in haptic glove
Movie_Robotic avatar.mp4