ASR System for the Hearing Impaired

Developed a retraining-free automatic speech recognition technology using user-defined dictionaries to support communication for people with hearing impairments. This approach enables accurate recognition of words that conventional systems often struggle with, such as personal names and technical terms. 

Related publications: 

• “Contextualized Automatic Speech Recognition with Dynamic Vocabulary”, in Proc. SLT, 2024. (🏆IEEE SLT Best Paper Award🏆)

• "DYNAC: Dynamic Vocabulary based Non-Autoregressive Contextualization for Speech Recognition", in Proc. INTERSPEECH, 2025. 

Integration of Sound Source Localization, Separation, and Classification

We proposed a novel approach called Environmental Sound Segmentation, which integrates sound source localization, separation, and classification into a unified framework. This method prevents the accumulation of errors commonly found in step-by-step processing, enabling highly accurate environmental sound recognition. The system handles 75 classes of everyday sounds, including coughing, glass breaking, and phone ringing.

Related publications: 

• "Multichannel Environmental Sound Segmentation with Separately Trained Spectral and Spatial Features", Applied Intelligence, 2021.

• "Sound event aware environmental sound segmentation with Mask U-Net", Advanced Robotics, 2020. 

Sound Event Localization and Detection Using Microphone Array for Automotive Applications

We developed a system for detecting and localizing abnormal sounds occurring during vehicle operation using a microphone array and machine learning techniques. Traditionally, inspection relied on subjective auditory evaluations by inspectors, resulting in inconsistent accuracy. Our method enables stable and objective automatic inspection, reducing variability and improving reliability. 

Patents: 

• Abnormal sound determination apparatus and determination method, Patent US10607632B2

• Abnormal sound detection apparatus and detection method, Patent US10475469B2

Sound Inspection Device in Noisy Factory Environments

Developed a high-precision inspection algorithm for operational sounds in noisy factory environments. This technology focuses on the difference in peak frequency stability between operational sounds and ambient noise, enabling accurate detection of sounds such as meter beeps even in environments where human auditory inspection is difficult due to high background noise. 

Patent: 

• Japanese Patent: Sound Inspection Method, JP6033718, filed on March 22, 2013, granted on November 4, 2016.