The project focuses on developing a deep learning model to accurately localize and separate sound sources in complex auditory environments. Using both spatial and temporal cues, the model aims to distinguish multiple sound sources by integrating audio and visual information. Advanced neural network architectures, including skip connections, are employed to ensure precise localization and separation. This system has potential applications in areas such as surveillance, multimedia, and robotics, where distinguishing individual sound sources is critical for performance enhancement.

The project centers on developing a diffusion-based deep learning model for generating audio from visual data. By learning from temporally and semantically aligned features of video frames with the corresponding Mel-Spec data, we generate a Mel-Spec through diffusion . The model captures audio characteristics that are related semantically and temporally. This approach has promising applications in fields like immersive media, virtual reality, and content creation, where audio generation for action can be automated.