🎧ABSTRACT

Auditory scene analysis (ASA) models extract information about individual sound sources from a complex auditory mixture. Recent ASA models process the mixture through a sequence of modules and split it into source-wise features (i.e., object features) only at the last layer. We observe a large performance gap across sources in the last layer, which degrades overall performance. A centered kernel alignment (CKA)-based analysis shows that the CKA value reliably proxies performance across sources and that the CKA gap between sources is smallest at the middle layers and widens sharply at the last layer. We conjecture that mixture splitting occurring before the CKA gap becomes large can resolve the issue. To this end, we propose PARASITE, a plug-in refinement architecture that splits object features at earlier layers and sequentially refines them through additional branches. Each additional branch combines object features separated from different layers through Object-wise Gated Fusion (OGF) and refines them through Windowed Cross-Object Attention (WCOA). We introduce the DeepASA++ model, which combines PARASITE with an ASA backbone and delivers state-of-the-art results on both ASA2 and STARSS23. DeepASA++ achieves a SELD score of 0.240 on STARSS23 and surpasses prior multi-stage inference architectures, including chain-of-inference.

Demos

Empty. It will be updated soon.

Architecture

Experimental Results

Ablation Studies