Sound localization using a deep graph signal-processing model for acoustic imaging
Date:
Wed, 08/23/2023 - 3:30pm - 4:30pm
Event Type:
Guest Lecture 
ABSTRACT:
Our research explores ways to leverage the architecture of DeepWave, originally used as an acoustic camera, to enable precise localization of sound sources. While DeepWave inherently generates spherical maps in the form of sound intensity fields, it has not been utilized for determining precise localization coordinates of sound sources.
We adopted two approaches to unfold DeepWave’s potential as a localization model. The first approach shows how DeepWave’s generated sound intensity fields can be processed for localization. We performed k-means clustering on DeepWave's output fields to identify centroids of relevant sound intensity areas. As a result of a fine-tuned k-means algorithm, DeepWave can compete with the baseline SELDNet model from the DCASE challenge using the LOCATA dataset. The second approach shows how DeepWave can also be integrated into larger networks for sound localization. We employ transfer learning using DeepWave as a feature extractor by taking its sound intensity fields as inputs to a Gated
Recurrent Unit (GRU) network. The results demonstrate how stacking DeepWave with a GRU network, like the one in the SELDNet model, outperforms the SELDNet model itself when trained and evaluated with LOCATA.
The proposed architecture that combines DeepWave with a GRU network presented in this project achieves robust localization accuracy compared to existing models. In conclusion, DeepWave can easily be used as a core building block in sound localization networks. The full implementation of DeepWave written in PyTorch used on this research can be found atRecurrent Unit (GRU) network. The results demonstrate how stacking DeepWave with a GRU network, like the one in the SELDNet model, outperforms the SELDNet model itself when trained and evaluated with LOCATA.
https://github.com/adrianSRoman/DeepWaveTorch.git.
NOTE: this event is part of the DL4MIR workshop series (ccrma-mir.github.io); guest speaker talks are open to the broader CCRMA community.
FREE
Open to the Public
