Auditory Separation of a Conversation from Background via Attentional Gating
Date:
Fri, 10/11/2019 - 10:30am - 12:00pm
Location:
CCRMA Seminar Room
Event Type:
Hearing Seminar 
The latest speech enhancement work has the potential to dramatically change the way we hear the world around us. This new work has dramatically improved the quality and latency of these algorithms, and it has the potential to change the way we hear the world around us, whether we have normal hearing or need assistance. These new systems build highly sophisticated models of speech, and can pick out the speech signal from the noise. Oh, yes. Lots of training data makes this possible, but that is easy to get by adding noise to clean speech signals.
Who: Shariq Mobin
We present a model for separating a set of voices out of a sound mixture containing an unknown number of sources. Our Attentional Gating Network (AGN) uses a variable attentional context to specify which speakers in the mixture are of interest. The attentional context is specified by an embedding vector which modifies the processing of a neural network through an additive bias. Individual speaker embeddings are learned to separate a single speaker while superpositions of the individual speaker embeddings are used to separate sets of speakers. We first evaluate AGN on a traditional single speaker separation task and show an improvement of 9% with respect to comparable models. Then, we introduce a new task to separate an arbitrary subset of voices from a mixture of an unknown-sized set of voices, inspired by the human ability to separate a conversation of interest from background chatter at a cafeteria. We show that AGN is the only model capable of solving this task, performing only 7% worse than on the single speaker separation task.
Speaker Bio:
Shariq Mobin is the founder of AudioFocus, a startup developing new hearing aid technology using deep learning. He recently obtained his PhD in auditory neuroscience at the Redwood Center for Theoretical Neuroscience under Professor Bruno Olshausen. His research focused on connecting human attention to deep learning models of sound.
What: Auditory Separation of a Conversation from Background via Attentional Gating
When: Fri, 10/11/2019 from 0:30am - 12:00pm
Where: CCRMA Seminar Room
Why: Speech is a seriously interesting signal and we want to make it clearer.
Where: CCRMA Seminar Room
Why: Speech is a seriously interesting signal and we want to make it clearer.
Bring your favorite speech perception system to CCRMA, and we’ll talk about tools to give it a better signal.
- Malcolm
Auditory Separation of a Conversation from Background via Attentional Gating
Shariq Mobin
Abstract: We present a model for separating a set of voices out of a sound mixture containing an unknown number of sources. Our Attentional Gating Network (AGN) uses a variable attentional context to specify which speakers in the mixture are of interest. The attentional context is specified by an embedding vector which modifies the processing of a neural network through an additive bias. Individual speaker embeddings are learned to separate a single speaker while superpositions of the individual speaker embeddings are used to separate sets of speakers. We first evaluate AGN on a traditional single speaker separation task and show an improvement of 9% with respect to comparable models. Then, we introduce a new task to separate an arbitrary subset of voices from a mixture of an unknown-sized set of voices, inspired by the human ability to separate a conversation of interest from background chatter at a cafeteria. We show that AGN is the only model capable of solving this task, performing only 7% worse than on the single speaker separation task.
Speaker Bio:
Shariq Mobin is the founder of AudioFocus, a startup developing new hearing aid technology using deep learning. He recently obtained his PhD in auditory neuroscience at the Redwood Center for Theoretical Neuroscience under Professor Bruno Olshausen. His research focused on connecting human attention to deep learning models of sound.
FREE
Open to the Public
