Streaming audio to neural network

Question

I am trying to create a neural network that performs speaker recognition. I would like to be able to serve it such that it takes streaming audio - i.e. I want to perform partial recognition on 100ms frames and then calculate an average at the end.

I would like to know which of the following two forseeable options is the best.

• Training the network on audio clips of 100ms
• Using audio clips of arbitrary lengths and feeding subsequent 100ms segments into some sort of recurrent network. I was thinking that similar to text analysis, maintaining some state information could be useful in real time speaker identification.

Does anyone have some guidance in this regard? Thanks.

Answer 1

You can get some ideas from Energy-Efficient Speaker Identification with Low-Precision Networks by Skanda Koppula, James Glass, Anantha P. Chandrakasan.

They propose Depth-Separable Convolution for efficient stream-like processing. That gonna work, you process each 100ms with lower layers and then you process the whole bigger chunk with upper layer. Note that reliable speaker identification requires 2s for text-dependent setting and up to 5-10s for text-independent setting.