0
$\begingroup$

I have an n->n seq2seq LSTM that takes a sequence of length n and produces a sequence of length n. The sequences I am dealing with are audio signals. I am trying to use an LSTM to "de-noise" the audio.

So in a perfect world, for 1 second of audio at a sampling rate of 16000Hz, I would have an input sequence where n=16000, and then the LSTM would output the "de-noised" audio sequence of length n=16000. However, this takes far too long to train. Also, what if the audio is longer than 1 second?

So, it seems like the best thing to do would be train an LSTM on small chunks of 1 second of audio, say n=800. However, it is pointless if the LSTM de-noises the first 50ms, then the next 50ms etc. independently of each other. I could perhaps have some sliding window that extracts chunks that overlap with the previous chunk. But I am not sure how I would aggregate the outputs into a final 1 second audio clip...

How can I work around this long sequence problem? If I do break it down into subsequences, how do I tell the LSTM to retain information about the previous subsequence, or how do I aggregate the outputs of a sliding window?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Tools like segan split audio on 1 second chunks without overlap, denoise each separately and concatenate them together.

RNNNoise does overlap and add of 20ms windows with 50% overlap.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Interesting. Would you mind please elaborating a bit on what RNNNoise does? I had a read, but still don't quite understand. $\endgroup$
    – Harry Stuart
    Commented Apr 6, 2020 at 4:41
  • $\begingroup$ Same as other packages, it cleans the noise based on very efficient neural networks. It applies some tricks in the middle - pitch estimation, etc. Not very fashionable but very compute efficient. $\endgroup$
    – Nikolay Shmyrev
    Commented Apr 6, 2020 at 10:09
  • $\begingroup$ Overall, the longer is the audio chunk, the more reliably you can detect the noise. So for best denoiseing you need longer chunks. $\endgroup$
    – Nikolay Shmyrev
    Commented Apr 6, 2020 at 10:10
  • $\begingroup$ Yep, I understand that. I just don’t understand how the RNNNoise aggregates outputs if it has overlapping windows. $\endgroup$
    – Harry Stuart
    Commented Apr 6, 2020 at 10:31
  • $\begingroup$ What is the problem? You just overlap chunks and add them. It doesn't hurt much as you might think. $\endgroup$
    – Nikolay Shmyrev
    Commented Apr 6, 2020 at 10:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.