1
$\begingroup$

I've $2000$ audio signals, each divided into a number of time intervals/time frames of $50$ miliseconds (ms) and these signals have overlaps for $25$ ms. Now, the audio signals being of different time lengths, these number of time frames/divisions depend on the signal, e.g. for the first signal, we've $20$ division/frames on which the signal was recorded, and for the second one we've $24$. For each such time frame for each signal, we've recorded $34$ features of the signal.

To give you a clarification, link to the documentation I'm using: https://github.com/tyiannak/pyAudioAnalysis/wiki/3.-Feature-Extraction

and the feature vectors for the subdivisions are extracted using the following code (in Python) in the quantity $F$:

from pyAudioAnalysis import audioBasicIO
from pyAudioAnalysis import audioFeatureExtraction
import matplotlib.pyplot as plt
[Fs, x] = audioBasicIO.readAudioFile("sample.wav");
F, f_names = audioFeatureExtraction.stFeatureExtraction(x, Fs, 0.050*Fs, 0.025*Fs);

So, by what I wrote before the code sample, $F$ is a (feature) matrix of dimension $34 \times N$ for each audio signal, where $N=$ number of time divisions we've for each signal, and $34$ is the number of features we've computed for each of the $N$ divisions.

I'm working on a classification problem for these signals as far what the audio says by using the above data. My problem is: how should I deal with the fact that the signals are not of the same lengths, i.e. column dimension of feature matrix depends on the signal? Possible solutions I'm thinking of:

(1) Computing statstical quantities: I guess one way to go about this would be to compute fixed, say $k$ no of statistical quantities representing the distribution of each feature vector. That is, for each of the $34$ feature vectors in $F$, i.e. for each of the $34$ rows in $F$, the mean, standard deviation, and may be some other distributional characteristics, like higher moments etc. that represents the distribution. And then we'll obtain a $34 \times N$ dimensional matrix for each signal. Then we can start to classify.

(2) Resampling: I've never used this before, but it appears there's a function in Python called scipy.signal.resample(x, num), which yields a (re-) sample from of dimension num, for any vector x of any length. But the documentation: https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.resample.html assumes the signal to be periodic. So we obviously face the question: is iur signal (appreximately) periodic? If not, can we still apply it?

Is there any other, and possibly better ideas?

$\endgroup$

1 Answer 1

1
$\begingroup$

One obvious solution is to use the time series of features (e.g. numbers by e.g. 64 frequency bands) as an input to a LSTM neural network (or similar), which should naturally deal with varying sequence lengths.

Alternatively, decide on a maximum length and pad with silence, if a clip is shorter.

Have a look at what Google did for Audioset with the VGGish model they published (https://github.com/tensorflow/models/blob/master/research/audioset/README.md).

$\endgroup$
1
  • $\begingroup$ Thank you-unfortunately I don't know LSTM apart from the fact that it's a deep learning architecture used to handle time series data (I've never used any DL architecture but did use standars ML algorithms). I'll check the links. If you've an LSTM code that I can use as a blakbox to classify these signals, I'd appreciate it. Thanks again! $\endgroup$
    – Mathmath
    Mar 21, 2019 at 17:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.