0
$\begingroup$

I need to classify whether a product sound "good" and "bad" based on FFT of its audio recording. The FFT magnitudes are show for frequencies from 0 to 7khz, with a frequency resolution of 5 hz, so there are lots of bins.

The problem is there are many ways a product can sound "good", and many ways a product can sound "bad". It's hard to come up with an explicit rule. Moreover, the recordings are done at a noisy factory environment where ambient noise can be random, although it only shows significant magnitudes at <1500 hz. But sometimes some machine sound can cause a peak at higher frequencies.

Therefore, I don't think a regression or random forest based on FFT magnitudes would work well. KNN seems ideal for this because as long as I have training data recorded for all different kinds of "good" and "bad" products, under various ambient conditions, it seems this problem is just a matter of finding the nearest neighbors, with the distance defined as Euclidean distance in a high dimension space (each frequency bin is a dimension).

I just feel KNN is perfectly suited for this because I don't need to calibrate the equipment or deal with decibels with some reference. But if "good" and "bad" products tend to have certain profiles (different peaks at different frequencies), then even with some ambient noise which can alter the magnitudes to an extent, KNN should still be more robust than a parametric model.

Is my understanding of the merit of KNN over parametric model valid in this scenario? Should I try to reduce the dimensions prior to KNN?

Thanks

$\endgroup$
  • $\begingroup$ Are you classifying based on a single FFT window? This would ignore all temporal effects in the audio, likely ignoring a lot of possible distinctions between perceptually good and bad. $\endgroup$ – jonnor Aug 1 at 19:33
0
$\begingroup$

Clustering might be a good approach (e.g. KNN, but e.g. UMAP might also be an option), but only if your theory that all things sound "good" if they have peaks at the same frequencies as other good sounding things (and "bad" if they in this sense resemble other "bad" sounding things). However, this does not exclude the possibility that you could feed these features into some other kind of model (e.g. some kind of time series model, e.g. a recurrent neural network - whether that's feasible really depends on how much data you have and how much is labelled).

Obvious parameter space reduction techniques could be to use the final embedding layers of some pre-trained neural network as your features (e.g. VGGish pre-trained on AudioSet), a lot of features constructed by human insights (e.g. certain extremely high frequency sounds might be unpleasant and you might be able to define features in terms of certain amplitudes at certain frequencies or certain peak to trough ratios that pick this up pretty well when used as a feature in a model) or a mixture of these.

$\endgroup$
0
$\begingroup$

FFT is a pretty low level feature representation, pretty far from the complex things people perceive when judging whether a sound is 'good' or not.

I would instead recommend using an audio embedding to compute the features. This is usually done by a CNN operating on a spectrogram (computed via short-time FFT). Pretrained models are available, such as VGGish and OpenL3.

On top of audio embedding KNN is definitely worth a try, along with a simple linear classifier (Logistic Regression).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.