My question is with respect to real-world custom data sets used for Machine Learning applications. The collected data set contains numerous audio signals (File1, File2, File3, etc) of a moving machine part captured over different points of time. Features are extracted using basic signal processing techniques such as FFT, signal envelope, etc. The machine is located in the real world and hence the audio signals have noise. Is it possible that the feature set for each data point in the audio signal has different distribution? If so, does this affect the ability of a Random Forest or Neural Network or any other ML model to generate a good fit? Also, is it possible that the audio signals themselves do not share a common distribution due to noise and other environmental factors?
1 Answer
Real world audio signals are a mixture of multiple signals. Some of these can be noise and some the sound of interest. Which degree the noise affects the distribution of features really depends a lot on the feature extraction. It is of course desirable to have features which are as independent of noise as possible. If you hand engineer features, this is something you manually have to account for. With deep learning it is possible to learn the feature extraction (representation learning). Techniques such as Data Augmentation with noisy signals can help the model learn noise-invariant features.
If no consideration is put into these things, the model is likely to perform badly on out of distribution data. If this can happen in a real world scenario, then it is important to evaluate the model on such cases. This is often referred to as mismatched conditions.
