Neural Network for sound classification I am currently trying to automate some identification process of characteristic noise sounds. For acoustic feature, I calculate MFCC. I have downloaded a free MATLAB toolbox from Dan Ellis'es website. As a classifier, I am trying Neural Network patternnet with Softmax transfer function from Neural Network Toolbox in MATLAB. I have built a training file which consists of ~ 200 sounds samples and ~ 200 additional sounds of "anything else" to give NN a sort of counter balance. I extract 13 MFCC coefficients, 13 delta coefficients calculated from MFCC, and 13 delta2 coefficients calculated from delta1. Also, I feed 5 consecutive frames in sequence at once to NN. So all together, I have like 13*3*5 = 195 inputs. I have also 10 outputs, 5 per frame, and 2 per each class.
When I analyse NN's performance and its gradient progress, I am not quite sure if NN is trained correctly. It looks OK at first as the gradient gets low but it doesn't reach its threshold $10^{-6}$. NN simply stops due to validation. When I test the model on another test file, NN seems to pick also other sounds. It's like its training is not done correctly. I did try different configurations such as 195, 390, 780 neurons in one, two, and three layer but it doesn't seem improve the performance.
Perhaps someone could give an advice on how to improve the classification.
Thanks

Below I have provided some screenshots from NN performances.
NN Gradient. 195 inputs, 195 neurons in a hidden layer, 10 outputs

NN Performance. 195 inputs, 195 neurons in a hidden layer, 10 outputs

NN Gradient. 195 inputs, 3 hidden layers, 780 neurons in each hidden layer, 10 outputs

NN Performance. 195 inputs, 3 hidden layers, 780 neurons in each hidden layer, 10 outputs

 A: 200 samples is likely too little for such a complicated model. Ideally your dataset should be at least 10x bigger. However, here are some things you can try:
Simplify the model
For example try using just (min,max) or (mean,std) summary across your MFCC frames: 2x13=26 features.
Normalize the sounds
Common for spectrogram data is to normalize each frame by subtracting the median or mean, and divide by RMS energy, standard deviation or max.
Use (log) melspectrogram
It can give better performance over MFCC with expressive classifiers like Neural Networks. Simple linear models often struggle with the co-
Use a less data-hungry classifier
Like Random Forests or kNN.
Data augmentation
Expand your data artificially using data augmentation. Common augmentations for audio data include time stretching, frequency response changes, adding white noise.
Unsupervised feature learning
Use unsupervised learning too learn more powerful features. A simple mechanism is spherical k-means on randomly selected spectrogram patches (3x3) to learn convolution kernels.
A: If it stops early due to validation errors instead of your gradient threshold it is because your network starts to overfit underfits.
The absolute best way to improve the situation is to acquire more data. You say to have 2 x 200 sound samples, which doesn't sound like a lot (although that obviously depends on the type of samples).
