Is it better to have one neural network per feature, or a single neural network for everything?

Question

I'm working on a simple audio-classification program, which is intended to indicate whether the most-recent n seconds of audio in an audio stream are "Speech" or "Music".

My program extracts various features from the audio (fundamental frequency, loudness, rolloff frequency, etc) and uses them to build up histograms that I can then feed to one or more neural networks (using the FANN API) as input data. Each neural network has a single output node whose value indicates whether it thinks its input data represents speech or music.

My question is: am I likely to get better results by feeding all of my different types of feature-data into One Really Big Neural Network, or am I better off training up a smaller, separate neural network for each feature/data type, and then combining the results of all of the small neural networks' output together (e.g. by addition) afterwards?

Is one approach typically more effective than the other, or is this more a case of "It Depends, You'll Just Have To Try It Both Ways And See"?

Answer 1

I think this would require some expert domain knowledge to decide which approach is better, although ultimately it is better to try both ways and see.

If you don't expect the interaction between the features to provide new information, then you should build separate networks and combine. However, if you do expect the interaction to matter, then you should build one network.

For example, suppose as an image classifier, I can break apart the image into the left half, and the right half, which are my two "features". If there is a dog in the middle of my image, which is split across the two halves, it would be much better to have a single network which can look at both the left half and right half of the image to determine whether it is a dog or not. It's much harder to succeed we had two networks, each which could only see half of the dog.

However, if the image was split up into three channels, R, G, and B, and those channels were used as features, then it's more acceptable to train three classifiers and combine them, since the combination of two colors is less likely to add more information about whether there is a dog in an image.