Does Neural Networks based classification need a dimension reduction?

Question

I am using a Neural Networks based classifier to run a classification for my data in n-dimensional.

Then I thought it may be a good idea to run dimension reduction like PCA for my data at first, and then put the PCA results into the classifier (I keep 3 PCs). However, the classification on the dimension reduced features are not as good as using the original high-dimensional features directly.

Then I came across this post NN as a DR1 that discussed Neural Networks as a dimension reduction method. Also some information can be found in this paper NN as a DR2 I am confusing now:

1. If I use Neural Networks based classification (in Matlab), does it automatically do the dimension reduction for me?
2. Should I run dimension reduction like PCA before running Neural Networks classification?
3. Are there any other reasons the classification on PCA results is not as good as using the original high dimensional features?

Answer 1

In principle, the linear transformation performed by PCA can be performed just as well by the input layer weights of the neural network, so it isn't strictly speaking necessary. However, as the number of weights in the network increases, the amount of data needed to be able to reliably determine the weights of the network also increases (often quite rapidly), and over-fitting becomes more of an issue (so using regularisation is also a good idea). The benefit of dimensionality reduction is that it reduces the size of the network, and hence the amount of data needed to train it. The disadvantage of using PCA is that the discriminative information that distinguishes one class from another might be in the low variance components, so using PCA can make performance worse.

Like most things in statistical pattern recognition, there is no single recipe that works reliably for all problems, and really the best thing to do is to try both approaches and see which works best.

Answer 2

If I use Neural Networks based classification (in Matlab), does it automatically do the dimension reduction for me? Answer: if you are using more than one layer for classification and the number of neuron in hidden layer are less than input neurons so you have a kind of dimensional reduction neural network. but it might not be a PCA.

Answer 3

Because the neural network can approximate any nonlinear mapping through learning and is free from the constraints of a non-linear model, there was no need to reduct dimension like PCA in advance. This also shows artificial neural network (ANN) has strong non-linear computing power.