Artificial Neural Network and PCA I have to make a forecast about one variable and I use some different methods to do so. I would try a nonlinear alternative also and I would to consider the Artificial Neural Network (ANN) models.
I have many predictors and I know that about linear models make no selection among them is not a good idea, because in this case become highly probable to encounter in overfitting. I don't know if with ANN the same is true, maybe overfitting occur primarily if we use too many hidden layers and/or nodes.
However I would to use some reduction/selection rule also. Now, I know that PCA is one possible technique for data reduction. Actually I have more that 130 predictors but with less than 30 component (after standardization) I capture more than 70% of total variability. I can use these components as predictors but my point is the follow.
I know that ANN is more effective if there are highly nonlinear links between predictors and predicted variable. Now, the PC are a linear combinations of predictors and then I fear that components that contain the most part of variability tend to obscure the nonlinear relations. If this is true seems me that PCA is not a great idea for reduce the number of predictors in ANN model. This is true?
If it is, what method is preferable as variables selection and or reduction for ANN? To use all predictors that I have is better?     
 A: Using PCA to project the data to a lower dimension can yield worse results. This is as true for neural networks as it is for any other model. This is because the projection to a lower dimension is not aware of the outcome. All it does is retain the PCs with the largest variance; if the informative features PCs with lowest variance, then PCA will make the model worse because you're excluding the signal. 


*

*Examples of PCA where PCs with low variance are "useful"

*Do PCA affect different classification methods?

*Is PCA always recommended?
On the other hand, if you only use PCA to rotate the data but not project it to lower dimension, PCA can be useful because it improves the optimization dynamics.
In "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift", Sergey Ioffe and Christian Szegedy suggest that whitening transformation are helpful during the optimization steps.

It has been long known (LeCun et al., 1998b; Wiesler & Ney, 2011) that the network training converges faster if its inputs are whitened – i.e., linearly transformed to have zero means and unit variances, and de-correlated.

How to select features for inclusion depends on the domain. The best feature selection is to use domain knowledge to eliminate features which are not related to the outcome. For tabular datasets where domain knowledge is unavailable, generic feature selection methods like boruta can be helpful, even though they're not foolproof. There's always some risk that you'll leave out important features, or include irrelevant features. Also, the results obtained from Boruta will depend on the model you use to measure feature importance, which introduces additional knobs to turn.
See also: Are dimensionality reduction techniques useful in deep learning
