I have a large dataset with 10s of millions of points in a 10 dimensional parameter space. I have tried training my regression neural network on the entire parameter space and got decent (ish) results.
Fiddling with the number of hidden layers, or nodes per layer didn't really change things. However, I removed two of my input parameters from the learning, and the regression was much better.
The excluded parameters are perhaps the most important two in terms of model setup, and all logic (thinking about the problem I am trying to model) dictates that they should be present.
Could anyone please give some insight into this behavior?
My thoughts are that these parameters could shape some of the other 8, which, given the dominance of the two excluded variables, is masking the behavior of the others.