I have a large dataset with 10s of millions of points in a 10 dimensional parameter space. I have tried training my regression neural network on the entire parameter space and got decent (ish) results.

Fiddling with the number of hidden layers, or nodes per layer didn't really change things. However, I removed two of my input parameters from the learning, and the regression was much better.

The excluded parameters are perhaps the most important two in terms of model setup, and all logic (thinking about the problem I am trying to model) dictates that they should be present.

Could anyone please give some insight into this behavior?

My thoughts are that these parameters could shape some of the other 8, which, given the dominance of the two excluded variables, is masking the behavior of the others.

  • $\begingroup$ What happens if you use a simpler model type? $\endgroup$ – David Jul 4 at 11:01

The problem when you use neural network is that you use a black-box method that only gives you prediction so if without those variables you got better results, well get them out.

If you want to dig the explanatory dimension, you will have to use other modeling, basically econometrics one like a logistic regression or even a decision tree since you only have 10 variables, it will be readible.

But since you suspect what can be called a problem of multicolinearity, other variable correlated to your two variables really heavily and mislead their importance in the model. For that you have the VIF.

Sometimes you know even if in all logic it should be those variables the most important, it's not always the case.

I built a regression on sales predictive for a channel and the price inputs that SHOULD be the most important wasn't, we found that mostly it was the foot games events through the season (big games like champions league) that were the most important, the price wasn't.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.