I am observing an increase in the predictive power of my logistic regression when I remove certain predictors. It might be a bug in my code, so I'm wondering: is this statistically possible? If so, what are the possible explanations?
- there is strong multicollinearity in my predictors
- I am not interested in interpreting my parameter values, I just want high predictive power - I have 15 regressors and a dataset size of 100k (10k for validation to avoid overfitting, 10k for testing)
- I am using a multi-layer perceptron (neural network)
- I measure predictive power by % of test cases correctly classified