This might be noob question, because I've just started to learn data analytics.
Why should we or should we not include a correlated predictor in our model?
While selecting predictors, for a class project, I observed that 4 variables exhibited more than 90% correlation with each other. I was using logistic regression for my model. And I only included one variable which gave me the lowest misclassification error during the kfold cross validation.
However, my classmates who included one other correlated variables as well the other predictors which I used, performed better on the Test Data.
On reflection, I reasoned that maybe the response could be function of both the correlated pairs ( something like y = ax + bx^2 + c).
I am still not wholly convinced of it. Could someone shed more light on this?