I was doing CS109 lab. There I saw this written:-
"By the way, there is a problem with pre-doing feature selection before doing cross-validation. Ideally one should be doing the feature selection separately in each fold. The reasons for this is basically that there is a high probability that a feature correlates strongly with y just by chance, if there are so many features. How to do this properly will become clear in the homework."
Can, someone explain me, what will happen if features are correlated to y?