0
$\begingroup$

I would like to start my feature engineering process by first selecting a subset of features that are highly correlated with the target feature.

However, if I do select let’s say the top k in terms of correlation, won’t I be implicitly selecting k features that are also highly correlated among one another?

If yes, isn’t this kind of a bad thing because in a way I am just picking the same vectors in space? Wouldn’t it be a good idea to also make sure that among these k features I am picking the ones that are less correlated with one another ?

$\endgroup$
1
$\begingroup$

Correlation between x and y and z and y does not imply correlation between x and z. For example if x and z are iid uniform (0,1) random variables and y = x + z you will find that x and z have correlation 0 but the correlation between y and x (and similarly y and z) will be about 0.7.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.