In machine learning, we perform feature engineering and selections in pipelines and crossvalidate to obtain results in order to avoid data leakage and avoid introducing prior knowledge into the learner.
Now, in a project I’m working on, I am starting to do some correlation analysis to create a subset of the features to work on in terms of feature engineering.
If I just take the whole dataset and calculate the correlation of each feature with the target feature, haven’t i technically introduced some prior knowledge into my process? Should I just calculate correlations between the features using the training set? If yes, what is technically the purpose of doing so since I’m not really evaluating the outcome of correlation analysis on a test set?