I have a theoretical question about creating an artificial feature based off of a binary classification label, and then adding it into my feature set to run my analysis. First, let me show you what I did to the predictor variable to change it, and then let me ask the question/theory behind the results I am seeing.
Copying the label set, this is before I apply any transformations:
array([1, 1, 1, 1, 1, 1, 1, 0, 1, 0])
This is after applying many transformations:
array([ 1.54, 1. , -0.25, -0.42, -0.35, 0.95, 0.73, 0. , -0.35, 0.54])
Correlation between new feature and the label is 0.12.
Prior to this feature, my decision tree (using gini and a max depth of 5) attained a 5-fold CV score of 0.58 and a balanced accuracy score of 0.60.
If I add in this "transformed" feature to my existing feature set, my decision tree model now has a perfect CV score.
Why would that be the case? I am confused as to how this new feature is making the decision tree train perfectly.
