I have a theoretical question about creating an artificial feature based off of a binary classification label, and then adding it into my feature set to run my analysis. First, let me show you what I did to the predictor variable to change it, and then let me ask the question/theory behind the results I am seeing.

Copying the label set, this is before I apply any transformations:

array([1, 1, 1, 1, 1, 1, 1, 0, 1, 0])

This is after applying many transformations:

array([ 1.54,  1.  , -0.25, -0.42, -0.35,  0.95,  0.73,  0.  , -0.35, 0.54])

Correlation between new feature and the label is 0.12.

Prior to this feature, my decision tree (using gini and a max depth of 5) attained a 5-fold CV score of 0.58 and a balanced accuracy score of 0.60.

If I add in this "transformed" feature to my existing feature set, my decision tree model now has a perfect CV score.

Why would that be the case? I am confused as to how this new feature is making the decision tree train perfectly.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.