10
$\begingroup$

When I compare on Permutation Feature Importance (PFI) on Train vs Validation set, some features has high values (of PFI) for train but the low values (PFI) for validation.

One the conclusion, for me, that features : c, d, f, g and i seams to be kind of noise or "label leakage".

Is there any additional conclusions?

The bar plots of permutation feature importance:

Permutation Feature Importance on Train set

Permutation Feature Importance on Train

Permutation Feature Importance on Validation set

Permutation Feature Importance on Test

N.B. The task is Regression (Forecasting) which is done by Random Forest Regression.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ One doesn’t do feature importance on the test set. To check the importance either you use the full dataset (not suggested) or you do it by cross validation. $\endgroup$
    – user289381
    Commented Jul 5, 2020 at 9:47
  • $\begingroup$ Absolutely agree regarding CV. What would be conclusions if instead of test set it was done on validation set? I also updated the question accordingly. $\endgroup$
    – Michael D
    Commented Jul 5, 2020 at 11:08
  • 1
    $\begingroup$ You don’t use out-of-sample data for anything but testing the model you trained on training data. Split up your data, and then ignore the non-training data while you construct your model. $\endgroup$
    – Dave
    Commented Jul 5, 2020 at 12:13
  • $\begingroup$ @Dave, in my understanding, this is exactly what I did. Model is trained on training data only. The "second" plot is on validation set. Please let me know if I miss something. $\endgroup$
    – Michael D
    Commented Jul 5, 2020 at 12:47
  • 1
    $\begingroup$ Then I don’t understand your question about using a validation set instead of a test set. $\endgroup$
    – Dave
    Commented Jul 5, 2020 at 12:54

3 Answers 3

Reset to default
9
$\begingroup$

The difference in the observed importance of some features when running the feature importance algorithm on Train and Test sets might indicate a tendency of the model to overfit using these features. This is indeed closely related to your intuition on the noise issue.

In other words, your model is over-tuned w.r.t features c,d,f,g,I.

Running feature importance on train or test sets hasn't been addressed enough in literature. However, a good, albeit simple, analysis of this issue is provided in this blogpost.

Improve this answer
$\endgroup$
5
$\begingroup$

First: ignore the results you have for the training set, they are worthless. Who cares how good a feature is at predicting for records that built the model?

Second: At this point you can’t do anything with features c,d,f,g. You’ve already built your model, yes these features are not useful in predicting the values for your test set, but if you were to remove them now, your test set would become part of the training process and you would be without a test set. All you can use this for at this point is to know that these features are uninformative for your model. To remove them and retrain the model would be akin to stepwise regression which we all know is bad.

If you wanted to use this measure in order to select features and improve your model I believe something like this would work: split your data into train/validation/test. Then split train into train/test, call it train2 and test2 I guess. Build a model on train2 and test the feature importance on test2. You could then remove any irrelevant features. Now continue as usual on train/test/validation as usual.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Why to slit train into train-1 and train-2? Why just using the validation set is not enough? $\endgroup$
    – Michael D
    Commented Jul 8, 2020 at 7:19
  • 1
    $\begingroup$ When you are doing feature selection and your model uses a training/validation/test split, you want to do the feature selection on the training set so that your validation set remains unseen and you can do hyper-parameter selection on it . However, using the permutation importance for feature selection requires that you have a validation or test set so that you can calculate the importance on unseen data. In order to achieve that you need to split your training set again. $\endgroup$
    – astel
    Commented Jul 8, 2020 at 16:09
5
+50
$\begingroup$

Well, let's think at what those numbers actually mean.

If we take feature g for example, we know that our model relies on it a bit. So much that if you shuffle its values when making predictions on train data, your model performance drops by around $0.002$ (if I am correct).
However, when we do the same thing and shuffle it before predicting unseen data, the model performace is on average unchanged, which means that the feature has no predictive power on your target, and that the importance it has with training data comes from using some pattern of your training data that does not generalize (aka, you are overfitting).

Random forests tend to build very deep trees (possibly, up to a point where no split is possible). This means that even if you have a feature that is just white noise some trees will end up using it for splits at some point because they will see some pattern in it. This means that overall, it is likely even for noise features to have a positive permutation importance on the training data - and this is why the permutation importance you should really care about is on you validation set!

Now, what should you do next? Random Forests are somewhat resistant to this kind of overfitting, and having a few variables that contain only noise is not too detrimental to the overall performance, as long as their relative importance (on the training data) is not excessive, and there is not too many of them.
However, as you now know that those features are useless for your regression (and in general as good practice), the best option would be to remove them and retrain your model.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Your suggestion of removing the features from the model is bad practice, see my answer for why. $\endgroup$
    – astel
    Commented Jul 7, 2020 at 16:52
  • 1
    $\begingroup$ He’s edited the question a bunch. Originally he said test set. But either way, I still don’t think it is appropriate to retrain your model after gleaning information from the validation set. $\endgroup$
    – astel
    Commented Jul 7, 2020 at 17:30
  • 2
    $\begingroup$ @astel that's what validation sets are made for. You train on your training test, and do model selection/hyperparameter search on the validation set (or via CV if you prefer). The important thing is not using the test set. $\endgroup$
    – Davide ND
    Commented Jul 8, 2020 at 8:23
  • 2
    $\begingroup$ How is he supposed to do hyper-parameter selection on the validation set when the validation set was already used to do feature selection? In order for hyper-parameter selection to be done correctly the validation set needs to be independent of the other processes. The way you are suggesting he do things, it is not $\endgroup$
    – astel
    Commented Jul 8, 2020 at 15:56
  • 2
    $\begingroup$ In an ideal situation you have infinite data and can take separate sets for hyperparameter optimization and for model selection. In practice, either you do one of the two using crossvalidation or OOB scores, or you do both on the validation set. It's not ideal, but it's not as bad as looking at the train set, or the test one. $\endgroup$
    – Davide ND
    Commented Jul 8, 2020 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.