How to interpret the feature importances for 'eli5.show_weights()' for regression?

Question

I am trying to understand how the interpret the values yielded by eli5's show_weights variable after feature importance. I have used this for several regression models, e.g. multiple linear regression, Support Vector Regression, Decision Tree Regression and Random Forest Regression. I am using it to interpret the importance of features for all these models.

This is my code for the Random Forest Regression:

from sklearn.model_selection import RepeatedKFold
import eli5
from eli5.sklearn import PermutationImportance
from eli5 import show_prediction, show_weights

Xfeature_names = X.columns

regressor =  RandomForestRegressor(n_estimators = 100, random_state = 0)

# K-fold cross validation with permutation testing
cv = RepeatedKFold(n_splits=6, n_repeats=100, random_state=1)
perm = PermutationImportance(regressor, cv = cv)
perm.fit(X,y)

show_weights(perm, feature_names = X.columns.tolist())

I then receive this output:

As I read from the eli5 documentation, what show_weights does is

Return an explanation of estimator parameters (weights) as an IPython.display.HTML object. from https://eli5.readthedocs.io/en/latest/autodocs/eli5.html

I read previously that this is an improvement in model performance as measured by r2, but I was not able to find this on the eli5 documentation.

So my question is, how do I interpret the feature weights meaningfully? Could I state based on this table that e.g.

Max span improved model performance as measured by r2 by 0.11 (sd = 0.32), and was therefore the most important contributor to model performance?

Or is there a better way to meaningfully and transparently report the results from the permutation importance testing?

Answer 1

Firstly, the high-level show_weights function is not the best way to report results and importances.

After you've run perm.fit(X,y), your perm object has a number of attributes containing the full results, which are listed in the eli5 reference docs.

perm.feature_importances_ returns the array of mean feature importance for each feature, though unranked - it will be in the order that the features are given in the input data. These will match the data in your show_weights output (the values to the left of the ± symbol).

I would also be more interested in the standard deviation of the permuted results rather than the full ± range given by show_weights. This is accessed with perm.feature_importances_std_

You could do your own ranking by putting these into a dataframe, e.g.

df_fi = pd.DataFrame(dict(feature_names=X.columns.tolist(),
                          feat_imp=perm.feature_importances_, 
                          std=perm.feature_importances_std_,
                          ))
df_fi = df_fi.round(4)
df_fi.sort_values('feat_imp', ascending=False)

You can also access the full results with perm.results_ - this returns an array with the results from each cross-validation for each permutation. In your case, with 6 splits and 100 repeats, there will be 600 arrays, each length of X.columns.

Ok, onto the more important question - what do these results mean?

• Each result in that array of arrays is the change in score when a feature is shuffled to random noise.
• What is the 'score'? eli5.sklearn.PermutationImportance takes a kwarg scoring, where you can give it any scorer object you like. Otherwise I believe it uses the default scoring of the sklearn estimator object, which for RandomForestRegressor is indeed R2.
• So, behind the scenes eli5 has calculated a baseline score with no shuffling. Each shuffle (per feature per cv per permutation) a model is refit and scored. This score is used to calculate a delta, so each 'result' in the array is baseline - score.

Positive vs Negative feature importances?

Somewhat confusingly, positive results indicate that:

• the score got worse when the feature was removed (i.e. score decreased)
• therefore the feature has some importance to the accuracy of the model

A negative result means the accuracy actually improved relative to the baseline when the feature was removed. This could occur for various reasons. It is definitely a good idea to remove features with negative feature importances.

So it's a change in score relative to the baseline...

You can calculate your own baseline with

from sklearn.model_selection import cross_val_score

baseline_cv_results = cross_val_score(regressor, X, y, cv=6)  # use the same number of cvs
baseline_score = baseline_cv_results.mean()

You could then, for example, scale the feature importance results in the example df_fi above with df_fi['percent_change'] = ((df_fi['feat_imp'] / baseline) * 100).round(2) Though it's always important to be careful when scaling scores like this, it can lead to odd behaviour if the denominator is close to zero.

There's several points to consider when interpreting results:

• the features with the largest (positive) feature importance are definitely the most important features.
• features with negative importances are probably confusing your model and should be removed
• features close to zero contain little-to-no useful data
• however, depending on the nature of your data, it may be that the change in score for even the top-ranked feature is small relative to the baseline. This occurs due to interaction with other features.
• if you create a 'percent_change' column as suggested above, you'll find that the percentages probably won't sum to 100%, even if ignoring negative values. Again, this would be due to interactions, where the effect of removing one feature on its own may not be huge, but if more were removed / shuffled at the same time, the model performance could deteriorate non-linearly.

Lastly, I always love a good graph.

Showing the full results as a set of boxplots is a good way to visualise these data. You'll need plotly for this example:

import plotly.express as px
import plotly.graph_objects as go

# create df with columns = each feature, rows = score for each permutation each cv (600 in your case)
df_results = pd.DataFrame(data=perm.results_, columns=X.columns)
# feat_imps values will be same as perm.feature_importances_, but as a pd.Series with index labels corresponding to the feature names
feat_imps = df_results.mean().sort_values(ascending=False)
# reorder columns from most to least important
df_results = df_results[feat_imps.index]
# create boxplots of full results. pd.melt() is a handy way to reformat into a longform dataframe that plays well with plotly express.
fig = px.box(df_results.melt(), x='variable', y='value', orientation='v')
# add a marker showing the mean feature importance for each
fig.add_trace(go.Scatter(x=feat_imps.index, y=feat_imps.values, mode='markers', marker=dict(color='red'), name = 'Mean'))
fig

An example from some of my own data is below: