I have build a model with a Gradient Boosting Machine (GBM) and calculated the feature importance. All features are factors.
Now, I know which features are most important. However, the features have different levels. For example, "dinner" has the levels "true"/"false" where NoiseLevel has the levels "high/average/low". How can I find out which factor level of a given variable is most important?
The data is about restaurants. The goal is to predict the review counts by the attributes of a restaurant. I want to know which attributes a restaurant should offer to get as many review counts as possible i.e. a high demand. So "dinner" alone is not very informative.
I also tried dummy variables. Therefore, every factor level has its own column (e.g. the column "dinner" became "diner-true" and "dinner-false" with the values 0 and 1). However, in this case, I got weird results. For example, "dinner-true" and "dinner-false" was almost equally important in the analyses.
Any Idea how to deal with this problem?
Here is an example of LIME. The goal is to predict "Sepal.Length" by all other features of the iris data set.
library(lime) library(mlr) library(tidyverse) #Train the model iris.task = makeRegrTask(data = iris, target = "Sepal.Length") lrn = makeLearner("regr.gbm", n.trees = 100) mod = train(lrn, iris.task) #Evaluate the model rdesc = makeResampleDesc("CV", iters = 5) set.seed(3) bmr = benchmark(lrn, iris.task, rdesc, measures = list(rmse, rsq)) bmr$results$iris$regr.gbm$aggr rmse.test.rmse rsq.test.mean 0.3535742 0.8168503 #Calculate LIME explainer = lime(iris, mod, quantile_bins = FALSE) explanation = explain(iris[1,], explainer, n_permutations = 5000, n_features = 6) plot_features(explanation, ncol = 1)
For case 1, a have an R2 of 0.54. If I understand the concept correctly case 1 is representative for the whole model through the permutation. So Petal.Length is the most important feature. My question is why is R2 so different from the cross-validation results? Furthermore, I don´t understand why the target variable is in the explanation. This makes no sense to me.
Is there something I missed?