# How to measure variable importance in a GAM model?

For concreteness:

library( mgcv )
set.seed( 1 )
RawData <- data.frame( y = rbinom( 1000, 1, 0.5 ), x1 = rnorm( 1000 ),
x2 = as.factor( rbinom( 1000, 1, 0.5 ) ), x3 = rnorm( 1000 ),
x4 = as.factor( rbinom( 1000, 1, 0.5 ) ) )
fit <- gam( y ~ s( x1 ) + x2 + s( x3, by = x2 ) + x4, data = RawData,
family = nb( link = log ) )


How to measure the importance of these four variables?

I understand that "variable importance" is not a well-defined concept, so I am looking for the most straightforward way, such as an explained variance approach.

The ANOVA table seems to be a natural choice, however, as explained in this answer, it is not working: for the smooth terms in GAM models they do not have an explained variance interpretation.

What is the sound approach then?

• What is importance here? Statistical significance? Commented Nov 15, 2017 at 22:53
• Many of these techniques in statistical learning use other metrics for variable importance; change in MSE when included in a random forest model bag sample, R2, or GCV score in MARS models. As there isn't a clear statistic that measures this that is routinely output by mgcv, it would help to think how might you otherwise measure variable importance, even conceptually or at a high level. This might indicate how to proceed with computing something you can use practically to answer your question. Commented Nov 17, 2017 at 18:14
• @GavinSimpson Sure, I happily share you my original problem that motivated this question. Let x3 be the age of a patient, x2 be the sex, x1 be his/her blood pressure, x4 be whether he/she received a certain drug, and y be the number of times a certain event happened to him/her. The question the doctors ask: "OK, I understand blood pressure is significant, but we have a very high sample size, so it doesn't mean a lot, I'd be more interested to see how it compares to the other predictors in explaining y". Commented Nov 18, 2017 at 14:59
• Yeah, I get that, but for you, what would be a good a good way to measure "explaining y"? Would a loss function work, and you could compare the Poisson loss of models with and without a particular covariate? You'd need to decide whether to reevaluate the other smooths or keep them fixed at their smoothness parameter estimates from the full model? Commented Nov 18, 2017 at 16:33
• @Gavin Simpson : Very, very good questions. I'd really appreciate to read a review of the possibilities, as honestly, I've no idea what would work, and what decisions are meaningful in the scenario I described. Commented Nov 18, 2017 at 22:52