Difference of variable selection and importance estimation

Question

Isn't variable importance estimation a necessary prerequisite for variable selection? Is there any use case where you want to select non-important variables for your model? So, why is variable selection considered different from variable importance estimation?

Answer 1

Isn't variable importance estimation a necessary prerequisite for variable selection?

There's a connection: e.g. excluding predictors with low standardized coefficients during LASSO, excluding predictors with low Wald's chi-squared during a backward stepwise selection. Nevertheless (1) measuring importance in some way or other doesn't tell you where to draw the line between "important" & "unimportant", & (2) the importance of one predictor tends to depend on the coefficients of other predictors in the model, e.g. LASSO paths cross, Wald's chi-squareds change as predictors are removed. So you could perform variable selection by simply using a cut-off on an importance measure, but most methods are more complex. Furthermore some variable-selection methods don't obviously involve any kind of importance measure at the level of individual predictors, e.g. comparing AICs across all possible subsets. So no; it's neither necessary nor sufficient in general.

Is there any use case where you want to select non-important variables for your model?

If you knew which predictors had no relationship at all with the response you wouldn't include them to start with. So any feature selection method introduces bias into estimates & predictions which may or may not be offset by a reduction in their variance (hence the importance of validating the model). Data reduction of the predictors & regularization without shrinking coefficients to zero (e.g. ridge regression) are alternatives to feature selection when the full model over-fits.

It's also perhaps worth noting the common enough case where you're interested in the estimates (point & interval) in their own right, & not just in the overall predictive performance of the model. Clearly setting a coefficient to zero isn't estimating it; less obviously the coefficient estimates of predictors kept in the model will become biased when other predictors are excluded. See Should covariates that are not statistically significant be 'kept in' when creating a model?.