Variable selection with glmnet (caret package)

Question

Is this a legit way to make a variable/predictor/dummy selection?

(My goal is forecasting with the selected variables)

fit <- train(train.values ~ .,data=train.data, method='glmnet') # train.data includes all variables

#getting the coefficients of the final model
coefficients <- coef(fit$finalModel, fit$bestTune$lambda)

#create a list of the selected coefficients
variables <- names(coefficients[which(Coefficients != 0),])

Due to reading lots of stuff on this platform i am aware of the fact that stepAIC() is not that great of a choice for a variable selection. After the variable selection is done i would use those variables for predictions with glmnet as well as for a linear model.

Answer 1

What you're trying to do here is to identify most "important" variables within glmnet and then trying to pass your features to another model. This is not optimal as Max Kuhn writes here:

In many cases, using these models with built-in feature selection will be more efficient than algorithms where the search routine for the right predictors is external to the model. Built-in feature selection typically couples the predictor search algorithm with the parameter estimation and are usually optimized with a single objective function (e.g. error rates or likelihood).

From theoretical perspective: different models allow for different degree of flexibility and protection against overfit. Why would you use set of optimal parameters from one model in another one?

The only scenario I can think of using glmnet as a feature filter if you are forced to use linear regression and you have a wide matrix of features (more features than cases). There is nothing wrong with this, simply the results (RMSE, R^2) may be suboptimal.

Answer 2

I have read academic papers citing the effectiveness of using Lasso for variable selection as well as actually putting it into practice myself.

The following code block identifies features from your data set.

require(glmnet)
##returns variables from lasso variable selection, use alpha=0 for ridge
ezlasso=function(df,yvar,folds=10,trace=F,alpha=1){
  x<-model.matrix(as.formula(paste(yvar,"~.")),data=df)
  x=x[,-1] ##remove intercept

  glmnet1<-glmnet::cv.glmnet(x=x,y=df[,yvar],type.measure='mse',nfolds=folds,alpha=alpha)

  co<-coef(glmnet1,s = "lambda.1se")
  inds<-which(co!=0)
  variables<-row.names(co)[inds]
  variables<-variables[!(variables %in% '(Intercept)')];
  return( c(yvar,variables));
}

(I cannot take 100% credit for this code as I am sure it is adapted from some place - most likely here: Using LASSO from lars (or glmnet) package in R for variable selection )

While on the topic of variable selection, I have also found that VIF (variable inflation factor) to be effective especially when cross-validated.

require(VIF)
require(cvTools);
#returns selected variables using VIF and kfolds cross validation 
ezvif=function(df,yvar,folds=5,trace=F,ignore=c()){
  df=discard(df,ignore);
  f=cvFolds(nrow(df),K=folds);
  findings=list();
  for(v in names(df)){
    if(v==yvar)next;
    findings[[v]]=0; 
  }
  for(i in 1:folds){   
    if(trace) message("fold ",i);
    rows=f$subsets[f$which!=i] ##leave one out 
    y=df[rows,yvar];
    xdf=df[rows,names(df) != yvar]; #remove output var    
    if(trace) say("trying ",i,yvar,nrow(df),length(y)," subsize=",min(200,floor(nrow(xdf))));
    vifResult=vif(y,xdf,trace=trace,subsize=min(200,floor(nrow(xdf))))
    if(trace) print(names(xdf)[vifResult$select]);
    for(v in names(xdf)[vifResult$select]){
      findings[[v]]=findings[[v]]+1; #vote
    }
  }
  findings=(sort(unlist(findings),decreasing = T))    
  if(trace) print(findings[findings>0]); 
  return( c(yvar,names(findings[findings==findings[1]])) )  
}

Both of the above ez functions return an vector of variable names. The following code block converts the return values to a formula.

#converts ezvif or ezlasso results into formula
ezformula=function(v,operator=' + '){
  return(as.formula(paste(v[1],'~',paste(v[-1],collapse = operator))))
}

I hope this is helpful.