# Why discrepancy between lasso and randomForest?

Following are 2 plots, one of lasso using glmnet package and other 2 from randomForest (variable importance) of the mtcars data set assessing variable mpg vs others. In the lasso plot, the blue and red lines indicate lambda.min and lambda.1se, respectively.

The randomForest plot gives high importance to disp and hp, which are close to 0 almost throughout the plot. Also am is of lowest importance in randomForest, though it has relatively high value in lasso plot.

What could be the reason for these discrepancies? Which one should one accept while determining important predictors of mpg in this dataset?

Edit: Both above plots was without scaling. Following are the plot after all variables (including mpg, the outcome variable) are scaled.

These plots are much more similar (wt, hp, cyl). But disp is still discrepant. It is highest in randomForest but very small in lasso plot.

• Did you standardize the variables? Random forest also takes into account interaction effects depending on the depth of the tree. – spdrnl Jun 2 '15 at 17:54
• @rnso Seconding spdrnl's comment. I'm very interested in whether the glmnet plot is on the standardized scale. It's be worth repeating the experiment after manually standardizing all the predictors. – Matthew Drury Jun 2 '15 at 18:05
• I had a moment's downtime and did a small experiment. The glmnet coefficient plot is not on the standardized scale, pre-standardizing changes the scale of the plot. – Matthew Drury Jun 2 '15 at 18:22
• You should also standardize the binary variables here for a fair comparison between coefficients. – shadowtalker Jun 2 '15 at 18:42
• The two results look remarkably similar. They both make exactly the same division into high- and low-importance variables and even rank them almost the same. Could you explain why you expect two completely different procedures to produce identical results on completely different scales? – whuber Jun 2 '15 at 18:43

So you're comparing apples and oranges. A fair comparison would be to re-fit both models without each variable, and compute the decrease in MSE (i.e. with cross-validation or a train/test split) due to omitting each variable. Or instead of dropping each predictor you could randomly permute it; this how %IndMSE is computed.