Random forest and LASSO regression both give different variable importances I have a dataset with 163 observations (all countries in the world with population > 1000000) and 290 variables related to their disease burden and performance. Because there are more variables than observations I cannot run a standard linear regression. Therefore I tried both a random forest and a LASSO regression. Both give different variables importances. Which one is the most reliable in this case?
 A: Before going deeper into the comparison make sure that each of the two methods agrees with itself. You can find this out by bootstrapping the entire variable importance process a few times.  Plot the original variable importance for each variable vs. the importance estimated from a bootstrap sample.
The bootstrap involves taking samples of size $n$ with replacement from the original dataset of $n$ observations, and repeating any analysis.  The repetitions have to be "from scratch."  Here is what the process looks like in R:
n <- NROW(mydata)   # mydata = data table, data frame, or matrix
for(i in 1 : 5) {
    s <- sample(1 : n, n, replace=TRUE)
    f <- whateveranalysis(mydata[s, ])
    # Print what you need and look across the 5 bootstraps to
    # see the volatility
}

A: No free lunch. You can't really know it beforehand. Unless you have strong reasons to favor one over the other1 you would have to test both solutions to make a proper choice.

1. Say, you really want a linear model and would feel fine with the implications of choosing that over another possible solution that could be actually better, but then you wouldn't even need to test it in the first place.
