Dense variables or sparse variables contribute more on random forest and svm? i have a GC-MS data set containing 256 variables and 150 observations. Of this 256 variables, about 70 variable are very dense and others are sparse. I performed a random forest and support vector machine base recursive feature elimination with caret package. And then I found almost all the important selected variables are dense variables, the sparse variables which are biological important seems skipped by rf and svm. Then I tried to replace all the 0 value with a super small value (Limit of detection), then it seems better than before but still dominant by dense variables. 
So, I want to know does that mean dense variables contribute more on the rf and svm? and how to solve the density-driving bias on the models?
Thank you very much.
 A: Think about how a decision tree works (random forest).  Imagine you have a sparse variable in which only 10 out of 1000 observations have a value of 1 and the rest have 0.  Now lets say you have the beginning node with 1000 observations.  If you were to then split on this sparse variable, you could move at most 10 observations into a new node.  The other node would still contain (at least) 990 observations and look quite similar to the node with 1000 observations.  Thus this decrease in impurity this would cause (how importance is measured for random forests) would be quite small.
A: I think this is a general property of sparse features, not just these models. A similar thing surprised me in my own research, when I had some features that were both very rare and very predictive compared to the others, and the math for my model showed the rare ones to be fairly worthless. For the rare occasions when you're classifying an instance that has that feature, the feature is amazing! But overall, on average across all features, it's not as helpful as those that are relevant for all instances.
I'd imagine this can be formalized in ways that would depend on the classifier's optimization function and/or loss function. But for hand-wavy intuition, I think it's similar to the entropy of a Bernoulli variable, in which p = P(feature = 1) and -log(p) = how strongly correlated the feature is with the class label. With sparse features, there's a small chance of a lot of info for classification, but overall a variable is more informative if it gives a small amount of information every time.
